OpenAI Fixes ChatGPT Agent Flaw That Could Let Attackers Forge an AI Insider

Zenity Labs has found and disclosed a critical vulnerability in OpenAI’s ChatGPT Workspace Agents, which it names AgentForger; a tailored cross-site request forgery (CSRF). With a single successful phish, an unsuspecting employee could be tricked into launching an invisible autonomous agent that is remotely controlled by the attacker.

Zenity explains in two blogs (Part 1 and Part 2) that the vulnerability was in ChatGPT’s Agent Builder allowing an over permissive parameter. Researchers found that the official agent build process could be usurped from within an initialization URL using two particular parameters. One names the agent template to be used, while the other (initial_assistant_prompt) provides instructions to the Builder. The first parameter, using the Chief of Staff template, builds a more powerful and flexible agent than other templates. The latter contains instructions that are automatically submitted and executed. More specifically, the ‘initial prompt’ can become the first command the Builder acts on.

With these two parameters embedded in the URL, the attacker can generate a powerful agent with prespecified instructions. One of the instructions exploiting this process is to automatically accept emails from the attacker as new instructions, allowing the attacker to control the agent remotely.