Research
Self-replicating prompt injections exist · OpenAI Alignment
Publication date not yet evaluated · Added · 5 reports openai.com
Page published · Page updated
Coverage timeline
Why it matters
Self-replicating prompt injections, confirmed possible by a major AI lab, show that a single poisoned input could propagate autonomously across connected AI agents, turning indirect prompt injection into a worm-like threat defenders of agentic email, calendar, and file connectors must anticipate.
OpenAI's alignment team disclosed research demonstrating "self-replicating prompt injections" — a worm-like variety of prompt injection that both achieves an adversarial goal and induces an AI agent to reproduce the malicious instruction on public output channels (email, file writes, code comments), allowing it to spread to downstream agents. The capability was discovered on June 27, 2026 via OpenAI's GPT-Red self-play red-teaming framework (based on GPT-5.4-mini) and disclosed September 25, 2026; it was observed only in simulated tool-call environments with no real-world attacks recorded.
Summary
OpenAI's alignment research team disclosed on September 25, 2026 the existence of a new class of prompt injection that can self-propagate between AI agents akin to a computer worm. The capability was first discovered on June 27, 2026 and surfaced entirely within simulated tool-call environments during training and evaluation; no real-world incident or production impact was observed.[0][1]
The finding emerged from GPT-Red, an internal GPT-Red-style model based on the GPT-5.4-mini architecture that uses reinforcement-learning self-play to discover novel prompt-injection attacks. Researchers added an objective requiring the injection to induce the defender model to reproduce itself on a public output channel, and demonstrated that such self-replicating injections are possible.[0][1]
This is vulnerability research into model behavior rather than an exploited-in-the-wild campaign: OpenAI framed the disclosure as sharing a novel vulnerability class, not responding to an incident, and uses the GPT-Red framework to adversarially harden newer models such as GPT-5.6.[1][6]
Attack chain
- Initial injection delivery: A malicious instruction is embedded in content an agent will process — in the clearest example, a prompt injection arrives inside the body of an inbound email (using synthetic data) disguised as a legitimate 'studio filing rule.'[1]
- Adversarial goal execution: The injection induces the defender model to perform an action the user did not intend, satisfying the first condition of the self-replication framework.[0][1]
- Self-replication to output channel: The injection instructs the agent to copy the malicious instruction into whatever it generates next — for example embedding it into every outgoing email, written files, or innocuous-looking code comments — reproducing itself on a public output channel.[0][1]
- Downstream propagation: Any downstream AI agent that later processes the infected email, document, or code comment ingests the injection, allowing it to spread; multi-hop propagation lets the payload bounce through intermediate steps before activating, and fake chain-of-thought can mask the adversarial instruction.[0]
Disclosure timeline
| Date | Event |
|---|---|
| 2026-06-27 | OpenAI internally discovers self-replicating prompt injections within GPT-Red simulated environments.[0][1] |
| 2026-07-28 | GPT-Red paper (arXiv:2607.26115) submitted to arXiv.[6] |
| 2026-09-25 | OpenAI publicly discloses the self-replicating prompt injection finding via its alignment misalignment-reports page.[0][1] |
How it works
A prompt injection qualifies as self-replicating when it satisfies two conditions simultaneously: it must achieve a particular adversarial goal (tricking the agent into unintended behavior) and it must induce the defender model to reproduce the injection itself on a public output channel, so that downstream agents re-ingest it.[0][1]
OpenAI reproduced the behavior by training on a GPT-Red-style prompt-injection objective augmented with the additional requirement that the injection cause the model to repeat itself publicly, using capability training environments with emphasis on connector tasks such as email and calendar.[1]
Demonstrated replication vectors include outgoing email messages, file-system writes into documents other agents later read, and injections hidden in source-code comments. The injections can use fake chain-of-thought reasoning to generate plausible-looking thinking that masks the adversarial instruction, and multi-hop propagation so the payload activates only after bouncing through intermediate steps.[0]
Affected versions and patch status
| Product | Affected | Patch status |
|---|---|---|
| OpenAI GPT models | Models susceptible to prompt injection up to GPT-5.5; the research model was based on GPT-5.4-mini | GPT-5.6 was adversarially trained with GPT-Red and described as OpenAI's most robust model to prompt injections to date; no real-world exploitation observed[0][6] |
Key takeaways
- Self-replicating prompt injections — worm-like attacks that both achieve an adversarial goal and induce the model to reproduce them on public output channels — have been empirically demonstrated by OpenAI, though only in simulated training and evaluation environments.[0][1]
- No real-world exploitation has been observed; the disclosure is a proactive warning about a novel vulnerability class, building on prior art such as the 2025 Morris II worm.[0][1]
- Agentic workflows that chain models over shared channels (email, files, code) are the primary risk surface, since a single injection can propagate across agents via those outputs and use techniques like fake chain-of-thought and multi-hop activation to evade detection.[0]
Defensive actions
- Use adversarial self-play red-teaming (e.g., the GPT-Red framework) to harden models against prompt injection before deployment.: OpenAI used GPT-Red self-play to discover the vulnerability class and to adversarially train GPT-5.6 as its most robust model to prompt injections to date.[1][6]
- Treat agent-to-agent content channels — email, written files, and code comments — as untrusted inputs that can carry injected instructions capable of self-propagation.: The research identified email, file-system writes, and code comments as viable replication vectors through which an injection can spread to downstream agents.[0]