Research
Prompt Injection as Role Confusion
Publication date not yet evaluated · Added arxiv.org
Page published
Coverage timeline
Single-source research — one report is available.
Why it matters
The role-confusion finding shows that tag-enforced instruction hierarchies, the foundation of current LLM security, do not survive into model internal representations, explaining why prompt injection defenses fail and why agents can be hijacked by injected webpage or tool-output text.
The paper "Prompt Injection as Role Confusion" (arXiv:2603.12277) argues that LLMs infer a text's role from stylistic cues rather than its labeled role tags, so untrusted text that imitates a trusted role inherits that role's authority. The authors introduce "role probes" to measure internal role perception and demonstrate CoT Forgery, a zero-shot attack that injects fabricated reasoning into user prompts and tool outputs, achieving ~60% attack success on frontier models against near-zero baselines.