Topic · Latest 50 of 53 active items

Prompt Injection

Items tagged for indirect prompt injection, instruction smuggling, retrieval poisoning, or related control-plane failures.

Matching wire items

AWS Kiro: Arbitrary Code Execution via Indirect Prompt Injection

AWS Kiro, an agentic coding IDE, was vulnerable to arbitrary command execution via indirect prompt injection: hidden text on a web page (or a comment in a source file) processed by the agent could make Kiro use its no-approval fsWrite tool to rewrite ~/.kiro/settings/mcp.json (or allowlist all Bash commands in .vscode/settings.json), causing it to launch attacker-specified MCP servers/commands and achieve RCE on the developer's machine, bypassing the human 'allow' approval boundary. Discovered by Intezer with Kodem Security and independently by Embrace The Red (Johann Rehberger); AWS has patched the issue. Details →

Indirect Prompt Injection Targets AI Agents | ThreatLabz

Zscaler ThreatLabz observed two in-the-wild campaigns using indirect prompt injection (IPI) embedded in malicious websites to manipulate web-browsing AI agents. One campaign uses SEO poisoning around a fake Python library (requests-secure-v2) with hidden prompts and schema-markup instructions telling agents to make a cryptocurrency payment to a hardcoded wallet to obtain an API key; the other typosquats the DeFi tracker DeBank, embedding prompts convincing agents the fraudulent site is legitimate. In testing an autonomous payment-capable agent across 26 LLMs, four (Llama 3.3 70B, Llama 3.2 90B Vision, Gemini 3 Flash, Gemini 2.5 Pro) were tricked into paying and two miscategorized the fake DeBank as trusted. Details →

GhostApproval: AI Coding Assistant Trust Boundary Flaw

Wiz researchers disclosed GhostApproval (CVE-2026-12958 and related), a symlink-following vulnerability pattern (CWE-61 plus CWE-451 UI misrepresentation) affecting six top AI coding assistants: Amazon Q Developer, Anthropic Claude Code, Augment, Cursor, Google Antigravity, and Windsurf. A malicious repository disguises a symlink as an innocuous config file so that when the agent 'sets up the workspace' it writes an attacker SSH key to ~/.ssh/authorized_keys — sometimes before any confirmation dialog — while the human-in-the-loop approval prompt conceals the true target, enabling remote code execution. AWS, Cursor, and Google fixed the issue; two vendors went silent and one called it outside its threat model. Details →

We put the exploit in a picture. Your AI code reviewer never opens it.

Researchers from the University of Missouri-Kansas City's ASSET Research Group demonstrated 'Ghostcommit,' an attack that hides malicious prompt-injection instructions inside a PNG image so AI code reviewers (CodeRabbit, Cursor Bugbot) never see them. A benign-looking AGENTS.md convention file points to build-spec.png, whose rendered text instructs a coding agent to read the repo's .env byte-by-byte and emit the secrets as an integer tuple; the payload sits dormant until an unrelated agent session triggers exfiltration. A proof-of-concept is published on GitHub and the findings were disclosed to affected vendors. Details →

Capability Gates Are Not Authorization: Confused-Deputy Failures in LLM Agent Frameworks

A security research paper, 'Capability Gates Are Not Authorization,' audits LangChain/LangGraph, LlamaIndex, and the Stripe Agent Toolkit and finds that all three provide capability gating by default but none enforce a deterministic fail-closed per-call value authorization gate, enabling classic confused-deputy abuse where an attacker-influenced model emits an unauthorized side-effecting call (e.g., a payout). The authors introduce SCOPEGATE, a five-stage PDP/PEP control (scope, authorization, money ceiling, idempotency, default deny), and report an identical unauthorized payout executing under LangChain's default dispatch but denied by SCOPEGATE, with an available artifact (github.com/raceksd-source/scopegate-runtime). Details →

Agent Data Injection Attacks are Realistic Threats to AI Agents

A research paper by Woohyuk Choi and colleagues introduces agent data injection attacks (ADI), a new category of indirect prompt injection in which malicious data is disguised as trusted data (such as security-critical metadata or agent context data like tool call/response formats) rather than as instructions. The authors demonstrate ADI against real-world agents, achieving arbitrary click attacks on web agents (Claude in Chrome, Antigravity, Nanobrowser) and remote code execution plus supply-chain attacks on coding agents (Claude Code, Codex, Gemini CLI), showing it bypasses existing IPI defenses because agents fail to isolate trusted from untrusted data. Details →

Prompt Injection as Role Confusion

The paper "Prompt Injection as Role Confusion" (arXiv:2603.12277, ICML 2026) by Charles Ye, Jasmine Cui, and Dylan Hadfield-Menell traces prompt injection to role confusion: LLMs perceive the source of text from how it sounds rather than its labeled role, so injected text occupies the same representational space as the trusted role it imitates. The authors introduce role probes to measure internal role perception and demonstrate CoT Forgery, a zero-shot attack injecting fabricated reasoning into user prompts and tool outputs that yields 60% attack success against frontier models with near-zero baselines. Details →

The Injection Paradox: Brand-Level Suppression in Safety-Trained LLM Recommendations via RAG Context Injection

The paper 'The Injection Paradox' by Hyunseok Paeng documents a reproducible failure mode in RAG-based LLM recommendation systems where indirect prompt injections embedded in retrieved documents backfire, suppressing the injected brand below its injection-free baseline. In safety-trained Claude models (e.g., Claude Opus 4.6), a single injected document drops the target brand from a 54% baseline to zero top-2 recommendations and propagates suppression to the brand's uninjected documents, while GPT models instead show increased recommendations. The authors note this enables a reverse-attack scenario where an adversary injects a competitor's documents to suppress that competitor, and release code, prompts, and results. Details →

Devil in the Lens: Analyzing and Defending Physical Prompt Injection Against Vision-Language Models on Wearable Devices

A research paper, "Devil in the Lens," characterizes physical prompt injection attacks against Vision-Language Models on wearable devices such as AI glasses, where malicious text embedded in real-world environments acts as an indirect prompt-injection channel that hijacks model behavior. Using photos captured from AI glasses across 200+ environments, the authors identify 6 threat vectors, evaluate 12 VLMs (achieving attack success rates up to 96% simulated and 60% real-world), and propose two defenses: a mask-based external filter and a semantic-vector-based internal detector. Details →

Solving GitHub’s Secure Code game with an AI red teaming agent

Adversa AI documents pointing its autonomous AI Red Teaming Agent at GitHub's open-source Secure Code Game 'ProdBot' challenge, an intentionally vulnerable agentic-AI teaching target where the goal is to coax the agent into leaking a flag stored one directory above its sandbox. The agent cleared the first three levels (Sandbox, Web, MCP) at full score in a single 57-second run, relying on 'context seeding' — fabricating a plausible prior workflow implying authentication had already occurred — rather than overt jailbreak language, illustrating that each added capability (web browsing, MCP tool chains, skills, multi-agent) opens a new attack surface via indirect injection, poisoned tool chains, and confused-deputy trust. Details →

One-Click Data Exfiltration via rovoChatPrompt URL Parameter (Confluence / Rovo) - CrowdStream

A disclosed and now-patched vulnerability in Atlassian Rovo, the default AI assistant across Confluence and other Atlassian products, allowed the `rovoChatPrompt` URL parameter to preload an arbitrary prompt into a victim's Rovo chat. When an authenticated user clicked a crafted link, Rovo executed the embedded prompt as a genuine query, using indirect task-framed language ('help me identify this image') to bypass guardrails and exfiltrate Confluence pages, secrets, and connected-app data (Jira, SharePoint, Outlook) to an attacker host via an image-fetch URL. Atlassian remediated the issue server-side and the reporter validated the fix. Details →

Data Exfiltration via Markdown Images: The Quiet AI Vulnerability

Wraith's attack guide by Anthony D'Onofrio details data exfiltration via markdown image rendering in AI products, where an injected payload (via prompt injection, RAG doc, shared document, or email) causes an LLM to emit a markdown image whose URL embeds secrets like system prompts, conversation history, or API keys; when the chat client renders the markdown, the browser silently fetches the attacker's URL, leaking the data with zero clicks and no visible artifact. The guide notes variants have hit ChatGPT, Microsoft Copilot, GitHub Copilot Chat, Slack AI Assistant, Google Bard, and Claude.ai, and covers four defensive patterns. Details →

When Context Bites: Detecting RAG Poisoning via Document-Level Attention Collapse | Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval

A SIGIR 2026 paper introduces D-SCAN (Document-level Signal Collapse Analysis), a lightweight detection framework that identifies RAG poisoning attacks by monitoring 'Attention Collapse' — a drop in attention entropy as the generator concentrates on injected adversarial documents. The authors show that output-side signals like perplexity fail because poisoned outputs can exhibit false confidence with even lower perplexity than benign ones, and that D-SCAN detects attacks even when they do not alter the final answer. Code is available at github.com/yingtaoren/D-Scan. Details →

DualView: Preventing Indirect Prompt Injection in Personal AI Agents

DualView is a defense presented in an arXiv paper that extends untrusted-data tracking from a personal AI agent's context out into the user's environment (file system, shell, network, and other agents) to block indirect prompt injection, including a newly-highlighted 'stored IPI' variant where an agent saves attacker-controlled data and later rereads it as trusted. Deployed as an OpenClaw plugin using only tool hooks, DualView gives each channel an AgentView (untrusted data shown as symbols) and a HumanView (original data), and reportedly blocked every IPI attack on an IPI benchmark and PinchBench while preserving utility. Details →

MUZZLE: Adaptive Agentic Red-Teaming of Web Agents Against Indirect Prompt Injection Attacks

MUZZLE is an automated agentic red-teaming framework, presented in a USENIX Security 2026 paper by researchers from Northeastern University and Mozilla, for evaluating the security of LLM-based web agents against indirect prompt injection attacks embedded in untrusted web content. Muzzle uses agent trajectories to identify high-salience injection surfaces and adaptively generates context-aware malicious instructions, discovering 44 new attacks across 4 web applications, including 3 cross-application prompt injection attacks and an agent-tailored phishing scenario. Details →

Prompt injection protection in Microsoft Defender for Office 365 - Microsoft Defender for Office 365 | Microsoft Learn

Microsoft documentation describes prompt injection protection built into Microsoft Defender for Office 365, which detects attacker-authored instructions embedded in inbound email (body, subject, quoted replies, attachments, or hidden markup) before that content reaches a user or an AI assistant such as Microsoft 365 Copilot. The feature explains indirect prompt injection techniques including direct instructions to the model and hidden/invisible text (white-on-white fonts, zero-size text, HTML/CSS tricks) and detects them as part of existing mail-flow inspection. Details →

DualView: Preventing Indirect Prompt Injection in Personal AI Agents

DualView is a research system from CompSec Lab @ SNU (arXiv:2607.03821) that defends personal AI agents like OpenClaw against indirect prompt injection, including a variant called 'stored IPI' where attacker-controlled text persists in the user's file system, shell, or network and is re-read later as trusted data. DualView extends untrusted-data tracking from the agent context into the user's environment by maintaining two views (AgentView shows untrusted data as symbols; HumanView preserves originals) and deploys as an OpenClaw plugin using tool hooks; in evaluation on an IPI benchmark and PinchBench it blocked every IPI attack while keeping utility near the unprotected baseline. Details →