Topic · Latest 50 of 211 active items

Prompt Injection

Items tagged for indirect prompt injection, instruction smuggling, retrieval poisoning, or related control-plane failures.

Matching wire items

Anthropic 'SneakPeek' Flaw Is a Silent Zero-Click Threat

The 'SneakPeek' vulnerability in Anthropic's Claude Code Desktop lets adversaries silently exfiltrate sensitive developer files — including SSH private keys, AWS credentials, and Claude OAuth tokens — via the automated HTML preview pane with no prompt or click. Noma Labs chained three flaws (missing content security policy, unrestricted origin access, and inactive request filtering) to bypass all sandbox protections; an attacker could plant a public repo with an HTML file padded past 256KB containing a malicious script. Anthropic's initial fix was bypassed by Noma, prompting deeper architectural fixes. Details →

Technique library

Basanos publishes a "Technique library" cataloging named AI attack primitives — including policy puppetry, many-shot, skeleton key, adversarial suffixes, multilingual jailbreaks, Crescendo, Echo Chamber, function-calling jailbreaks, indirect prompt injection, RAG chunk-boundary splitting, poisoned tool descriptors, and markdown image exfiltration — each explained with mechanism, testing, and defense, citing published research where applicable. The page emphasizes out-of-band verification (Ingested → Triggered → Exfiltrated) as the standard for demonstrating AI-security findings, and states it publishes mechanisms and defenses but never working payloads. Details →

Who Is Your Agent Serving? Provider-Side Indirect Prompt Injection in Proactive Agents

The paper "Who Is Your Agent Serving? Provider-Side Indirect Prompt Injection in Proactive Agents" demonstrates that an external content provider can redirect a benign proactive personal agent toward its own objectives without accessing private user context, compromising the agent, or gaining extra permissions. Through techniques the authors call Target Control, Private Binding, and Prospective Support, the attack increased target authorization across all tested environment-user-model combinations, with a macro gain of up to 77.4 percentage points across three agent environments and six simulated user models. Details →

Visual Studio Copilot and SSMS Remote Code Execution (CVE-2026-103572)

CVE-2026-103572 is a path-validation-bypass vulnerability in the shared Microsoft.VisualStudio.Copilot.Core.dll used by Visual Studio Copilot and SQL Server Management Studio, reported by embracethered and patched by Microsoft (rated Important, Remote Code Execution). A trailing-dot filename trick (e.g. `.mcp.json.`) bypasses the check meant to stop Copilot from writing to its own settings and MCP configuration files; because Win32 strips the trailing dot, the write lands on the protected file, letting the AI agent escape containment and achieve arbitrary command execution. The researcher demonstrated exploitation by a third party via indirect prompt injection. Details →

A Review of Indirect Prompt Injection and Its Defenses Using Large-Scale Public Red-Teaming Competition Data

A research paper by authors from Georgia Tech and GraySwan.ai evaluates 12 published indirect prompt injection (IPI) defenses against 13k+ unique human-authored IPI attacks and 26k+ successful break trajectories from public red-teaming competitions. The study finds that most defenses fail to generalize, four of six classifier defenses fall short of published precision/recall, and system-level designs like Plan-Then-Execute, Code-Then-Execute, and Dual-LLM generalize best (one cutting attack success from 61.9% to 1.6%) but still leave residual attack surfaces and reduce utility. Details →

Mitigating Prompt Injection Attacks in Large Language Model Applications: A Defense-In-Depth Framework for Secure Generative AI Deployment and Governance by Mohammad Shaidujjaman :: SSRN

A research paper by Mohammad Shaidujjaman proposes a Defense-in-Depth Prompt Injection Mitigation Framework for LLM applications, combining layered controls such as input sanitization, instruction-context separation, trust-aware retrieval, privilege restriction, tool authorization, runtime monitoring, and output validation. The study evaluates whether coordinated defenses outperform standalone controls using metrics like attack success rate, false positive rate, and utility degradation across direct/indirect prompt injection, jailbreaks, and data-exfiltration scenarios. Details →

After GHOSTJACKING: What Agent Security Hardening Actually Requires

Forkast analyzes the fallout from GHOSTJACKING, an indirect prompt injection attack against AI agents formalized in an ngCERT advisory (October 6, 2026) that recommends nine structural hardening steps such as deny-by-default egress, human-in-the-loop approval, and least-privilege MCP servers. The piece argues enterprise inertia—low adoption of defensive configs like Tenet Security's agent-jackstop, lack of formal AI policy, and shadow-agent sprawl—is a governance failure, citing Tenet's Agentjacking demo that hijacked over 100 agents at a Fortune 100 company via injectable Sentry DSNs. Details →

ngCERT GHOSTJACKING Advisory Formalizes Agentjacking — MCP Integrations Now a Government-Level Threat

Nigeria's ngCERT issued an October 6, 2026 advisory formalizing 'Ghostjacking' (also called Agentjacking) — an indirect prompt injection against AI agents via Model Context Protocol (MCP) integrations such as Cloudflare, Datadog, and Sentry, where agents treat retrieved external data as trusted instructions and execute in-scope actions like code execution and cloud credential theft that evade EDR, WAF, and IAM. Tenet Security researchers demonstrated 85-90% exploitation success rates against Claude Code, Cursor, and Codex (notably via Sentry DSN injection), audited 2,388 organizations with injectable public Sentry DSNs, and released an open-source hardening tool, agent-jackstop; Anthropic patched a separate Claude Desktop egress-sandbox JWT reuse zero-day before DEF CON 34. Details →

GitHub Copilot CLI vulnerability leaks developer secrets

Adversa researchers (Rony Utevsky) demonstrated Cryptographic Context Injection (CCI) against GitHub Copilot CLI: a single attacker-controlled web page delivers malicious instructions as ciphertext plus key material, which the agent decrypts inside its own shell and then trusts as its own commands. In autopilot mode this makes the agent read local files (including .env secrets outside the working directory) and exfiltrate them to an attacker endpoint, while identical plaintext instructions are caught and refused. GitHub validated the finding but declined to treat it as a vulnerability, and the chain still reproduced as of October 1, 2026. Details →

From A2A Attacks to Envelope-Layer Defense: Red-Teaming Evaluation of LLM Agents and a Three-Layer Isomorphic Attack–Defense Model

A research paper introduces A2A-TIBA, an attack combining indirect prompt injection with bypass circumvention against LLM agents communicating over Agent-to-Agent (A2A) and ACP protocols, using implant–command–exfiltration steps to deploy a callback program and issue commands bypassing the agent layer. The authors also propose GDA Measurement (a red-team testbed with LLM-gateway context capture and agent-based judging), an extended taxonomy of four attack outcomes, and ELA-ITL, a three-layer isomorphic attack–defense model, validated across 15 agent front-end × LLM back-end combinations and a 1,000-case dataset. Details →

Render Before Reading: Visual Rendering as a Prompt Injection Defense

Researchers from ETH Zurich and Leiden University present Pictionary, a training-free defense against prompt injection that exploits a 'modality gap' in multimodal LLMs: models follow adversarial instructions far more readily as text than when the same payload is delivered as an image or audio. By rendering all untrusted payloads as typographic images before they reach the model, the authors show consistent reductions in attack success across ten models and two benchmarks (DirectInject and AgentDojo), even against adaptive attacks and human red teamers, while preserving benign utility. Code is published at github.com/zj-jayzhang/Pictionary. Details →

Mistral Vibe Permission Bypass and Arbitrary Code Execution

SecMate disclosed two flaws in Mistral Vibe 2.25.0 (CVE-2026-87987 and CVE-2026-87984) where the coding agent's Bash tool used Tree-sitter to extract a reduced representation of a shell command for its permission allowlist, then executed the original untouched string. Hostile repository content could lead the model to emit a crafted tool call (e.g. an environment assignment prefixing an allowlisted command) that the permission parser classified as read-only, bypassing the approval prompt and enabling arbitrary code execution and reads/writes outside the authorized workspace. Mistral released Vibe 2.25.4 on September 12, 2026. Details →

AgentBreaker: Evaluating Context-Aware Indirect Prompt Injection Risks in Modern Web Agents | Proceedings of the ACM on Software Engineering

AgentBreaker is an indirect prompt injection (IPI) attack framework presented in an ACM research paper that autonomously composes context-aware adversarial phrases embedded as DOM/HTML elements to manipulate LLM-powered web agents into clicking attacker-designated elements, posting attacker text, and disclosing internal agent secrets. Evaluated against five state-of-the-art web agents across 60 webpages from Online-Mind2Web, it achieved attack success rates of 71.7%-100%, while the authors' proposed defenses reduce success to 1.7%. Details →

Self-replicating prompt injections exist · OpenAI Alignment

OpenAI's alignment team disclosed research demonstrating "self-replicating prompt injections" — a worm-like variety of prompt injection that both achieves an adversarial goal and induces an AI agent to reproduce the malicious instruction on public output channels (email, file writes, code comments), allowing it to spread to downstream agents. The capability was discovered on June 27, 2026 via OpenAI's GPT-Red self-play red-teaming framework (based on GPT-5.4-mini) and disclosed September 25, 2026; it was observed only in simulated tool-call environments with no real-world attacks recorded. Details →

One Link Away: What 1,350 Runs Taught Us About Prompt Guardrails

A study by Humanbound.ai ran 1,350 experiments against a pricing agent to measure how much a single system-prompt line about prompt injection actually protects against indirect prompt injection, finding four of five non-OpenAI models exfiltrated a confidential unit cost (£118.40) to an attacker server 100% of the time when unguarded. The attack splits across two pages: a clean competitor listing links one hop to an attacker-controlled 'live-offer exchange' page that requests the secret via a GET request, evades keyword filters by never naming the secret, and obfuscates the value with inserted 'x' characters. Details →

Trust propagation and structural containment in Multi-agent LLM pipelines

Researchers empirically study attack propagation in a four-agent LangGraph pipeline (Supervisor, Researcher, Validator, Executor), evaluating shared-memory poisoning and indirect prompt injection via forged approvals in retrieved documents. They introduce the Judgment Bypass Rate metric and show that structural authorization using task-bound signed tokens and a separately verified policy oracle can contain compromised agent behavior (100% JBR but 0% Unsafe Action Rate) even when upstream LLM judgment fails, with an Observer layer cutting hijacking false positives from 49% to 7%. Details →

SalesBleed shows how trusted AI agents can become data-exfil

Zenity Labs disclosed SalesBleed on September 24, 2026, a set of Salesforce Agentforce weaknesses (associated with CVE-2026-73570) that let attacker-supplied CRM data hijack an AI agent for zero-click data exfiltration and Slack phishing. An attacker submits a Web-to-Lead record containing hidden prompt-injection instructions; when an internal user later asks Agentforce to review recent leads, the agent queries sensitive account data and leaks it via DNS lookups triggered automatically by HTML image rendering or Slack URL unfurling. Salesforce says it has remediated the issues and found no evidence of exploitation against customers. Details →

Two prompt injection paths into Rovo: one fixed (RovoBlast), one open.

Martin Runge's community write-up analyzes two prompt-injection techniques against Atlassian's Rovo AI assistant: RovoBlast (disclosed by Varonis Threat Labs at DEF CON 34), which abused a rovoChatPrompt URL parameter to inject instructions into an authenticated session and was fixed server-side by Atlassian on 8 July 2026; and an indirect prompt-injection method from PromptArmor that hides malicious instructions in content Rovo processes (Jira issues, Confluence, PDFs) and exfiltrates data via Markdown image and URL-retrieval requests. The second path is noted as still open, and disabling org-level web search does not stop it because the URL retrieval tool remains available. Details →

GitHub - rudratoshs/buried-injections: 🛡️ Regex catches 0%, Meta's Prompt Guard 2 catches 1% of 629 realistic AgentDojo injection attacks when they're buried in tool output. Reproducible benchmark. · GitHub

The 'buried-injections' GitHub project is a reproducible benchmark showing that regex-based detection catches 0% and Meta's Prompt Guard 2 catches roughly 1% of 629 realistic AgentDojo prompt-injection attacks when the malicious instructions are buried inside tool output (indirect prompt injection). The benchmark is published as runnable code demonstrating the gap between existing detectors and realistic indirect-injection scenarios. Details →