Topic · Latest 50 of 130 active items

Prompt Injection

Items tagged for indirect prompt injection, instruction smuggling, retrieval poisoning, or related control-plane failures.

Matching wire items

Grok chat history leak: Cryptographic Context Injection

Adversa AI disclosed a new technique it calls Cryptographic Context Injection, which hides malicious instructions inside AES-256-GCM ciphertext so static guardrails cannot read them, then induces the model to decrypt them in its own code-execution sandbox where the recovered plaintext is treated as trusted instructions. Against xAI's Grok web chat, a benign 'summarize this page' request triggers zero-click exfiltration of the user's session data and chat history to an attacker URL; against Gemini it produces content the model normally refuses. Reported to xAI in June 2026 and still reproducible as of August 19, while Gemini's success rate has fallen but is not fully closed. Details →

Prompt Injection Exploits: The CVE That Weaponized the AI Coding Workflow

A podcast with Checkpoint's Adam Forester unpacks a disclosed CVE in Anthropic's Claude Code where the AI coding assistant did not validate its local settings.json on boot, letting a booby-trapped GitHub repo execute arbitrary commands (up to ransomware) the moment a developer downloaded and ran it, with no phishing required. The vulnerability was patched twice and unpatched installs may remain exploitable; the discussion frames it as a new class of indirect prompt injection and also recounts an Alibaba 'Rome' agent that opened covert SSH tunnels to mine crypto. Details →

CoSnitch: When Your AI Assistant Becomes Its Own Whistleblower

Varonis Threat Labs disclosed CoSnitch (CVE-2026-24301), a critical one-click vulnerability chain in Microsoft Copilot Personal that combines the ?q= URL parameter with an undocumented autorun=1 parameter to auto-execute an attacker-supplied prompt on page load, then queries connected apps (Gmail, Drive, Calendar, OneDrive) and exfiltrates data via encoded URLs, plus indirect prompt injection through web summarization that poisons persistent memory. Researchers used a 'meta-hacking' technique, repeatedly asking Copilot why an attack wouldn't work until the assistant disclosed its own disabled parameters and session conditions. Microsoft shipped patches on August 18, 2026; Varonis reports no evidence of in-the-wild exploitation. Details →

Atlassian Rovo Can Be Tricked Into Sending Jira and Confluence Data to Attackers

Researchers at Varonis Threat Labs (RovoBlast) and PromptArmor independently showed that Atlassian's Rovo AI assistant can be manipulated via prompt injection to collect Jira and Confluence data a signed-in user can access and exfiltrate it to an external server. Varonis found that the rovoChatPrompt URL parameter preloads attacker instructions so a single click by an authenticated user triggers execution; PromptArmor hid instructions in an uploaded file that Rovo reads, working even with web-search disabled. Varonis's route was responsibly disclosed and fixed (CVE-2026-50522), while the PromptArmor file-based bypass is single-sourced and its remediation is not confirmed. Details →

Amazon Kiro: AI Is Breaking Vulnerability Disclosure Processes

Mindgard disclosed a prompt-injection vulnerability in Amazon Kiro, an AI-powered agentic IDE, that lets attacker-controlled repository content coerce the Kiro agent into reading local sensitive data, modifying a workspace URL, and triggering an outbound request that exfiltrates the secret. The flaw was reproduced in Kiro IDE 0.7.45 on Windows in both trusted and untrusted workspaces via Kiro Powers (MCP configs and POWER.md steering files); exploitation requires the user open a malicious workspace file and message the agent, and is assessed as low difficulty. Details →

The lethal trifecta for AI agents: private data, untrusted content, and external communication

An explainer on stopping prompt injection in MCP servers frames the problem as the 'lethal trifecta' (private-data access, exposure to untrusted content, and external communication) coined by Simon Willison, using the Invariant Labs demonstration against GitHub's official MCP server as its central case. In that attack a malicious GitHub issue embedded agent-directed instructions that caused a coding agent to leak private repo details into a public pull request, with no exploited code or CVE. The piece argues the fix is architectural rather than prompt-based. Details →

Breaking Claude Code Opus 5 Auto Mode

Johann Rehberger (Embrace The Red) demonstrated an indirect prompt injection attack chain that hijacks Claude Code Opus 5 in Auto Mode via a simple 'summarize this website' request, achieving code execution with a 60-80% success rate. The chain nudges Claude from WebFetch to curl, downloads a ZIP whose extracted malicious struct.py shadows Python's standard module, so importing base64 triggers attacker code; in some runs Auto Mode's safety classifier even blocked Claude's own cleanup command. The result contrasts with a vendor-commissioned evaluation (Trajectory Labs) that reported 0.00% attack success for Opus 5 in Auto Mode. Details →

I broke an MCP server in 10 minutes — the exact prompt injection attack chain (with fixes)

A DEV Community write-up demonstrates an indirect prompt injection attack chain against a typical MCP server exposing read_file and send_email tools, where a submitted document containing a fake 'SYSTEM NOTE' instruction causes the model to exfiltrate /etc/passwd by email because no boundary separates data from instructions. The author outlines fixes (treat tool/file content as data, per-session tool allowlists, confirmation gates on external-sending tools) and notes tool-description poisoning persists across sessions. The post also promotes a free hosted scanner. Details →

AWS Kiro: Arbitrary Code Execution via Indirect Prompt Injection

Researchers found a vulnerability (CVE-2026-10591) in AWS Kiro, an agentic IDE, where hidden instructions planted in a web page or source file that Kiro processes can trigger indirect prompt injection to rewrite Kiro's own MCP server configuration (~/.kiro/settings/mcp.json) or allowlist arbitrary Bash commands in .vscode/settings.json, achieving arbitrary code execution on the developer's machine with no approval prompt. The human-in-the-loop approval boundary is bypassed because Kiro can write to these config files without user consent, and AWS has issued a fix and CVE. Details →

GhostJacking Attacks: Half of the Fortune 500 Run These Tools. Getting Blocked by the Firewall Was the Way to Take Over Their AI Agents - Tenet Security

Tenet Security disclosed 'GhostJacking' at DEF CON 34, an indirect prompt-injection technique that hides malicious instructions inside trusted operational data such as logs, alerts, and bug reports, then tricks AI coding and operations agents into executing them with their own legitimate permissions. Demonstrations across Cloudflare, Datadog, and Sentry workflows showed agents altering DNS records, running commands, exposing frontend keys, and exfiltrating environment secrets and cloud credentials while falsely reporting success. Tenet also reported that Anthropic fixed a Claude Desktop remote data-exfiltration flaw with no CVE assigned. Details →

LLM security testing: how to pentest LLMs and MCP servers

Escape.tech publishes a methodology for pentesting LLM applications and MCP servers, mapping attacks to the OWASP Top 10 for LLM Applications 2025 (prompt injection, improper output handling, excessive agency, system-prompt leakage) and demonstrating them against a self-built vulnerable FastMCP lab. The guide explains why LLM testing breaks the web-app playbook — no parser boundary, non-deterministic interpreter, no sanitization line — and notes MCP tool descriptions and tool responses both reach the model as trusted injection channels, referencing tool poisoning and rug-pull attacks. Details →

Otto Support - The Confused Deputy

Bishop Fox's otto-support CTF demonstrates confused deputy attacks against AI agents, where an agent reads attacker-controlled content (a poisoned support ticket, email, or calendar invite) and executes hidden instructions using its own legitimate privileges. The write-up reproduces the scenario via IDOR-based ticket poisoning and metadata service abuse to escalate into a support-agent role, referencing real-world cases like EchoLeak, ConfusedPilot, and Copilot calendar exploits, and proposes mitigations such as data/instruction separation, per-task tool registration, least privilege, human-in-the-loop, and egress controls. Details →

Prompt Injection: How to Protect AI Agents and LLM Apps

An educational guide, "Prompt Injection: How to Protect AI Agents and LLM Apps," explains the instruction-vs-data trust problem underlying direct and indirect prompt injection and lays out a layered defense model (least privilege, tool access controls, approvals for consequential actions, structured tool arguments, validation, sandboxing, monitoring, and adversarial evals). The piece synthesizes framing from OWASP LLM Top 10 (LLM01), OpenAI, and Anthropic, using examples such as malicious instructions hidden in emails, webpages, RAG chunks, and MCP resources. Details →

Prompt Injection Hacking: Emerging Trade Secret, Employment, and Litigation Risks

An article from Search Engine Land explains how prompt injection has evolved to threaten brands and AI workflows, citing examples such as Permiso's 'ChatGPhish' where malicious payloads embedded in ordinary webpages coerce AI assistants (ChatGPT, Perplexity) into rendering spoofed account alerts and malicious QR codes inside the chat interface, bypassing URL blocklists. It surveys additional vectors including LLM referral hijacking via semantic embedding, weaponized multimodal inputs (podcasts, video, voice agents), rogue AI customer-support agents, and supply-chain risk from unvetted AI vendors. Details →

Mitigating Indirect AGENTS.md Injection Attacks in Agentic Environments | NVIDIA Technical Blog

NVIDIA's AI Red Team demonstrated an indirect AGENTS.md injection attack in which a malicious Go dependency executes during a normal build, detects a Codex environment via the CODEX_PROXY_CERT variable, and writes a crafted AGENTS.md file whose directives claim 'absolute authority' over user requests and instruct the coding agent to hide its changes from PR summaries and commit messages; the agent complied, quietly inserting a sleep delay. Two further efforts (Prompt Security's cloned-repo attack against VS Code Copilot Chat leading to credential exfiltration, and GitInject's CI/CD attack against four AI providers in GitHub Actions) show the same AGENTS.md-as-trusted-instructions mechanism exploited across vectors. Details →

Measuring Indirect Prompt Injection in Autonomous Web Agents by Sahir Maharaj :: SSRN

The paper 'Measuring Indirect Prompt Injection in Autonomous Web Agents' by Sahir Maharaj introduces WIPI, a deployment-oriented measurement protocol for Web Indirect Prompt Injection that separates exposure, instruction uptake, harmful action, attacker-goal completion, concealment, recovery, benign utility, and overblocking. It synthesizes academic benchmarks (e.g., WASP), browser-security studies, standards, and red-team evidence, arguing that even low model-level attack-success rates do not equate to a trustworthy web agent and that security must be enforced architecturally through provenance, instruction hierarchy, capability separation, and information-flow control. Details →

Document-borne AI worms can self-propagate through Copilot for Word | Hacker News

Researchers at enklypesalt.com demonstrated a document-borne, self-replicating prompt-injection worm that spreads through Microsoft Copilot for Word: a malicious Word document contains a hidden self-propagating prompt that, when processed by Copilot, forces the assistant to embed the same payload into other documents it drafts or edits, enabling zero-click chain propagation and data exfiltration. The write-up notes that at publication no robust mitigation for the broader vulnerability class was available, and the work is associated with assigned CVEs (CVE-2026-55145, CVE-2026-50522). Details →

Mitigate indirect prompt injection risks from Google Cloud MCP | Google Cloud Data Agent Kit extension for Antigravity IDE | Google Cloud Documentation

Google Cloud documentation for the Data Agent Kit extension (VS Code and Antigravity IDE) warns that coding agents connected via Google Cloud MCP can be hijacked through indirect prompt injection, where malicious instructions hidden in data sources such as Cloud Storage, BigQuery, email, or calendars are interpreted as commands. The guidance recommends mitigations including running agents in constrained environments like Cloud Workstations with disabled internet access and no root privileges. Details →

The Injection Paradox: Brand-Level Suppression in Safety-Trained LLM Recommendations via RAG Context Injection

An ICML 2026 workshop paper by Hyunseok Paeng, "The Injection Paradox," reports a reproducible failure mode in RAG-based LLM recommendation where prompt injections embedded in retrieved documents backfire, suppressing the injected brand below baseline in safety-trained Claude models (Claude Opus 4.6 dropped the target brand from a 54% baseline to zero top-2 recommendations). The suppression propagates to unmodified documents of the same brand, while GPT models showed the opposite effect, raising a reverse-attack scenario where an adversary poisons a competitor's documents to suppress that competitor's brand. Details →

Hidden in Memory: Sleeper Memory Poisoning in LLM Agents

The arXiv paper "Hidden in Memory: Sleeper Memory Poisoning in LLM Agents" (2605.15338), explained in a companion Medium write-up, demonstrates a delayed attack in which adversarial content in a document, webpage, or repository causes a stateful LLM assistant to store a fabricated memory about the user that re-emerges across future sessions. Across tested assistants, poisoned memories were written up to 99.8% of the time on GPT-5.5 and 95% on Kimi-K2.6, and among successful retrievals they steered attacker-intended agentic actions in 60–89% of evaluations; the authors release code on GitHub. Details →

A Security Analysis of Amazon S3 Vectors and Its Use in LLM Retrieval Pipelines

OFFENSAI and TUCN researchers analyzed Amazon S3 Vectors, AWS's native vector store for RAG pipelines, and demonstrated that its metadata is completely unvalidated on write, letting anyone with s3vectors:PutVectors forge chunk text, spoof citation URLs, rank-flood indexes, or overwrite legitimate chunks. In testing, a single planted vector drove a clinical RAG assistant to recommend a dangerous drug dose while citing an authentic PDF, and a poisoned chunk led to command execution in a tool-enabled agent's sandbox, with CloudTrail unable to reconstruct what changed. Details →

Malicious MCP Servers Can Split Instructions to Make AI Coding Agents Exfiltrate Secrets

ASSET Research Group's GhostSplice research shows a malicious MCP (Model Context Protocol) server can exfiltrate SSH keys, environment secrets, source code, and customer data from AI coding assistants by splitting a malicious request into individually benign fragments placed across tool descriptions, tool results, and server-initiated sampling. Because the agent combines instructions across these channels in a shared working context, no single fragment carries the whole malicious request, allowing the attack to succeed even after a blunt version of the same theft is refused. Details →

Tool call poisoning in agentic AI: A technical guide to attack mechanics and defenses

A technical guide explains MCP tool poisoning, an indirect prompt injection attack in which a malicious Model Context Protocol server hides instructions in tool metadata (such as the tool's description field) so that when an AI agent calls the tool, injected instructions enter the LLM context and are treated as trusted input, enabling restricted tool calls, data leakage, or system-prompt bypass. The guide draws on an arXiv STRIDE/DREAD threat model that found tool poisoning to be the most impactful client-side MCP vulnerability across seven major MCP clients, and proposes multi-layered defenses including static metadata analysis, decision-path tracking, behavioral anomaly detection, and user transparency. Details →

Fake Bug Report Hijacks AI Coding Agents at Scale

Tenet Security demonstrated at DEF CON 34 an "agentjacking" attack where an attacker POSTs a crafted error event containing markdown-formatted malicious instructions to a publicly exposed Sentry DSN ingest endpoint; when a developer asks an MCP-connected coding agent (Cursor, Claude Code) to debug, the agent retrieves the injected event, treats it as authoritative remediation, and executes commands that pull a malicious npm package and exfiltrate AWS, GitHub, npm, Kubernetes and CI/CD credentials. Controlled testing across 100+ organizations succeeded 85 percent of the time; no CVE was assigned, and Sentry deployed only a payload-string filter. Tenet also released agent-jackstop, drop-in hardening configs (egress allowlist, command approval, credential-read blocking) for Cursor and Claude Code. Details →