First reported arxiv.org
Research · latest
First reported arxiv.org
KidnapRAG: A Black-Box Attack for Hijacking Reasoning in Agentic Retrieval-Augmented Generation Systems
KidnapRAG is a research paper presenting a black-box poisoning attack against Agentic RAG systems in which the attacker only publishes externally retrievable poisoned documents. The method uses three role-specific documents (Bait, Chain-Link, and Mal-Ins) to hijack an agent's multi-step reasoning chain, outperforming existing poisoning baselines across multiple frameworks, LLM backbones, and benchmarks; code is released on GitHub. Details →First reported google.com
Beware of Agentic Botnets: Scalable Untargeted Promptware Attacks via Universal and Transferable Adversarial HalluSquatting
Researchers from Tel Aviv University, Technion, and Intuit disclosed 'HalluSquatting' (adversarial hallucination squatting), a technique that exploits the predictable tendency of LLMs to hallucinate resource identifiers (repos, skills, URLs) in tool calls. By preemptively registering the hallucinated resources, attackers can achieve scalable, untargeted remote tool execution and remote code execution across popular agentic LLM applications without any direct injection channel, potentially building agentic botnets. Details →First reported arxiv.org
Your Agent's Memories Are Not Its Own: Forged Reasoning Attacks on LLM Agent Memory and Defenses
Researchers at Penn State introduce FARMA (Forged Amplifying Rationale Memory Attack), which poisons an LLM agent's remembered reasoning history rather than its factual knowledge, inserting forged reasoning traces with evasive language that bypass keyword filters and self-referential reinforcement that defeats consensus-based defenses like A-MemGuard, achieving up to 100% attack success. They also present SENTINEL, a layered defense pipeline whose Reasoning Guard structurally analyzes entries using five weighted signals, reducing FARMA's success rate to as low as 0% with no false positives across 326 benign traces. Details →First reported dev.to
How I Used Automated Red Teaming to Evaluate My AI Agent's Safety - DEV Community
A DEV Community walkthrough demonstrates using automated red teaming (the Strands Evals red-teaming module with AdversarialCaseGenerator and CrescendoStrategy multi-turn escalation) against an internal helper AI agent built on Strands Agents and Amazon Bedrock. The author shows how a bash-equipped agent can be coaxed via gradual multi-turn escalation into reading AWS credentials and how auto-generated adversarial cases surface data-exfiltration, excessive-agency, and system-prompt-leak breaches, going from 6/9 detected breaches to 0 after adding guardrails. Details →First reported arxiv.org
Bad Memory: Evaluating Prompt Injection Risks from Memory in Agentic Systems
"Bad Memory" is a research paper studying prompt injection attacks in memory-based agentic systems using a sandboxed synthetic workspace, evaluating Anthropic Claude Code and OpenAI Codex across four models. The authors find that while it is hard to make an agent overwrite its own memory files using untrusted external content, payloads already planted in persistent memory files can successfully compromise current and future sessions, with attack success and persistence varying by system, model, and adversarial goal. Details →First reported · updated · 2 reports openai.com
GPT-Red: Unlocking Self-Improvement for Robustness
OpenAI describes GPT-Red, an internal automated red-teaming model trained at large post-training compute scale to discover prompt injection vulnerabilities in its models and generate adversarial training data. OpenAI reports using GPT-Red to adversarially train GPT-5.6 Sol, claiming 6x fewer failures on its hardest direct prompt injection benchmark versus a prior production model. Details →First reported tracebit.com
Now, defenders are embracing the prompt injection, too
Researchers at Tracebit disclosed a defensive technique they call "context bombing," in which prompt injections placed alongside decoy AWS secrets trigger an attacking LLM's own guardrail refusal mechanism, causing autonomous AI hacking agents to shut down. Across 152 attack runs against five models (Opus 4.8, Gemini 3.1 Pro, GLM 5.2, DeepSeek 4 Pro, Kimi 2.6), planting a forbidden-content string cut full account admin compromise from 57% to 5% and complete compromise with persistence from 36% to 1%. Details →First reported promptarmor.com
Connecting AI agents to outside services explodes the risk radius
The Register reports on PromptArmor research finding that AI agent connectors — OpenAI/ChatGPT and Anthropic/Claude MCP-based integrations with services like Gmail, Slack, and Dropbox — change constantly, with 931 of 2,517 connectors (37%) changing over six weeks, 1,686 new tools added and 1,127 tool descriptions rewritten. The study found connectors gaining write and destructive capabilities (Dropbox went from 8 to 24 tools, 0 to 4 destructive), permission scopes shifting, injected model instructions appearing, and about 2 in 5 Claude connectors likely calling additional external AI services. Details →First reported · updated · 4 reports arxiv.org
Agent Data Injection Attacks are Realistic Threats to AI Agents
A research paper by Woohyuk Choi and colleagues introduces agent data injection attacks (ADI), a new category of indirect prompt injection in which malicious data is disguised as trusted data (such as security-critical metadata or agent context data like tool call/response formats) rather than as instructions. The authors demonstrate ADI against real-world agents, achieving arbitrary click attacks on web agents (Claude in Chrome, Antigravity, Nanobrowser) and remote code execution plus supply-chain attacks on coding agents (Claude Code, Codex, Gemini CLI), showing it bypasses existing IPI defenses because agents fail to isolate trusted from untrusted data. Details →First reported · updated · 5 reports arxiv.org
Prompt Injection as Role Confusion
The paper "Prompt Injection as Role Confusion" (arXiv:2603.12277, ICML 2026) by Charles Ye, Jasmine Cui, and Dylan Hadfield-Menell traces prompt injection to role confusion: LLMs perceive the source of text from how it sounds rather than its labeled role, so injected text occupies the same representational space as the trusted role it imitates. The authors introduce role probes to measure internal role perception and demonstrate CoT Forgery, a zero-shot attack injecting fabricated reasoning into user prompts and tool outputs that yields 60% attack success against frontier models with near-zero baselines. Details →First reported arxiv.org
The Injection Paradox: Brand-Level Suppression in Safety-Trained LLM Recommendations via RAG Context Injection
The paper 'The Injection Paradox' by Hyunseok Paeng documents a reproducible failure mode in RAG-based LLM recommendation systems where indirect prompt injections embedded in retrieved documents backfire, suppressing the injected brand below its injection-free baseline. In safety-trained Claude models (e.g., Claude Opus 4.6), a single injected document drops the target brand from a 54% baseline to zero top-2 recommendations and propagates suppression to the brand's uninjected documents, while GPT models instead show increased recommendations. The authors note this enables a reverse-attack scenario where an adversary injects a competitor's documents to suppress that competitor, and release code, prompts, and results. Details →First reported arxiv.org
Devil in the Lens: Analyzing and Defending Physical Prompt Injection Against Vision-Language Models on Wearable Devices
A research paper, "Devil in the Lens," characterizes physical prompt injection attacks against Vision-Language Models on wearable devices such as AI glasses, where malicious text embedded in real-world environments acts as an indirect prompt-injection channel that hijacks model behavior. Using photos captured from AI glasses across 200+ environments, the authors identify 6 threat vectors, evaluate 12 VLMs (achieving attack success rates up to 96% simulated and 60% real-world), and propose two defenses: a mask-based external filter and a semantic-vector-based internal detector. Details →First reported · updated · 3 reports acm.org
Benchmarking and Defending against Indirect Prompt Injection Attacks on Large Language Models | Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.1
PromptShield is a benchmark introduced in an ACM SIGKDD paper for training and evaluating deployable prompt injection detectors, curated to include both conversational and application-structured data. The authors also fine-tune a new prompt injection detector that achieves higher performance in the low false-positive-rate regime than prior schemes. Details →First reported arxiv.org
BraveGuard: From Open-World Threats to Safer Computer-Use Agents
BraveGuard is a self-evolving defense framework, presented in an arXiv paper (arXiv:2606.01166), for training guard models to monitor computer-use agents that interact with files, terminals, browsers, and external tools. The framework mines open-world threat signals, instantiates them as executable agent tasks, and derives trajectory-level supervision; on the AgentHazard benchmark it raised detection accuracy from 38.79% to 82.38% over off-the-shelf guards like Qwen3-Guard and Llama-Guard variants. Details →First reported · updated · 2 reports crowdstrike.com
CrowdStrike Uncovers New Prompt Injection Techniques
CrowdStrike's AI security research team disclosed 18 new additions to its prompt injection taxonomy, expanding coverage to over 200 techniques, and detailed five notably: Trigger-Activated Rule Addition (PT0201), Cognitive Token Suppression (PT0197), Algorithmic Payload Decomposition (PT0200), Special Token Injection (PT0198), and one further method. The techniques target AI agents that crawl webpages, access file stores, and run shell commands, using indirect injection to hide malicious instructions in consumed data. Details →First reported aclanthology.org
Rethinking Jailbreak Detection of Large Vision Language ...
Researchers from Washington University in St. Louis and Texas A&M propose Representational Contrastive Scoring (RCS), a framework for detecting multimodal jailbreak attacks against Large Vision-Language Models by inspecting the models' internal representations. The two instantiations, MCD and KCD, claim state-of-the-art detection performance and better generalization to unseen attack types while reducing over-rejection, with code released on GitHub. Details →First reported arxiv.org
Scalable Hierarchical Attention Transformers for Multi-Turn Jailbreak Detection in Long Conversations
Researchers from Zscaler present a hierarchical attention transformer for detecting multi-turn jailbreaks that spread unsafe intent across a conversation via gradual escalation, reframing, and role manipulation. The two-level architecture encodes individual turns with a multilingual encoder then reasons over turn summaries with a lightweight conversation module, reporting an F1 of 0.9394 on a 14,038-conversation benchmark and outperforming a strong LLM baseline while halving false positives. Details →First reported adversa.ai
Solving GitHub’s Secure Code game with an AI red teaming agent
Adversa AI documents pointing its autonomous AI Red Teaming Agent at GitHub's open-source Secure Code Game 'ProdBot' challenge, an intentionally vulnerable agentic-AI teaching target where the goal is to coax the agent into leaking a flag stored one directory above its sandbox. The agent cleared the first three levels (Sandbox, Web, MCP) at full score in a single 57-second run, relying on 'context seeding' — fabricating a plausible prior workflow implying authentication had already occurred — rather than overt jailbreak language, illustrating that each added capability (web browsing, MCP tool chains, skills, multi-agent) opens a new attack surface via indirect injection, poisoned tool chains, and confused-deputy trust. Details →First reported phishfort.com
Multimodal AI Jailbreak Attacks: The Image-Based Threat to Enterprise AI
Researchers at Florida International University (Hadi Amini and Md Jueal Mia) developed JaiLIP (Jailbreaking with Loss-guided Image Perturbation), a technique that embeds instructions into images via pixel-level perturbations invisible to humans but readable by vision-language models. Tested against BLIP-2, JaiLIP nearly doubled the rate of policy-violating outputs, and because most enterprise guardrails only inspect the text channel, the malicious payload in the image tensor bypasses safety filters entirely. Details →First reported arxiv.org
PVDetector: Detecting Prompt Injection Attacks on Purpose-Specific LLM Agents through Policy-Violation Concept Analysis
PVDetector is a training-free framework presented in an arXiv paper for detecting prompt injection attacks against purpose-specific LLM agents. It works by analyzing the model's hidden activation space for latent policy-violation concepts derived from contrastive pairs of policy-violating and policy-compliant prompts, measuring hidden-state alignment during inference. Details →First reported akamai.com
AI Reconnaissance: The Missing Layer in Chatbot Security
Akamai's AI security research team describes "AI reconnaissance," a technique in which deployed AI assistants block direct requests for sensitive data but readily leak operational context — capabilities, boundaries, knowledge-base access, and tool/action surface — when asked simple, benign-looking questions. This leaked context lets attackers craft targeted, application-specific prompt injections that bypass model-level safety, and the authors argue runtime guardrails must detect this early probing stage of the attack kill chain. Details →First reported arxiv.org
DualView: Preventing Indirect Prompt Injection in Personal AI Agents
DualView is a defense presented in an arXiv paper that extends untrusted-data tracking from a personal AI agent's context out into the user's environment (file system, shell, network, and other agents) to block indirect prompt injection, including a newly-highlighted 'stored IPI' variant where an agent saves attacker-controlled data and later rereads it as trusted. Deployed as an OpenClaw plugin using only tool hooks, DualView gives each channel an AgentView (untrusted data shown as symbols) and a HumanView (original data), and reportedly blocked every IPI attack on an IPI benchmark and PinchBench while preserving utility. Details →First reported arxiv.org
MUZZLE: Adaptive Agentic Red-Teaming of Web Agents Against Indirect Prompt Injection Attacks
MUZZLE is an automated agentic red-teaming framework, presented in a USENIX Security 2026 paper by researchers from Northeastern University and Mozilla, for evaluating the security of LLM-based web agents against indirect prompt injection attacks embedded in untrusted web content. Muzzle uses agent trajectories to identify high-salience injection surfaces and adaptively generates context-aware malicious instructions, discovering 44 new attacks across 4 web applications, including 3 cross-application prompt injection attacks and an agent-tailored phishing scenario. Details →First reported phoenix.security
Supply Chain Attacks 2026: npm, PyPI, VS Code, AI Agents — 0 CVEs
Phoenix Security's Malware Package Intelligence report analyzes 59 supply chain attack campaigns and 657 malicious package-versions from June 2024 to June 2026, documenting an acceleration across npm, PyPI, and the VS Code Marketplace. It highlights a May 2026 self-propagating worm that turned one compromised maintainer token into 226 poisoned packages, and finds AI agent tooling — MCP server injection, .cursorrules poisoning, CLAUDE.md hidden instructions, and AI coding assistant SessionStart hooks — used as a confirmed delivery mechanism in at least 14 of the 59 campaigns. Details →First reported aminrj.com
Deleting the Malicious MCP Server Doesn't Save You | Amine Raji, PhD
A lab-built demonstration (mcp-attack-labs, Lab 08) chains MCP tool-description poisoning into an Agent-to-Agent (A2A) intrusion: a poisoned tool instructs the compromised agent to register a rogue A2A agent, hijack routing via shadowing, exfiltrate data, and persist even after the malicious MCP server is deleted. Each stage maps to a named vulnerability class (OWASP MCP Top 10 MCP03, ASI10, ASI07) and is paired with a detection that fires on it. Details →First reported arxiv.org
A Layered Security Framework Against Prompt Injection in RAG-Based Chatbots
A research paper by Gulshan Saleem and colleagues presents a three-layer middleware framework to defend RAG-based chatbots against both direct and indirect prompt injection, combining input screening, a provenance-based instruction hierarchy during context assembly, and output auditing. Evaluated on 5,080 samples across GPT-4o, Llama 3, and Mistral 7B, the framework reduced Attack Success Rate from 71.4% to 11.3% while maintaining a 4.8% false positive rate and 61.2 ms median latency overhead. Details →First reported arxiv.org
Prompt Injection as Role Confusion
The paper "Prompt Injection as Role Confusion" (arXiv:2603.12277) argues that LLMs infer a text's role from stylistic cues rather than its labeled role tags, so untrusted text that imitates a trusted role inherits that role's authority. The authors introduce "role probes" to measure internal role perception and demonstrate CoT Forgery, a zero-shot attack that injects fabricated reasoning into user prompts and tool outputs, achieving ~60% attack success on frontier models against near-zero baselines. Details →First reported · updated · 2 reports canopii.dev
State of MCP Security 2026 v2
Canopii's "State of MCP Security 2026" report scanned more than 11,000 published Model Context Protocol servers and found 830 graded D or F, 232 with confirmed dangerous code sinks (eval, shell injection, unsafe deserialization) that can turn prompt injection into host code execution, 184 versions that silently altered tool definitions after publication ("rug pulls"), 1,617 servers shipping dependencies with known vulnerabilities, 260 running install-time scripts, and 7 confirmed typosquats. Popular servers with 1,000+ GitHub stars were over 5x more likely to be high-risk. Details →First reported arxiv.org
Prismata: Confining Cross-Site Prompt Injection in Web Agents
Prismata is a research defense for autonomous web agents that confines cross-site prompt injection by enforcing contextual least privilege, using dynamic trust derivation to produce permission labels for page content and mechanical confinement to redact content and restrict agent capabilities. The authors report it substantially reduces attack success across recent published web-agent attacks, including adaptive variants, while preserving benign task utility and requiring no developer annotations. Details →First reported koreshield.ai
Reproducing an Indirect Prompt Injection Against a RAG Pipeline
A Koreshield write-up reproduces an indirect prompt injection against a RAG customer-support pipeline, where a poisoned knowledge-base document (doc-002) contains a bracketed fake 'system note' instructing the model to dump all retrieved documents and exfiltrate internal tokens with an 'EXFIL::' marker while hiding the action. The author provides runnable code, a public GitHub harness (UncleTeslim/prompt-injection-repro), and real model logs demonstrating the attack against a local Llama3-8B model. Details →First reported arxiv.org
DualView: Preventing Indirect Prompt Injection in Personal AI Agents
DualView is a research system from CompSec Lab @ SNU (arXiv:2607.03821) that defends personal AI agents like OpenClaw against indirect prompt injection, including a variant called 'stored IPI' where attacker-controlled text persists in the user's file system, shell, or network and is re-read later as trusted data. DualView extends untrusted-data tracking from the agent context into the user's environment by maintaining two views (AgentView shows untrusted data as symbols; HumanView preserves originals) and deploys as an OpenClaw plugin using tool hooks; in evaluation on an IPI benchmark and PinchBench it blocked every IPI attack while keeping utility near the unprotected baseline. Details →First reported reliable-ai.review
On the Impossibility of Mitigating AI Jailbreaks – AI RELIABILITY REVIEW
A blog post (an intuitive version of the NeurIPS 2025 paper 'Mission Impossible: A Statistical Perspective on Jailbreaking LLMs') argues that alignment post-training only reshapes a model's output distribution without imposing hard constraints, making jailbreaks and prompt injections systematically exploitable. It illustrates this with reported real-world failures (McDonald's bot solving python puzzles, xAI chatbots giving bomb instructions, ChatGPT reproducing copyrighted characters). Details →First reported simonwillison.net
What happened after 2,000 people tried to hack my AI assistant
Fernando Irarrázaval ran a public challenge at hackmyclaw.com inviting people to leak secrets from his OpenClaw test instance via email-based prompt injection. After roughly 6,000 attempts by ~2,000 people, nobody succeeded in extracting the secret, with the instance protected by anti-prompt-injection system rules on the underlying model. Details →First reported openaca.dev
Your Agent Risk Isn't in One Plugin. It's in the Composition.
OpenACA analyzes how agentic AI risk emerges from the composition of plugins, MCP servers, skills, and npm dependencies rather than any single component, using the Claude Code 'imessage' plugin as an example (a local MCP server reading chat.db, sending iMessages, with skills that rewrite access policy). They scanned the official Claude plugin marketplace (62 manifests, 530 components) and found all 124 known-vulnerability advisories clustered in the four message-channel plugins (discord, telegram, fakechat, imessage) that ingest untrusted input and can exfiltrate local files, arguing SCA and runtime monitoring miss this composition-level exposure. Details →First reported adversa.ai
Solving the “Breaking the Prompt” DEF CON AI CTF with AI Red Teaming Agent
A write-up describing how Adversa AI's AI Red Teaming Agent solved a five-stage prompt injection/jailbreak CTF challenge ('Breaking the Prompt' by TrendAI at DEF CON), detailing which LLM jailbreak techniques worked and how they relate to guardrail design. Details →First reported lateos.ai
GPT-5 Nano IPI Assessment — LLM Vulnerability Research
A black-box prompt injection susceptibility assessment of GPT-5 Nano using the IPI Taxonomy v0.13 across 201 analyzed test cases, reporting a 38.3% overall susceptibility rate. The model was fully resistant to surface-level attacks (CSS concealment, HTML cloaking, SEO phishing, RAG corpus poisoning) but highly vulnerable to recursive instruction framing (100%) and MCP tool description poisoning (80%). Details →First reported trendmicro.com
Pwning Agentic AI Part I: Your AI Agent Is Already Compromised | Trend Micro (US)
Trend Micro's TrendAI Research describes a new agentic-AI exploitation pattern they call return-to-tool (RTT) exploits, where embedded instructions in benign-looking untrusted input cause an AI agent to invoke its authorized tools to perform attacker-intended actions such as exfiltrating production database credentials. The research notes a vulnerable PostgreSQL MCP server image pulled over 100,000 times from Docker Hub as a realistic exposure vector. Details →First reported triagewall.io
I Found a Prompt Injection in My Own IDS Triage Tool
The author of Triagewall, a local LLM tool that classifies Suricata IDS alerts using Foundation-Sec-8B via Ollama, demonstrated an indirect prompt injection where attacker-controlled URL fields could dictate the model's verdict and confidence. A crafted URL embedding directives caused the model to return exactly the attacker-chosen classification (false_positive, 0.99), bypassing canary-token and schema-validation defenses. Details →How the wire is made
Poll & cluster
Internet is crawled for AI security news and near-duplicate coverage is embedded and grouped into durable items.
Curate
AI Agent filters for agentic-AI relevance, classifies and tags each item, scores severity for threats, and writes the summary.
Every item here is one machine-curated intelligence object, not a headline.
Read the wire for free. There is a small charge to ask the index questions.
The wire, open
The complete curated feed, no key required.
- GET /feed.xml — RSS 2.0, every item
- GET /api/items — read-only
The vector desk
Query the index by meaning, not just keyword.
- GET /api/items?tags=&minSeverity=&itemType=
- GET /api/search?q= — keyword
- GET /api/semantic?q= — vector