First reported openai.com
Topic · Latest 50 of 83 active items
AI Security Research
Research artifacts and technical findings that clarify how AI systems fail, can be evaluated, or can be defended.
Matching wire items
First reported darkreading.com
Choose Wisely: AI-Generated Coding Risk Varies, A Lot
Secure Code Warrior, in collaboration with RMIT University, released its AI Trust Index, a study evaluating 1,760 complete codebases generated by 16 frontier LLMs from vendors including OpenAI, Anthropic, and Google. The research found AI-generated code introduces roughly 15 vulnerabilities per codebase on average, with actual risk depending more on the development framework pairing than on the model chosen. Details →First reported arxiv.org
(A)I Sees What You Don't: Exploiting New Attack Surfaces in Third-Party Mobile Agents
Researchers from Simon Fraser University, CUHK, Shandong University, and QAX's Xingtu Lab published an arXiv paper (arXiv:2607.00333) demonstrating seven concrete attacks against five open-source mobile AI agent frameworks (AppAgent, AppAgentX, Mobile-Agent-v3, Open-AutoGLM, and MobA). A malicious Android app without privileged permissions can slip invisible on-screen text that the VLM-driven agent reads and acts on, exploiting a 'Screen Perception' surface (human-vs-machine vision gap) and a 'Misused Channel' surface to hijack agent actions and even achieve arbitrary command execution on the host PC driving the agent. All five frameworks fell to at least six of the seven attacks; no CVE was assigned and authors report no evidence of in-the-wild use. Details →First reported anthropic.com
Measuring LLMs' impact on N-day exploits
Anthropic's Frontier Red Team measured how much large language models can accelerate N-day exploit development, finding that its Claude Mythos Preview model autonomously built 8 working code-execution exploits across 18 recent Firefox patches and produced 8 full privilege-escalation chains from 21 Windows kernel patches. The research demonstrates that even public models with safeguards disabled can reverse-engineer patch diffs into exploits, collapsing the traditional weeks-long patch gap to hours. Details →First reported youtube.com
Your LLM Deception Monitor Is Broken. The Fix Is in the Training Data - Sachin Kumar, LexisNexis
Sachin Kumar (LexisNexis) presents peer-reviewed research (accepted at IJCNN, with open-source code) on detecting sleeper-agent backdoors in fine-tuned LLMs that pass behavioral evals yet flip to harmful output on an untested trigger. The proposed 'delta monitor' trains a sparse autoencoder on the difference between base and fine-tuned model activations, which in a controlled SQL-injection backdoor experiment flagged the backdoor with 40x the signal of crosscoder joint features, perfect precision, and zero false positives from a single layer. Details →First reported aclanthology.org
MM-PoisonRAG: Disrupting Multimodal RAG with Local and Global Knowledge Poisoning Attacks
MM-PoisonRAG is a research paper presenting local and global knowledge poisoning attacks against multimodal retrieval-augmented generation (RAG) systems used by multimodal large language models (MLLMs). The work shows how injecting poisoned entries into the retrieval knowledge base can manipulate MLLM outputs, exploiting RAG's reliance on retrieved external content. Details →First reported sciencedirect.com
A systematic analysis of LLM backdoor-integrated cyberattacks: Feasibility, impact factors, and realistic case studies in agent systems
A peer-reviewed study in Applied Soft Computing proposes an LLM backdoor-integrated cyberattack framework with a five-stage attack flow, from poisoned data construction to final execution, and empirically validates it across 5 LLM-based agent configurations and multiple models in realistic scenarios including flight control and database interaction. The authors analyze how backdoor vulnerabilities in LLMs propagate into agent systems and identify six key factors affecting attack success. Details →First reported arxiv.org
KidnapRAG: A Black-Box Attack for Hijacking Reasoning in Agentic Retrieval-Augmented Generation Systems
KidnapRAG is a research paper presenting a black-box poisoning attack against Agentic RAG systems in which the attacker only publishes externally retrievable poisoned documents. The method uses three role-specific documents (Bait, Chain-Link, and Mal-Ins) to hijack an agent's multi-step reasoning chain, outperforming existing poisoning baselines across multiple frameworks, LLM backbones, and benchmarks; code is released on GitHub. Details →First reported google.com
Beware of Agentic Botnets: Scalable Untargeted Promptware Attacks via Universal and Transferable Adversarial HalluSquatting
Researchers from Tel Aviv University, Technion, and Intuit disclosed 'HalluSquatting' (adversarial hallucination squatting), a technique that exploits the predictable tendency of LLMs to hallucinate resource identifiers (repos, skills, URLs) in tool calls. By preemptively registering the hallucinated resources, attackers can achieve scalable, untargeted remote tool execution and remote code execution across popular agentic LLM applications without any direct injection channel, potentially building agentic botnets. Details →First reported arxiv.org
Your Agent's Memories Are Not Its Own: Forged Reasoning Attacks on LLM Agent Memory and Defenses
Researchers at Penn State introduce FARMA (Forged Amplifying Rationale Memory Attack), which poisons an LLM agent's remembered reasoning history rather than its factual knowledge, inserting forged reasoning traces with evasive language that bypass keyword filters and self-referential reinforcement that defeats consensus-based defenses like A-MemGuard, achieving up to 100% attack success. They also present SENTINEL, a layered defense pipeline whose Reasoning Guard structurally analyzes entries using five weighted signals, reducing FARMA's success rate to as low as 0% with no false positives across 326 benign traces. Details →First reported · updated · 2 reports arxiv.org
Refused in Chat, Written in Code: Workflow-Level Jailbreak Construction in IDE Coding Agents
Alan Turing Institute researchers Abhishek Kumar and Carsten Maple demonstrated a "workflow-level jailbreak construction" against GitHub Copilot in VS Code, showing that harmful requests refused in direct chat succeed when decomposed across ordinary multi-turn IDE coding tasks. Across 204 prompts from Hammurabi's Code, HarmBench, and AdvBench, four closed-weight backends (Claude Sonnet 4.6, Claude Haiku 4.5, Gemini 3.1 Pro, Gemini 3.5 Flash) refused in 808/816 direct tries but produced unsafe outputs in all 816/816 runs when the harmful objective was embedded as an input to a coding workflow. Details →First reported · updated · 2 reports arxiv.org
Mitigating Taint-Style Vulnerabilities in MCP Servers via Security-Aware Tool Descriptions
The paper 'Mitigating Taint-Style Vulnerabilities in MCP Servers via Security-Aware Tool Descriptions' systematically analyzes vulnerabilities in Model Context Protocol (MCP) servers, finding that taint-style vulnerabilities (e.g., SSRF as in the Markdownify server, CVE-2025-5276) make up a substantial fraction and are slow to be remediated. It proposes SpellSmith, which builds tool-level risk profiles and embeds behavioral guidance into MCP tool descriptions plus LLM self-reflection to mitigate exploitation without code-level fixes. Details →First reported dev.to
How I Used Automated Red Teaming to Evaluate My AI Agent's Safety - DEV Community
A DEV Community walkthrough demonstrates using automated red teaming (the Strands Evals red-teaming module with AdversarialCaseGenerator and CrescendoStrategy multi-turn escalation) against an internal helper AI agent built on Strands Agents and Amazon Bedrock. The author shows how a bash-equipped agent can be coaxed via gradual multi-turn escalation into reading AWS credentials and how auto-generated adversarial cases surface data-exfiltration, excessive-agency, and system-prompt-leak breaches, going from 6/9 detected breaches to 0 after adding guardrails. Details →First reported arxiv.org
Bad Memory: Evaluating Prompt Injection Risks from Memory in Agentic Systems
"Bad Memory" is a research paper studying prompt injection attacks in memory-based agentic systems using a sandboxed synthetic workspace, evaluating Anthropic Claude Code and OpenAI Codex across four models. The authors find that while it is hard to make an agent overwrite its own memory files using untrusted external content, payloads already planted in persistent memory files can successfully compromise current and future sessions, with attack success and persistence varying by system, model, and adversarial goal. Details →First reported · updated · 2 reports openai.com
GPT-Red: Unlocking Self-Improvement for Robustness
OpenAI describes GPT-Red, an internal automated red-teaming model trained at large post-training compute scale to discover prompt injection vulnerabilities in its models and generate adversarial training data. OpenAI reports using GPT-Red to adversarially train GPT-5.6 Sol, claiming 6x fewer failures on its hardest direct prompt injection benchmark versus a prior production model. Details →First reported tracebit.com
Now, defenders are embracing the prompt injection, too
Researchers at Tracebit disclosed a defensive technique they call "context bombing," in which prompt injections placed alongside decoy AWS secrets trigger an attacking LLM's own guardrail refusal mechanism, causing autonomous AI hacking agents to shut down. Across 152 attack runs against five models (Opus 4.8, Gemini 3.1 Pro, GLM 5.2, DeepSeek 4 Pro, Kimi 2.6), planting a forbidden-content string cut full account admin compromise from 57% to 5% and complete compromise with persistence from 36% to 1%. Details →First reported promptarmor.com
Connecting AI agents to outside services explodes the risk radius
The Register reports on PromptArmor research finding that AI agent connectors — OpenAI/ChatGPT and Anthropic/Claude MCP-based integrations with services like Gmail, Slack, and Dropbox — change constantly, with 931 of 2,517 connectors (37%) changing over six weeks, 1,686 new tools added and 1,127 tool descriptions rewritten. The study found connectors gaining write and destructive capabilities (Dropbox went from 8 to 24 tools, 0 to 4 destructive), permission scopes shifting, injected model instructions appearing, and about 2 in 5 Claude connectors likely calling additional external AI services. Details →First reported · updated · 2 reports arxiv.org
Capability Gates Are Not Authorization: Confused-Deputy Failures in LLM Agent Frameworks
A security research paper, 'Capability Gates Are Not Authorization,' audits LangChain/LangGraph, LlamaIndex, and the Stripe Agent Toolkit and finds that all three provide capability gating by default but none enforce a deterministic fail-closed per-call value authorization gate, enabling classic confused-deputy abuse where an attacker-influenced model emits an unauthorized side-effecting call (e.g., a payout). The authors introduce SCOPEGATE, a five-stage PDP/PEP control (scope, authorization, money ceiling, idempotency, default deny), and report an identical unauthorized payout executing under LangChain's default dispatch but denied by SCOPEGATE, with an available artifact (github.com/raceksd-source/scopegate-runtime). Details →First reported · updated · 4 reports arxiv.org
Agent Data Injection Attacks are Realistic Threats to AI Agents
A research paper by Woohyuk Choi and colleagues introduces agent data injection attacks (ADI), a new category of indirect prompt injection in which malicious data is disguised as trusted data (such as security-critical metadata or agent context data like tool call/response formats) rather than as instructions. The authors demonstrate ADI against real-world agents, achieving arbitrary click attacks on web agents (Claude in Chrome, Antigravity, Nanobrowser) and remote code execution plus supply-chain attacks on coding agents (Claude Code, Codex, Gemini CLI), showing it bypasses existing IPI defenses because agents fail to isolate trusted from untrusted data. Details →First reported · updated · 5 reports arxiv.org
Prompt Injection as Role Confusion
The paper "Prompt Injection as Role Confusion" (arXiv:2603.12277, ICML 2026) by Charles Ye, Jasmine Cui, and Dylan Hadfield-Menell traces prompt injection to role confusion: LLMs perceive the source of text from how it sounds rather than its labeled role, so injected text occupies the same representational space as the trusted role it imitates. The authors introduce role probes to measure internal role perception and demonstrate CoT Forgery, a zero-shot attack injecting fabricated reasoning into user prompts and tool outputs that yields 60% attack success against frontier models with near-zero baselines. Details →First reported arxiv.org
The Injection Paradox: Brand-Level Suppression in Safety-Trained LLM Recommendations via RAG Context Injection
The paper 'The Injection Paradox' by Hyunseok Paeng documents a reproducible failure mode in RAG-based LLM recommendation systems where indirect prompt injections embedded in retrieved documents backfire, suppressing the injected brand below its injection-free baseline. In safety-trained Claude models (e.g., Claude Opus 4.6), a single injected document drops the target brand from a 54% baseline to zero top-2 recommendations and propagates suppression to the brand's uninjected documents, while GPT models instead show increased recommendations. The authors note this enables a reverse-attack scenario where an adversary injects a competitor's documents to suppress that competitor, and release code, prompts, and results. Details →First reported arxiv.org
Devil in the Lens: Analyzing and Defending Physical Prompt Injection Against Vision-Language Models on Wearable Devices
A research paper, "Devil in the Lens," characterizes physical prompt injection attacks against Vision-Language Models on wearable devices such as AI glasses, where malicious text embedded in real-world environments acts as an indirect prompt-injection channel that hijacks model behavior. Using photos captured from AI glasses across 200+ environments, the authors identify 6 threat vectors, evaluate 12 VLMs (achieving attack success rates up to 96% simulated and 60% real-world), and propose two defenses: a mask-based external filter and a semantic-vector-based internal detector. Details →First reported · updated · 3 reports acm.org
Benchmarking and Defending against Indirect Prompt Injection Attacks on Large Language Models | Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.1
PromptShield is a benchmark introduced in an ACM SIGKDD paper for training and evaluating deployable prompt injection detectors, curated to include both conversational and application-structured data. The authors also fine-tune a new prompt injection detector that achieves higher performance in the low false-positive-rate regime than prior schemes. Details →First reported acm.org
Confused Deputy Attack Against Model Context Protocol | ACM Transactions on Software Engineering and Methodology
An ACM TOSEM paper uncovers the "confused deputy attack" against the Model Context Protocol (MCP), where an adversarial server with subtly manipulated metadata overshadows a benign server and intercepts tool invocations without overt malicious behavior. The authors built Puppet, an automated evaluation framework that rewrites benign tool descriptions to hijack tool selection, achieving hijacking rates up to 90.89% and payload execution up to 86.46% across 14 models, while evading MCP-Scan and McpSafetyScanner which cannot detect metadata-level manipulation. Details →First reported proofpoint.com
Account Compromise in the Agentic Workspace | Proofpoint US
Proofpoint threat research reviewing ATO activity across 50M+ accounts (November 2024–November 2025) found 99% of organizations were targeted by account takeover threats, 67% were successfully compromised, and 88% of impacted organizations experienced post-access abuse, with spear phishing succeeding twice as often as non-targeted attacks. The analysis argues that in agentic workspaces a compromised identity extends the blast radius into downstream AI agents, OAuth apps, and automated workflows tied to that identity, so login-only controls no longer suffice. Details →First reported arxiv.org
BraveGuard: From Open-World Threats to Safer Computer-Use Agents
BraveGuard is a self-evolving defense framework, presented in an arXiv paper (arXiv:2606.01166), for training guard models to monitor computer-use agents that interact with files, terminals, browsers, and external tools. The framework mines open-world threat signals, instantiates them as executable agent tasks, and derives trajectory-level supervision; on the AgentHazard benchmark it raised detection accuracy from 38.79% to 82.38% over off-the-shelf guards like Qwen3-Guard and Llama-Guard variants. Details →First reported · updated · 2 reports crowdstrike.com
CrowdStrike Uncovers New Prompt Injection Techniques
CrowdStrike's AI security research team disclosed 18 new additions to its prompt injection taxonomy, expanding coverage to over 200 techniques, and detailed five notably: Trigger-Activated Rule Addition (PT0201), Cognitive Token Suppression (PT0197), Algorithmic Payload Decomposition (PT0200), Special Token Injection (PT0198), and one further method. The techniques target AI agents that crawl webpages, access file stores, and run shell commands, using indirect injection to hide malicious instructions in consumed data. Details →First reported quentinkaiser.be
AI Assisted Vulnerability Research on Embedded Targets | QTNKSR
Security researcher Quentin Kaiser documents an experiment using OpenAI's Codex coding harness (running gpt-5.x models) as an autonomous agent for vulnerability research and exploit development against embedded real-time operating systems, including eCos and Broadcom BFC cable modems. The agent is equipped with skill sets (Trail of Bits marketplace, ghidra-rpc, and a custom eCos offensive-research skill) to navigate firmware, reverse-engineer with Ghidra/radare2, and stage exploits. Details →First reported icml.cc
ICML Poster MultiBreak: A Scalable and Diverse Multi-turn Jailbreak Benchmark for Evaluating LLM Safety
MultiBreak, presented as an ICML 2026 poster by Jialin Song and colleagues, is described as a scalable and diverse multi-turn jailbreak benchmark for evaluating LLM safety. The poster page provides only a truncated abstract, but frames the work as a benchmark contribution for measuring how models withstand multi-turn jailbreak attacks. Details →First reported aclanthology.org
Rethinking Jailbreak Detection of Large Vision Language ...
Researchers from Washington University in St. Louis and Texas A&M propose Representational Contrastive Scoring (RCS), a framework for detecting multimodal jailbreak attacks against Large Vision-Language Models by inspecting the models' internal representations. The two instantiations, MCD and KCD, claim state-of-the-art detection performance and better generalization to unseen attack types while reducing over-rejection, with code released on GitHub. Details →First reported aclanthology.org
Towards Understanding Jailbreak Attacks in LLMs: A Representation Space Analysis
An EMNLP 2024 paper, "Towards Understanding Jailbreak Attacks in LLMs: A Representation Space Analysis" by Yuping Lin et al., studies why some LLM jailbreak strategies succeed and others fail by analyzing how harmful and harmless prompts behave in the model's representation space. The work aims to identify intrinsic properties of successful jailbreak attacks. Details →First reported arxiv.org
Scalable Hierarchical Attention Transformers for Multi-Turn Jailbreak Detection in Long Conversations
Researchers from Zscaler present a hierarchical attention transformer for detecting multi-turn jailbreaks that spread unsafe intent across a conversation via gradual escalation, reframing, and role manipulation. The two-level architecture encodes individual turns with a multilingual encoder then reasons over turn summaries with a lightweight conversation module, reporting an F1 of 0.9394 on a 14,038-conversation benchmark and outperforming a strong LLM baseline while halving false positives. Details →First reported arxiv.org
Behind the Refusal: Determining Guardrail Activation via Behavioral Monitoring
Researchers from Mindgard and Lancaster University (William Hackett, Peter Garraghan) present the first black-box guardrail reconnaissance methodology, which detects whether a target AI system has a guardrail by monitoring HTTP, lexical, and timing signals during benign versus malicious prompt sets. The approach assumes zero prior knowledge and reportedly detects guardrail presence with 100% accuracy, letting adversaries distinguish a guardrail block from an LLM safety rejection to better select bypass techniques. Details →First reported adversa.ai
Solving GitHub’s Secure Code game with an AI red teaming agent
Adversa AI documents pointing its autonomous AI Red Teaming Agent at GitHub's open-source Secure Code Game 'ProdBot' challenge, an intentionally vulnerable agentic-AI teaching target where the goal is to coax the agent into leaking a flag stored one directory above its sandbox. The agent cleared the first three levels (Sandbox, Web, MCP) at full score in a single 57-second run, relying on 'context seeding' — fabricating a plausible prior workflow implying authentication had already occurred — rather than overt jailbreak language, illustrating that each added capability (web browsing, MCP tool chains, skills, multi-agent) opens a new attack surface via indirect injection, poisoned tool chains, and confused-deputy trust. Details →First reported semgrep.dev
Researcher poisons open-weight AI model for under $100
Researcher Katie Paxton-Fear of Manchester Metropolitan University and Semgrep demonstrated that she could install a reliable backdoor into an open-weight AI model in about an hour for under $100, using only ten fine-tuning examples to make the model emit remote-code-execution-vulnerable code even for novel prompts, with larger models being easier to poison. The Register also cites a related PoC by Origin's David Kaplan (originsec/lora-backdoor-poc) that creates a compromised model designed to exfiltrate data via a send_email tool call. Details →First reported · updated · 6 reports paloaltonetworks.com
Phantom Squatting: AI-Hallucinated Domains as a Software Supply Chain Vector
Palo Alto Networks Unit 42 documents "Phantom Squatting," a software supply-chain technique in which attackers register web domains that large language models frequently hallucinate, so that when developers or agents follow LLM-suggested URLs they land on attacker-controlled infrastructure. The research extends the concept of slopsquatting from hallucinated package names to hallucinated domains and measures how LLM temperature and prompting produce consistent, registrable phantom domains. Details →First reported arxiv.org
AI-Generated PowerShell Malware: An Experimental Framework and Dataset
Researchers Pianese, Orbinato, Liguori, and Natella present an experimental framework (arXiv:2606.30819) to assess LLM-generated PowerShell malware, including a novel sandbox for dynamic analysis and a manually curated, natural-language-annotated dataset of real-world PowerShell malware. Their evaluation of permissive open-weight LLMs adapted for malware generation found high similarity between real and AI-generated malware, with a median Jaccard index of 84.5% for triggered OS malicious events. Details →First reported phishfort.com
Multimodal AI Jailbreak Attacks: The Image-Based Threat to Enterprise AI
Researchers at Florida International University (Hadi Amini and Md Jueal Mia) developed JaiLIP (Jailbreaking with Loss-guided Image Perturbation), a technique that embeds instructions into images via pixel-level perturbations invisible to humans but readable by vision-language models. Tested against BLIP-2, JaiLIP nearly doubled the rate of policy-violating outputs, and because most enterprise guardrails only inspect the text channel, the malicious payload in the image tensor bypasses safety filters entirely. Details →First reported aclanthology.org
Data Exfiltration via Backdoored Tool Use
Researchers present Back-Reveal, a data exfiltration attack that embeds semantic triggers into fine-tuned tool-use LLM agents; when triggered, the backdoored agent invokes memory-access tool calls to retrieve stored user context and covertly transmits it to an attacker-controlled server disguised as legitimate retrieval/search API requests. The paper shows multi-turn interaction amplifies leakage, as attacker-controlled retrieval responses steer subsequent agent behavior to induce cumulative disclosure, and evaluates defenses like NeMo Guardrails and LLM Guard as targets. Details →First reported · updated · 2 reports paloaltonetworks.com
TuxBot v3: Inside an IoT Botnet Framework With LLM-Assisted Development
Palo Alto Networks Unit 42 analyzes TuxBot v3, an IoT botnet framework whose development shows signs of LLM assistance, detailing its C2 infrastructure, DGA, XOR-based obfuscation, Docker-compose deployment, and exploitation of numerous IoT CVEs for hyper-volumetric DDoS attacks. The report ties the framework into the broader Aisuru/Kimwolf DDoS botnet ecosystem. Details →First reported arxiv.org
PVDetector: Detecting Prompt Injection Attacks on Purpose-Specific LLM Agents through Policy-Violation Concept Analysis
PVDetector is a training-free framework presented in an arXiv paper for detecting prompt injection attacks against purpose-specific LLM agents. It works by analyzing the model's hidden activation space for latent policy-violation concepts derived from contrastive pairs of policy-violating and policy-compliant prompts, measuring hidden-state alignment during inference. Details →First reported bleepingcomputer.com
We built a vulnerability vending machine: AI tokens in, zero-days out
Intruder describes building an automated pipeline that pairs LLMs with the Joern code-scanning engine and a 'program slice' technique to find and exploit vulnerabilities in production software with no human in the loop. The team reports discovering a remote, multi-stage SQL injection zero-day (CVE-2026-3985) in a WordPress plugin with over 300,000 users, fully automated from discovery through exploitation. Details →First reported arxiv.org
Conflict-Aware Retriever Editing for Knowledge Injection Attacks on LLM-Based RAG Systems
CareAttack is a model-centric attack framework that edits the parameters of open-source dense retrievers (Qwen3-Embedding-0.6B and BGE-M3) to promote malicious passages into the top-k retrieved evidence of RAG systems, bypassing detectable corpus-manipulation approaches. Evaluated on three benchmark datasets, it uses conflict-aware retriever editing and attack-preserving anchor repair to inject malicious knowledge for batches of target prompts while preserving behavior on non-target prompts; code is published. Details →First reported arxiv.org
VEXA_IoT: Autonomous IoT Vulnerability EXploitation using AI Agents
VEXA_IoT is an autonomous multi-agent framework by Swinea et al. that uses LLM-based reasoning combined with offensive security tools (Nmap, Metasploit, bettercap) to perform reconnaissance, plan attack sequences, and exploit IoT vulnerabilities. Evaluated across IoTGoat and Metasploitable environments and ten OWASP IoT attack scenarios, it achieved a 95.0% overall success rate across 260 attack executions with execution times under two minutes. Details →First reported acm.org
When Context Bites: Detecting RAG Poisoning via Document-Level Attention Collapse | Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval
A SIGIR 2026 paper introduces D-SCAN (Document-level Signal Collapse Analysis), a lightweight detection framework that identifies RAG poisoning attacks by monitoring 'Attention Collapse' — a drop in attention entropy as the generator concentrates on injected adversarial documents. The authors show that output-side signals like perplexity fail because poisoned outputs can exhibit false confidence with even lower perplexity than benign ones, and that D-SCAN detects attacks even when they do not alter the final answer. Code is available at github.com/yingtaoren/D-Scan. Details →First reported arxiv.org
An Evaluation of Data Leakage Risks in Tool-Using LLM Agents in Realistic Scenarios
A joint evaluation by the Singapore AI Safety Institute and the Korea AI Safety Institute (arXiv:2606.17114) tested three tool-using LLM agents across 12 realistic, non-adversarial tasks spanning customer support, DevOps, web automation, and productivity, measuring five data-handling risk types. None of the agents achieved fully correct and fully safe execution, with successful task completion often coinciding with data-handling failures such as accessing unnecessary information or disclosing data to inappropriate recipients. Details →First reported icml.cc
ICML Poster An Empirical Study of Memory Poisoning Defenses for LLM Agents
An ICML 2026 poster titled "An Empirical Study of Memory Poisoning Defenses for LLM Agents" presents research evaluating defenses against memory poisoning attacks on LLM agents, with an associated code artifact (AMemGuard on GitHub). Details →First reported akamai.com
AI Reconnaissance: The Missing Layer in Chatbot Security
Akamai's AI security research team describes "AI reconnaissance," a technique in which deployed AI assistants block direct requests for sensitive data but readily leak operational context — capabilities, boundaries, knowledge-base access, and tool/action surface — when asked simple, benign-looking questions. This leaked context lets attackers craft targeted, application-specific prompt injections that bypass model-level safety, and the authors argue runtime guardrails must detect this early probing stage of the attack kill chain. Details →First reported arxiv.org
DualView: Preventing Indirect Prompt Injection in Personal AI Agents
DualView is a defense presented in an arXiv paper that extends untrusted-data tracking from a personal AI agent's context out into the user's environment (file system, shell, network, and other agents) to block indirect prompt injection, including a newly-highlighted 'stored IPI' variant where an agent saves attacker-controlled data and later rereads it as trusted. Deployed as an OpenClaw plugin using only tool hooks, DualView gives each channel an AgentView (untrusted data shown as symbols) and a HumanView (original data), and reportedly blocked every IPI attack on an IPI benchmark and PinchBench while preserving utility. Details →First reported arxiv.org