AI Agent Memory Poisoning: Persistent Agent Attacks

Summary

This is a defensive analysis of memory poisoning against LLM-based AI agents, arguing that the defining danger over prompt injection is persistence: an adversary who writes a single malicious record into an agent's long-term memory establishes a durable foothold that is retrieved and executed thousands of times later, in sessions the attacker was never present for.^[0]

The piece grounds the threat in published red-team research — AgentPoison, which backdoors generic and RAG-based agents by poisoning memory or knowledge bases at scale, and MINJA, which injects malicious records into an agent's memory using only ordinary queries and observed outputs, requiring no privileged access to the store.^[0][5][6]

It maps the attack surface to a three-tier memory model (episodic, semantic, procedural) plus shared cross-tenant stores, and prescribes an architectural fix rather than an input filter: authorise writes outside the model, stamp provenance on every record, retrieve by similarity weighted by trust, and quarantine new and consolidated writes until consistency checks against high-trust facts pass.^[0]

How it works

Memory poisoning is the adversarial insertion, mutation, or deletion of entries in an agent's persistent memory so that later retrievals change the agent's behaviour. It touches no model weights — the model is fine while the state around it is corrupted — and its defining property is persistence across sessions rather than a one-shot hijack.^[0]

Two write channels exist. Direct writes occur when the agent exposes a save_memory-style tool and untrusted content reaches it (for example a user instructing the agent to 'remember' a false policy, or a retrieved web page being summarised into a durable note). Indirect writes are the MINJA contribution: the attacker never calls a write tool but crafts a sequence of benign-looking queries whose reflections the consolidation pipeline stores, gradually shifting the semantic neighbourhood future queries land in.^[0]

MINJA uses bridging steps to link victim queries to malicious reasoning steps and an indication prompt with a progressive shortening strategy, so the malicious record is easily retrieved when later victim queries are processed — enabling any ordinary user to influence agent memory via query-only interaction.^[6]

AgentPoison frames trigger generation as constrained optimization that maps triggered instances to a unique embedding space, so a user instruction containing the optimized backdoor trigger reliably retrieves malicious demonstrations while benign instructions retain normal performance; it requires no additional model training or fine-tuning.^[5]

The three memory tiers fail differently: a poisoned episodic write fabricates a past interaction (durable but limited to one user), a semantic write installs a false fact for everyone sharing the store, and a procedural write rewrites a trusted routine affecting every future execution. Shared cross-tenant stores make the isolation boundary invisible at the vector-similarity layer, creating a cross-tenant instruction channel.^[0]

Key takeaways

• Memory poisoning is an architecture problem, not a model problem: no better model fixes a design that lets untrusted content become trusted persistent state, so memory writes must be treated as a security boundary.^[0]
• The attacker economics differ fundamentally from prompt injection — one successful write yields a persistent, dormant foothold retrieved against future users — and published research (AgentPoison, MINJA, and a systematic taxonomy of six attack classes) shows the technique is practical and that existing prompt-injection defenses do not cover it.^{[0][5][6][27]}
• Shared cross-tenant memory stores built for richer personalisation are the highest-loss failure mode, effectively a cross-tenant instruction channel; isolate by default and share only provenanced, verified facts.^[0]
• Common anti-patterns give a false sense of security: input sanitisation, single-trust-level stores, auto-consolidating reflections into shared memory, and implicitly trusting procedural routines each leave the persistent write path open.^[0]

Defensive actions

• Authorise memory writes in code outside the model, barring untrusted-source content (web, uploaded docs, other tenants' turns) from reaching semantic or procedural tiers without a separate verification step.: Input filters lose to encoding and paraphrase; the decision of whether content may become durable memory must live where the model cannot talk its way past it, and users must be prevented from installing global facts.^[0]
• Stamp provenance on every memory record — who wrote it, from what source class (first-party, user, retrieved-untrusted), under which session, and when.: Without provenance a false fact is indistinguishable from a true one at retrieval time; with it, retrieval can down-weight untrusted classes and incident response can determine what an attacker touched in a single query.^[0]
• Replace pure cosine-similarity retrieval with trust-weighted retrieval (similarity × trust) and gate procedural memories behind the highest trust floor.: A high-similarity record from an untrusted source should lose to a slightly less similar first-party record, and procedural memories steer actions so must require the strongest trust before firing.^[0]
• Route new writes — especially consolidation-generated reflections — into a quarantine tier retrievable only by the writing user, promoting to shared memory only after consistency checks against high-trust facts, and flag writes that contradict established high-trust facts.: Consolidation is the exact write primitive MINJA exploits; the contradiction rule breaks most MINJA-style gradual shifts, and quarantine prevents untrusted reflections becoming shared trusted state.^[0][6]
• Detect existing poisoning through provenance audits (count records by source class), contradiction scans (embed high-trust facts and search for near-neighbours asserting the opposite), and retrieval-anomaly detection (records suddenly retrieved across many unrelated users or co-occurring with tool-call failures and refunds).: Teams inherit stores built before these controls; because a single poisoned note rides silently into a large context, the actionable signal is the retrieval pattern, not the payload size.^[0]