Threat · curated 23 Sep 2026
Attackers Manipulate AI Chatbots in Mass Disinformation, Phishing Campaign
First reported darkreading.com
Coverage timeline
Single-source incident — first reported, latest, and curated coincide.
Why it matters
The "Dark Sourcery" campaign shows attackers can weaponize search-connected AI assistants at scale to deliver phishing and disinformation while impersonating trusted brands, turning AI answers into a mass social-engineering vector.
Researchers from Vigilance Security identified a campaign dubbed "Dark Sourcery" that poisons AI chatbots including OpenAI's ChatGPT, Google Gemini, and Google AI Overview by seeding the web with optimized posts, PDFs, reviews, and fake support pages. The manipulated content causes the chatbots to serve users fraudulent phone numbers, email addresses, and phishing login pages as trusted facts.
Summary
Researchers at Vigilance Security identified an ongoing content-manipulation campaign, dubbed 'Dark Sourcery,' in which threat actors poison the answers produced by ChatGPT, Google Gemini, and Google AI Overview by seeding the web with malicious links and disinformation and then optimizing that content for retrieval.[1]
Rather than issuing prompts or instructions to the AI, attackers flood the web with optimized posts, PDFs, reviews, and fake support pages so that chatbots present fraudulent phone numbers, email addresses, and login pages to users as trustworthy facts, effectively bypassing AI defenses tuned for prompt injection.[1]
The campaign has swept up at least 374 companies — including Fortune 100 organizations, major airlines, banks, travel companies, and software providers such as Delta, Lufthansa, Qatar Airways, Chase, Bank of America, Airbnb, and TripAdvisor — and Vigilance has already observed victims scammed into giving up payment details via chatbot-supplied phone numbers.[1]
Attack chain
- Content seeding: Attackers flood the web with carefully optimized posts, PDFs, reviews, and fake support pages containing fraudulent phone numbers, email addresses, login pages, and software-update information — tens of thousands of malicious pages in total.[1]
- Amplification / SEO poisoning: Attackers apply search engine optimization and other content-distribution techniques, reportedly leveraging high-authority domains (universities, government) combined with public opinion sources (social media, forums, reviews) to raise the likelihood that AI models retrieve and trust the content.[1]
- AI answer poisoning: AI chatbots (ChatGPT, Gemini, Google AI Overview) retrieve the seeded content and present the fraudulent details as helpful, factual guidance within their answers, without the user necessarily viewing the source page.[1]
- Victim contact and fraud: Users act on chatbot-supplied fraudulent phone numbers or login pages; researchers who called the numbers reached representatives who offered to 'move our flight' or 'unlock our bank account' and requested credit card details, with real victims reportedly scammed into surrendering payment data.[1]
Disclosure timeline
| Date | Event |
|---|---|
| August 2026 | Exploding Topics study cited by Vigilance published, finding 91% of AI chatbot users do not verify answers.[1] |
| September 23, 2026 | Vigilance Security research on the 'Dark Sourcery' campaign published, and Dark Reading article reporting the findings released.[1] |
How it works
The technique exploits the trust chain of retrieval-augmented AI answers: attackers do not send prompts or instructions to the model but instead poison the external content the model retrieves. Because the disinformation is displayed as fact rather than delivered as an instruction, it bypasses defenses designed to block prompt injection, and the fraudulent phone number, email, or login page becomes part of the AI's own answer.[1]
Retrieval bias amplifies the attack: models are said to trust content more when it originates from authoritative entities, so attackers combine high-authority domains with public-opinion sources to increase the chance their seeded content is retrieved and repeated. Blind user trust in AI — 91% of users reportedly do not verify answers — compounds the impact.[1]
Affected versions and patch status
| Product | Affected | Patch status |
|---|---|---|
| OpenAI ChatGPT, Google Gemini, Google AI Overview | Answer-generation affected when models retrieve and relay attacker-seeded web content; no version specifics provided. | No vendor fix described; campaign reported as ongoing.[1] |
Key takeaways
- AI answer poisoning is a distinct threat from prompt injection: by manipulating retrievable web content rather than issuing instructions, attackers make disinformation part of the model's answer and bypass instruction-focused defenses.[1]
- Blind trust in AI is the core enabler — with 91% of users reportedly not verifying answers, controlling AI outputs effectively lets attackers influence user behavior at scale.[1]
Defensive actions
- Users should verify AI-provided results — especially phone numbers, email addresses, and URLs — against verified company records rather than trusting chatbot answers as infallible.: Fraudulent details are presented by the AI as factual guidance, and researchers confirmed some chatbot-supplied numbers led to scam operators requesting payment data.[1]
- Targeted brands should take customer scam complaints seriously and monitor the AI answers customers receive along with the sources those answers cite, prioritizing support, account recovery, refunds, payments, and software downloads.: Content manipulation causes reputational damage and directs victims to fraudulent contact points; monitoring cited sources helps detect poisoned content and repeated indicators across unrelated sites.[1]
- Organizations using AI chatbots and agents should monitor agents at runtime to verify every source and piece of content entering their context, and verify critical details such as phone numbers, links, software packages, and command lines.: Poisoned external content can enter agent workflows and propagate erroneous or malicious answers to employees.[1]