Threat
OpenAI works to stop ChatGPT generating 'sex crime scene' images
Publication date not yet evaluated · Added bbc.com
Page published
Coverage timeline
Single-source incident — one report is available.
Why it matters
The Grok/ChatGPT jailbreak shows that trivial, innocuous-looking prompts can defeat AI image-generation safety filters at scale, enabling harmful and potentially illegal synthetic content including deepfakes.
Researchers at Mindgard demonstrated that a simple, slightly-altered prompt jailbreaks SpaceXAI's Grok (and previously OpenAI's ChatGPT/GPT-5.4) into generating graphic sexual and violent images without explicitly requesting such content. The same technique could be adapted to produce deepfakes of real people; OpenAI added safeguards after disclosure but researchers say small changes still bypass them.