Research · curated 6 Aug 2026
AI struggles to patch vulns without adult supervision
First reported · updated · 2 reports theregister.com
Coverage timeline
Why it matters
As defenders increasingly turn to AI agents and initiatives like Project Glasswing and Project Daybreak to auto-generate vulnerability patches, this research shows LLMs produce broken or newly-vulnerable fixes over half the time on complex patches, warning against unsupervised automated remediation.
1Password's Off-by-1 Labs studied whether frontier LLMs can generate correct patches for six recently disclosed, novel open-source CVEs, producing 6,080 patches with two cyber-capable reasoning models. They found only 26.0% of patches fully resolved the vulnerability without changing app behavior, while 53.9% failed to fix the flaw, introduced a new vulnerability, or both — dubbed Fix-Like Artifacts with Embedded Defects (FLAWED). The team released tooling, datasets, and a research paper.