Research · curated 6 Aug 2026
Why AI-generated vulnerability patches still require expert human review
First reported 1password.com
Coverage timeline
Single-source research — first reported, latest, and curated coincide.
Why it matters
With defenders increasingly turning to AI agents to auto-generate security fixes (e.g. OpenAI's Project Daybreak), this study shows unsupervised LLM patches are frequently incomplete or actively harmful, underscoring the need for human review before deploying agent-generated remediations.
Researchers at 1Password's Off-by-1 Labs studied whether frontier LLMs (ChatGPT 5.5 and Claude Opus 4.8) can autonomously patch vulnerabilities, generating 6,080 patches across six recently disclosed CVEs. Only 26% of autonomous patches fully and cleanly fixed the flaw, while 53.9% failed to remediate, changed application behavior, or introduced new vulnerabilities; the team coined the acronym FLAWED (Fix-Like Artifacts With Embedded Defects) and released an evaluation harness, datasets, and paper.