Research
Be Careful When Fine-tuning On Open-Source LLMs: Your Fine-tuning Data Could Be Secretly Stolen!
Publication date unknown · Discovered arxiv.org
Page published
Publication date unknown · First observed: 17 Oct 2026
Coverage timeline
Single-source research — one report is available.
Why it matters
Backdoored open-weight LLM checkpoints can turn proprietary fine-tuning into a data-exfiltration channel even when attackers have only API access to the resulting model.
Tsinghua-led researchers demonstrated that a malicious open-weight LLM provider can backdoor a checkpoint to extract downstream fine-tuning prompts using only black-box access to the resulting model. Experiments recovered up to 76.3% of 5,000 private queries, rising to 94.9% with prior knowledge of query beginnings. The attack depends on downstream training computing loss on query tokens; the report describes controlled experiments, not observed exploitation.