How can something win in every group but lose overall?
A kidney treatment can beat its rival for small stones and for large stones, yet look worse when you add the two together.
▶ Start the storyIt happens when the groups being combined are very different in size and difficulty. This is Simpson's paradox: a trend that shows up in every group can vanish or flip once you pool the groups together.
A real medical study of kidney stone treatments shows how. Treatment A worked better than treatment B on small stones, and also better on large stones. Yet when all patients were added up, B looked better. The reason: doctors tended to give A to the hard cases, large stones, and B to the easy ones, small stones. B's overall score was flattered by its easy patients.
A toy example makes the arithmetic visible. In week one, Lisa edits 60 of the 100 articles she reads, while Bart edits 9 of 10: Bart wins, 90% to 60%. In week two, Lisa edits 1 of 10, Bart 30 of 100: Bart wins again, 30% to 10%. But over both weeks, each read 110 articles, and Lisa edited 61 while Bart edited only 39. Lisa wins overall.
Share of articles edited
| Lisa | Bart | |
|---|---|---|
| Week 1 | 60% | 90% |
| Week 2 | 10% | 30% |
| Both weeks | 55% | 35% |
The lesson: before trusting a pooled statistic, ask what hidden factor might differ between the groups.
Quiz me
0/3
Recap
Before trusting a total, ask whether the groups inside it differ in size and difficulty.
Surprising fact · Treatment A beat treatment B for both small and large kidney stones, yet B looked better overall.
Sources (2)
No source, no claim. Every fact in this lesson (15 claims) cites at least one of these.