Named after Simpson
The British statistician Edward H. Simpson described the effect in a 1951 paper on contingency tables. The name "Simpson's paradox" was introduced by Colin Blyth in 1972.
Treatment A works better than treatment B for small kidney stones and for large ones. Put all the patients together, and B comes out ahead. Both statements come from the same numbers. Let's see how that happens and which number to trust.
Share of successful treatments, data from Charig et al., 1986. Pink marks the winner in each row.
Simpson's paradox happens when a trend that holds in every group of data reverses once the groups are combined. The classic example comes from a 1986 British study of kidney stone treatments.
Treatment A: open surgery. Treatment B: percutaneous surgery through a small puncture. Each was given to 350 patients.
Treatment A succeeded in 273 of 350 cases (78%), treatment B in 289 of 350 (83%). At first glance B is better.
Now split the patients into two groups: small stones and large stones. For small stones A gets 93%, B 87%. For large stones A gets 73%, B 69%.
How can A be better in each group and worse overall? And which treatment would you recommend to a patient whose stone size you know?
"If A is better for small stones and for large stones, it must be better for everyone together"
It feels obvious, but it is not a rule of arithmetic. The overall rate also depends on how many patients of each kind a treatment got. Treatment A got mostly the hard cases.
Stone size affects both the choice of treatment and the outcome. Such a variable is called a confounder. More in round 4.
Imagine you skipped the puzzle above and see only the overall results. Pick a treatment, then look at the split by stone size.
Tap the treatment you would choose.
The sliders set the number of patients and successes in each cell of the table. The bars on the right update instantly. Try to make the paradox disappear, or pick another example.
The computer generates thousands of random 2×2 tables and counts the share where one treatment wins in both groups and the other wins overall. The answer depends on how the tables are generated, so there are two different generators here.
Each line shows what a treatment's overall rate would be for a different share of small-stone patients. Line A is above line B everywhere. But the overall rates are points at different places on their lines: A has few easy patients, B has many.
The key is that the two treatments were given to different patients. Here are five ways to see it.
Only 25% of A's patients had small stones, so its overall rate sits close to its 73% for large stones. For B the share is 77%, so its rate sits close to its 87% for small stones. The different weights outweigh A's advantage.
The overall comparison mixes two effects: the real difference between the treatments, and the fact that A more often got the harder patients. Comparing treatments within a group of the same stone size removes the second effect.
Each group is an arrow: right by the number of patients, up by the number of successes. The slope is the success rate. Both of A's arrows are steeper than B's matching arrows. But A has a short steep arrow and a long shallow one, and B has the reverse. So B's path, built head to tail, ends higher. To make the slopes visible, the chart subtracts 70% of the patients from the successes: an arrow with a 70% rate would be flat.
The data alone do not say which table to use. The causal picture decides. Here stone size affects which treatment is given, so for "which treatment is better for a patient" we use the numbers by group. But if a variable is itself a result of the treatment (say, blood pressure after taking a drug), you should not split by it: then the overall number is the right one.
The first two lines show that the overall rate is a weighted average of the group rates, with each treatment's own weights. The last two lines give both treatments the same patient mix, the one in the whole sample (357 small and 343 large). Then the comparison is fair, and A wins. The paradox can only happen when the two treatments have different patient mixes.
Simpson's paradox is not just a medical curiosity. Here are three well-known settings: sports statistics, university admissions, and the case with more than two groups.
Batting average: hits divided by at-bats.
Justice is better in 1995 and in 1996, yet Jeter is better over the two years combined. The reason is the same: in 1995 Jeter had only 48 at-bats, and he spent his better year, 1996, with 582. Justice had it the other way round.
The overall numbers looked like bias against women. In 1975 the statisticians Bickel, Hammel and O'Connell looked at the data department by department. They found no bias against women there, and a small bias in their favour in several departments.
Women more often applied to departments with fierce competition, which rejected most applicants regardless of sex. Men more often applied where many were admitted. The choice of department plays the role of stone size here.
The study showed how the data were distributed, not why women chose those departments. That is a separate question.
Here A has the same edge over B in every group. Groups run from easy to hard, and A gets more of the hard patients while B gets more of the easy ones. 2,000 patients per treatment. Change the number of groups and A's edge.
The British statistician Edward H. Simpson described the effect in a 1951 paper on contingency tables. The name "Simpson's paradox" was introduced by Colin Blyth in 1972.
Karl Pearson (1899) and G. Udny Yule (1903) described similar effects earlier. That is why it is sometimes called the Yule–Simpson effect.
Graduate admissions data from the University of California, Berkeley, for 1973 became the textbook example. The paper by Bickel, Hammel and O'Connell about them appeared in Science in 1975.
The 700-patient example comes from a study by Charig and colleagues, published in the British Medical Journal in 1986. It is now the classic illustration of the paradox in statistics textbooks and courses.
David Justice had a better batting average than Derek Jeter in both 1995 and 1996. Over the two seasons combined Jeter hit .310 and Justice only .270.
Simpson's paradox is a standard example in causal inference. Judea Pearl discusses it at length in The Book of Why (2018), written with Dana Mackenzie: choosing the right table takes a causal model.
Share your own explanation, an objection or a variant of the puzzle. Commenting needs a GitHub account.
I keep adding new puzzles, new ways to explain them and write-ups of the discussions they start. Leave your email and I'll write when something new is out. No spam, one-click unsubscribe.
The sign-up form will appear here soon.