In town, good-looking people are on average neither kinder nor meaner than anyone else. You only agree to a date when looks and kindness together clear your bar. Will looks and kindness be related among the people you date? Lock in your answer, then check it with a game, a simulation and five explanations.
Among your dates the better-looking ones are clearly less kind, even though in town there is no link at all. To clear the bar, a plain-looking person has to be very kind, while a stunning one need not be. So selecting on the sum of two traits turns them into rivals. Check it with a game, a simulation and five explanations.
The mathematician Jordan Ellenberg tells this story, and the biostatistician Joseph Berkson described the effect in 1946 using hospital data. First the model, then the question.
The mathematician Jordan Ellenberg tells this story, and the biostatistician Joseph Berkson described the effect in 1946 using hospital data. The paradox is that the link you see is real in your data, yet the selection itself created it.
Every person has two traits: looks and kindness. Both are spread like ordinary scores: most people are in the middle, few are at the extremes.
In town the two traits are independent. Knowing how good-looking someone is tells you nothing about their kindness. The correlation is zero.
You agree to a date only when looks plus kindness is high enough. 30% of the town clears the bar. A stunning person can be not so kind, and a very kind one not so stunning.
What is the correlation between looks and kindness among the people you date?
"The traits are independent, so there will be no link among my dates either"
In fact the correlation among your dates is −0.58. The bar cuts off the people who are low on both traits. Among those who are left, low looks almost always come with high kindness, and the other way round.
Both traits are normally distributed with the same spread. You can move the bar and change the selection rule in rounds 2 and 5.
In town the correlation between looks and kindness is zero. You only date the 30% of people with the highest looks plus kindness. What is the correlation between looks and kindness among them?
Pick an answer. A simulation of a town of 5,000 people and your dates will appear here.
Hard to decide? Open the hints one at a time: the easy one first, then stronger ones.
Think about who makes it onto your list of dates. What does someone who is not good-looking at all have to be like to clear the bar anyway?
The bar is on the sum of two traits. A low score on one has to be made up by a high score on the other. Among those who pass, what happens to kindness as looks go down?
Imagine each trait is scored from 0 to 10 and you need a total of at least 18 to pass. How kind is someone with looks 8? And someone with looks 10?
Each dot is a person in town: looks across, kindness up. Drag the dashed bar on the chart or use the slider. The dots above the bar are your dates. Watch how the correlation among them changes as you get pickier.
| Your dates, split by looks | Average kindness |
|---|
Kindness is shown in standard deviations from the town average: 0 is a typical person, +1 is kinder than 84% of the town. The chart has only 400 people, so the correlation in the game jumps a little. The theoretical value is shown next to it.
We generate a town of 100,000 people, sort them by looks plus kindness and compute the correlation among the top 70%, 50%, 30% and so on down to 1%. The line is the exact value from the formula in round 4, the dots are the simulation. The higher the bar, the closer the correlation gets to −1.
Exact correlation for different bars:
The key is that the selection depends on both traits at once. Here are five ways to see it. The numbers in the first two cards come from the round 3 simulation with a 30% bar.
In town the cloud of dots is round. The bar cuts off its lower left part, and what remains is a band running from top left to bottom right. A plain-looking person makes your list only if they are very kind. So on that list low looks almost always come with high kindness.
The person has already cleared the bar, so something got them over it. If they are not very good-looking, the "reason" must be kindness. If they are stunning, looks explain it and kindness can be anything. Statisticians call this explaining away. Values are in standard deviations: 0 is a typical person in town.
| In town | Admitted | Of them with B | |
|---|---|---|---|
| Have A | 1 000 | 525 | 14.3% |
| Don't have A | 9 000 | 1 260 | 35.7% |
A town of 10,000 people. Disease A (say, diabetes) affects 10%, disease B (gallbladder inflammation) also 10%, independently of A. People with neither are admitted for other reasons with probability 10%, with one disease 50%, with both 75%. In town one in ten has B, with A or without. In hospital, patients with A have B less than half as often as the other patients.
Arrows run from both traits to the date: it is a "collider", the point where two causes collide. While you look at the whole town, the path between looks and kindness through the collider is closed. As soon as you keep only the people you date, the path opens (dashed) and a link appears with no cause behind it.
Select on looks alone and the correlation stays at zero: nobody checked kindness. Require each trait separately to be above average ("both") and it also stays at zero for independent traits. A negative link appears when one trait can make up for the other: a sum above a bar, or "at least one of the two".
Round 5 has a lab where you can switch the selection rule and check this yourself.
Looks X and kindness Y are independent standard normals. You keep people whose sum S is above a bar c. The difference D does not depend on S, because their covariance is Var X − Var Y = 0.
Since X = (S + D)/2 and Y = (S − D)/2, the selection squeezes the spread of S and leaves the spread of D alone. Let v be the variance of S among the selected.
The variance of a normal tail: here a is the bar in standard deviations of S, and φ and Φ are the standard normal density and distribution function.
A 30% bar: a = Φ⁻¹(0.7).
The hospital table from the card above: the share with B among patients with and without A.
The higher the bar, the smaller the spread of the sum among the selected, the closer v gets to zero and r to −1. With no bar v = 2 and r = 0. If the traits in town have correlation ρ, the formula becomes r = ((1 + ρ)w − (1 − ρ)) / ((1 + ρ)w + (1 − ρ)), where w = 1 + aλ − λ²: a high enough bar turns even a positive correlation negative.
From dating to the big picture in three steps: first the maths (which selection rules create a link and how strong it is), then the logic (why a common effect ties its causes together), then real studies where this has already happened.
| Diagram | Not conditioning on C | Selecting or splitting on C |
|---|---|---|
| A → C → B | linked | link disappears |
| A ← C → B | linked | link disappears |
| A → C ← B | not linked | a link appears |
In "The Book of Why" Judea Pearl reduces all such stories to three diagrams. In a chain and a fork (a common cause, like stone size in Simpson's paradox) splitting on C removes a spurious link, and that is the right thing to do. A collider is the reverse: splitting on a common effect creates a link. That is why the advice "control for every variable" is dangerous: controlling for a collider breaks the analysis.
The battery is flat with probability 10%, the tank is empty with probability 10% too, independently. The car won't start if at least one happens.
| Flat battery, knowing nothing | 10% |
| … the car won't start | 52.6% |
| … won't start, and the tank is empty | 10% |
| … won't start, and the tank is full | 100% |
An empty tank "explains" the failure and clears the battery. Tank and battery are independent, yet among cars that won't start they are linked. It is the same mechanism as with your dates.
The model parameters are illustrative: they show the mechanism, not the numbers from the studies.
Joseph Berkson of the Mayo Clinic published "Limitations of the Application of Fourfold Table Analysis to Hospital Data" in Biometrics Bulletin in June 1946. He showed that two diseases can look linked among patients just because each of them leads to hospital. His example: diabetes and gallbladder inflammation (cholecystitis).
For 30 years, from 1934 to 1964, Berkson headed the Division of Biometry and Medical Statistics at the Mayo Clinic. In 1944 he introduced the logit model and the word "logit" itself. Today it underlies logistic regression, one of the most widely used methods in statistics and machine learning.
Berkson knew well how the selection of patients distorts conclusions, and he long doubted that smoking causes lung cancer. Even after the US Surgeon General's report of 1964, Life magazine quoted him as saying it was very doubtful that smoking causes lung cancer. Today the link is considered proven.
That is the title of an essay by the mathematician Jordan Ellenberg, and the same story is in his book "How Not to Be Wrong" (2014). He draws a "great square" of men by niceness and looks and shows that among the men the heroine dates, the handsome ones turn out less nice for no real reason.
Among low-birth-weight babies, those of smoking mothers die less often than those of non-smokers. In 2006 Sonia Hernández-Díaz, Enrique Schisterman and Miguel Hernán explained it as selection on a collider: low weight is caused either by smoking or by other, often more dangerous, causes. Smoking harms babies at any weight.
Early in the pandemic, some studies of hospital patients found suspiciously few smokers among them, and headlines suggested that smoking protects against COVID-19. Gareth Griffith, Gibran Hemani and colleagues showed in Nature Communications (2020) how such conclusions can arise from the way participants are selected.
Share your own explanation, an objection or a variant of the puzzle. You need a GitHub account to comment.
I keep adding new puzzles, new ways to explain them and write-ups of the discussions they start. Leave your email and I'll write when something new is out. No spam, one-click unsubscribe.
The sign-up form will appear here soon.