Every result is part skill and part luck. Whoever comes out on top was usually lucky, so next time they drift back towards the average. Whoever comes last moves up. It happens with praise, with criticism and with no feedback at all. Check it with a game, a simulation and five explanations.
Francis Galton gave the effect its name in an 1886 paper on the heights of parents and children. The paradox lies in the conclusions we draw: we see a cause where there is only chance.
Each player has a hidden skill. It does not change from round to round. Nobody can see it, not even the coach.
A score in a round is skill plus luck. The luck is new every time: some have a good day, some don't.
After round 1 the coach praises the best and scolds the worst. Or says nothing.
What happens to the best and the worst in round 2? And does it prove anything about praise and criticism?
"The praised ones relaxed and the scolded ones pulled themselves together. So criticism works better"
In fact the best drop and the worst rise even when the coach says nothing. The best of round 1 are mostly the ones who got lucky. In round 2 the luck is fresh, and their score moves back towards their real level.
Scores have mean 50 and standard deviation 10. You can change how much of them is luck in round 3.
A flight instructor praises cadets for good landings and criticises them for bad ones. He has noticed that after praise the next landing is usually worse, and after a telling-off it is usually better. He concludes that criticism works and praise backfires. What do you say?
Pick an answer. A simulation of 5,000 cadets who were told nothing will appear here.
30 players have played round 1. Magenta marks the top five, black the bottom five. Choose what to tell the team, then look at round 2. Play several groups with and without feedback: the table on the right compares them.
| Groups | Top 5 | Bottom 5 |
|---|
Theory: on average the top five of 30 score 64.5 in round 1 and 57.2 in round 2. The bottom five score 35.5 and 42.8. In this game skill and luck weigh the same.
Each dot is a player: round 1 across, round 2 up. If scores depended on skill alone, the dots would sit on the dashed diagonal. The more luck, the flatter the line: its slope equals the correlation between rounds.
Slope r for different luck shares:
The key is that we select people by their score, and a score mixes skill with luck. Here are five ways to see it. The numbers in the first two cards come from the round 3 simulation.
Take the best players of round 2 and look at how they did before. They were worse then too. Time plays no role here, and nobody got "tired of success". Any selection on one measurement shows regression in the other.
To know whether praise works, compare two similar groups: one is praised, the other is not. The effect of praise is the difference between the two changes. Here it is zero. Without a control group, the 7-point drop is easy to blame on the praise.
Regression does not mean bad luck must follow good luck. Luck in round 2 is zero on average. The best fall back to their real level and, on average, no further. Their round 2 is still above the overall average.
And the spread does not shrink: round 2 has new leaders and new laggards, who were lucky or unlucky this time.
The expected round 2 score sits only a fraction r as far from the mean as round 1 did. The rest of the gap was luck. With no luck (r = 1) there is no regression. With pure luck (r = 0) the expected round 2 score is the overall mean, whatever round 1 was.
Regression appears wherever people are picked for one extreme measurement and then measured again. Pick an example, or set the luck share and the size of the selected group. The example parameters are illustrative; they only show the mechanism.
Francis Galton compared the heights of adult children with those of their parents. Children of very tall parents were tall on average, but shorter than their parents, and children of short parents were taller than theirs. In an 1886 paper he called this "regression towards mediocrity".
Linear regression, one of the main tools of statistics, takes its name from Galton's work on height. The word stuck, even though most regression models have nothing to do with drifting back to the mean.
In "Thinking, Fast and Slow" (2011) Daniel Kahneman tells how he explained to Israeli Air Force instructors that praise teaches better than punishment. One instructor objected: after praise cadets flew worse, and after shouting they flew better. Kahneman realised the man was seeing regression to the mean.
In the US people talk about the "Sports Illustrated cover jinx": after appearing on the magazine's cover, an athlete often does worse. Athletes make the cover after an exceptional run in which luck was also on their side. The next run is closer to their usual level.
In a 1933 book the economist Horace Secrist found that the most successful firms became more ordinary over time and the weakest ones improved, and concluded that business tends towards mediocrity. In a review, the statistician Harold Hotelling pointed out that this was regression to the mean.
The trial of streptomycin for tuberculosis, published by the UK Medical Research Council in 1948, is regarded as one of the first randomised controlled trials. Comparing with a control group separates the effect of a drug from regression to the mean and the placebo effect.
Share your own explanation, an objection or a variant of the puzzle. Commenting needs a GitHub account.
I keep adding new puzzles, new ways to explain them and write-ups of the discussions they start. Leave your email and I'll write when something new is out. No spam, one-click unsubscribe.
The sign-up form will appear here soon.