Glossary / Law of large numbers
During the Second World War the mathematician John Kerrich found himself in an internment camp in Denmark and flipped a coin 10,000 times. He got 5,067 heads, that is 50.67%: almost exactly half. But counted in coins, heads were 134 ahead of tails. The longer you flip, the closer the share gets to half, while the gap in coins usually just grows. Both facts are true at once, and that is the heart of the law of large numbers.
“In plain words” is hidden on all puzzles.
Five people each flip their own coin. The top chart shows each person's share of heads: the lines squeeze towards 50%. The bottom one shows by how many coins heads are ahead of tails: the lines usually spread away from zero. These are the same flips, only counted differently.
The flips axis is logarithmic: each mark is ten times further than the previous one. At most 100,000 flips per person.
Say 8 of the first 10 flips came up heads: that's 80%. The coin remembers nothing, so from then on heads and tails come equally on average. The 6 extra heads never go away, but they drown in an ever larger number of new flips:
Nobody "pays back" the debt with tails. The share approaches half only because the old surplus becomes an ever smaller part of the whole.
“Rigorous” is hidden on all puzzles.
Repeat the same trial many times, with the attempts not affecting one another. An event has probability p. Then the share of attempts in which it happens approaches p as the number of attempts n grows. More precisely: however small a tolerance you pick, 1% or 0.1%, the chance of falling outside it goes to zero.
Now it's 200 people instead of five, each flipping a coin n times. At the top is everyone's share of heads, at the bottom heads minus tails in coins. Move the slider: the top cloud shrinks, the bottom one spreads.
This is a classic problem by the psychologists Daniel Kahneman and Amos Tversky. One hospital has 15 births a day, another 45. Boys and girls are born about equally often. Where will there be more days in a year with over 60% boys? Most people answer "the same". Check it: each cell is one day, a filled cell is a day with more than 60% boys.
The small hospital usually has twice as many such days: the exact calculation gives 55 days a year on average against 25. A small sample lands on extremes more often. That is why small schools, villages and teams so often turn up among both the best and the worst of any ranking: not always because they are special, but because they have few people.
“Deeper” is hidden on all puzzles.
σshare = √( p(1 − p) / n )
For a coin p = ½, so the spread of the share of heads is 0.5/√n: 5 percentage points after 100 flips, only 0.5 after 10,000. To make the share ten times more precise you need a hundred times more flips. In the 200-player simulation n = – right now: the formula gives –, and the players got –.
H − T = 2H − n · σ(H − T) = √n · E|H − T| ≈ √(2n/π) ≈ 0.8 √n
Each flip adds +1 or −1 to the difference, and these steps are independent. The spreads of independent steps add up, so after n steps the spread of the difference is √n. So the typical gap is about 8 coins after 100 flips and about 80 after 10,000. But divide by the number of flips: 8 out of 100 is 8%, 80 out of 10,000 just 0.8%. The gap grows, but more slowly than n, so the share still converges to half. Kerrich's gap of 134 with σ = 100 is a perfectly ordinary result. In the 200-player simulation at n = – the formula gives – coins, and the average came out at –.
P( |X̄ − p| ≥ ε ) ≤ p(1 − p) / (n ε²)
Pafnuty Chebyshev proved a simple thing: a quantity rarely strays from its mean by more than a few spreads. The chance of straying by ε is at most the variance divided by ε². The variance of the share is p(1 − p)/n and falls as n grows, so for any ε the chance of falling outside the tolerance goes to zero. That is the law of large numbers. Take ε = 5 points: for a coin the bound is 100/n. At n = – Chebyshev guarantees that at most – of the players end up more than 5 points from 50%, and in the simulation it is –. The bound is crude, but it works for any independent identical trials, not just coins.
The middle value: line everyone up in order, and half are below it, half above.
A millionaire walks into a bar: the average income shoots up, the median barely moves.
The sum of all values divided by how many there are: what everyone would get if it were shared equally.
Almost everyone has more legs than average: the average is a little under two.
The average result you can expect if you repeat the same thing very many times.
A die's expected value is 3.5, yet no face shows 3.5.
A number from −1 to +1 showing how closely two quantities move together.
In summer both ice cream sales and drownings go up. Ice cream isn't to blame: the heat drives both.
A measure of spread: how far values typically stray from the mean. The standard deviation is its square root, in the same units.
Two towns with the same average temperature: one stays near +15 all year, the other swings from −20 to +50.
A bell-shaped distribution: most values sit near the mean, and the further away, the rarer.
Every ball on a Galton board falls at random, yet thousands of balls always build the same bell.
The chance of an event given that something else is already known.
Almost all NBA players are tall, but almost all tall people are not NBA players.
Events are independent if knowing about one doesn't change the chances of the other.
After five heads in a row tails is no more likely: the coin remembers nothing.
How many times you multiply the base by itself to get a number: log₂ 8 = 3, because 2 · 2 · 2 = 8.
A sheet of paper folded in half 42 times would be thicker than the distance to the Moon.
A sample some people get into more easily than others, so it describes everyone badly.
In 1936 a magazine polled millions of Americans and still picked the wrong election winner.
How often chance alone would produce a gap at least as big as yours if there were really no difference.
When there is no effect at all, p below 0.05 turns up as often as p between 0.50 and 0.55.
A range around a measured number, built by a method that covers the true value in a chosen share of samples, such as 95%.
Of twenty honest 95% intervals, on average one misses the truth, and none of them knows whether it is the one.
The probability that a test notices a real effect of a given size. A power of 80% means the test misses one working effect in five.
A weak A/B test that does find an effect almost always exaggerates it: a real +10% shows up on the dashboard as +24%.
How far an average or a rate computed from a sample usually misses the truth: σ divided by the square root of the sample size.
Halving the error takes four times as much data.