Glossary / Independence
On 18 August 1913, at the Monte Carlo casino, the roulette ball landed on black 26 times in a row. The longer the streak lasted, the surer the players were that red was about to come, and the more they bet on it. They lost a great deal of money. A roulette wheel doesn't remember its previous spins, and neither does a coin. That is independence: the past doesn't change the chances of the future.
“In plain words” is hidden on all puzzles.
We flip a coin many times. Every time a few heads in a row came just before a flip, we write down what comes next. If tails really were "due", the right column would be taller.
A different game: we draw cards from a shuffled deck and don't put them back. If the first few cards are red, fewer red cards than black remain in the deck. Here the next card really does "remember" the earlier ones, and the chance of red drops.
“Rigorous” is hidden on all puzzles.
Two events are independent if knowing about one doesn't change the chances of the other. Formally:
P(A and B) = P(A) · P(B)
Roll one die. A is "even" (2, 4, 6), chance ½. B is "1 or 2", chance ⅓. Both together means just a two, chance ⅙ = ½ · ⅓. So A and B are independent, even though it is the same roll: knowing you rolled 1 or 2, you still have exactly half a chance of even. Now C is "1, 2 or 3", chance ½. A and C together is again just a two, ⅙, but ½ · ½ = ¼. These events are dependent: if you know the roll is at most 3, even has only a one-in-three chance.
Two neighbours don't coordinate with each other. Each takes an umbrella on about a third of days. So do both take one on ⅓ · ⅓ ≈ 11% of days? No: the rain pushes both. Each cell is one day out of a thousand: coloured means both have umbrellas, grey means only one.
The model: rain on 30% of days; when it rains each takes an umbrella with a 90% chance, in dry weather 10%. That makes 34% of days for each.
In 1999 in the UK, the solicitor Sally Clark was convicted of murdering her two baby sons, who had died one after the other. The paediatrician Roy Meadow told the court that the chance of two sudden infant deaths in one well-off family was 1 in 73 million. He took the estimate for one death, 1 in 8,543, and multiplied it by itself.
“Deeper” is hidden on all puzzles.
P(A₁ ∩ … ∩ Aₙ) = P(A₁) · … · P(Aₙ) · P(n heads in a row) = (½)ⁿ
Five heads in a row, seen in advance, is rare: (½)⁵ = 1/32, about 3%. But after four heads have already come, the fifth has the chance P(5 | 4) = (1/32) / (1/16) = ½. It was the whole run that was rare, not the last flip. In the first simulation a run of – heads before a flip happened in a share – of all flips, while the formula (½)ᵏ gives –. Monte Carlo is the same: 26 blacks in a row on a wheel with 18 black pockets out of 37 have the chance (18/37)²⁶, about 1 in 137 million, counted in advance. Yet the chance of a 27th black after 26 is the same 18/37, about 49%.
Flip two coins. A is "the first is heads", B "the second is heads", C "both are the same". Each event has chance ½. Any two of them are independent: for example, A and C together means "both heads", ¼ = ½ · ½. But all three together is also just "both heads", ¼, not ½ · ½ · ½ = ⅛. Knowing A and B, you know C for sure.
| Coins | A | B | C |
|---|---|---|---|
| H H | ✓ | ✓ | ✓ |
| H T | ✓ | · | · |
| T H | · | ✓ | · |
| T T | · | · | ✓ |
That is why independence of several events means more than independence of each pair: the product rule must hold for every subset of them.
If two quantities are independent, their correlation is zero. The reverse doesn't hold. Let x be spread evenly from −1 to 1, and y = x². The correlation is exactly zero, because the right half of the parabola rises and the left half falls. Yet y is completely fixed by x: knowing x, you know y without any error. Correlation sees only a straight-line link, while independence demands no link of any kind.
The middle value: line everyone up in order, and half are below it, half above.
A millionaire walks into a bar: the average income shoots up, the median barely moves.
The sum of all values divided by how many there are: what everyone would get if it were shared equally.
Almost everyone has more legs than average: the average is a little under two.
The average result you can expect if you repeat the same thing very many times.
A die's expected value is 3.5, yet no face shows 3.5.
A number from −1 to +1 showing how closely two quantities move together.
In summer both ice cream sales and drownings go up. Ice cream isn't to blame: the heat drives both.
A measure of spread: how far values typically stray from the mean. The standard deviation is its square root, in the same units.
Two towns with the same average temperature: one stays near +15 all year, the other swings from −20 to +50.
A bell-shaped distribution: most values sit near the mean, and the further away, the rarer.
Every ball on a Galton board falls at random, yet thousands of balls always build the same bell.
The more repetitions, the closer the share of an outcome gets to its true probability.
The share of heads creeps towards 50%, while the gap between heads and tails in coins usually grows.
The chance of an event given that something else is already known.
Almost all NBA players are tall, but almost all tall people are not NBA players.
How many times you multiply the base by itself to get a number: log₂ 8 = 3, because 2 · 2 · 2 = 8.
A sheet of paper folded in half 42 times would be thicker than the distance to the Moon.
A sample some people get into more easily than others, so it describes everyone badly.
In 1936 a magazine polled millions of Americans and still picked the wrong election winner.
How often chance alone would produce a gap at least as big as yours if there were really no difference.
When there is no effect at all, p below 0.05 turns up as often as p between 0.50 and 0.55.
A range around a measured number, built by a method that covers the true value in a chosen share of samples, such as 95%.
Of twenty honest 95% intervals, on average one misses the truth, and none of them knows whether it is the one.
The probability that a test notices a real effect of a given size. A power of 80% means the test misses one working effect in five.
A weak A/B test that does find an effect almost always exaggerates it: a real +10% shows up on the dashboard as +24%.
How far an average or a rate computed from a sample usually misses the truth: σ divided by the square root of the sample size.
Halving the error takes four times as much data.