Where the paradox comes from
Carl Hempel described it in "Studies in the Logic of Confirmation", published in two parts in the journal Mind in 1945. Early versions of the idea appeared in his work from 1937.
The statement "all ravens are black" means the same as "everything that is not black is not a raven". You see a red apple. It is not black and not a raven, so it fits the second statement. Does the apple then confirm that all ravens are black? The philosopher Carl Hempel asked this in 1945. It sounds absurd: you could do ornithology without leaving the kitchen.
The apple really is evidence, but very weak evidence. In a park with 100 ravens and 10,000 non-black things, one red apple weighs about a hundredth of a black raven: the chance of the rule rises from 50% to 50.0025%, and after a black raven to 50.25%. A white raven, if there is one, hides among the ravens as 1 in 100, and among the non-black things as 1 in 10,000. So a check with an apple could almost never fail, and that is exactly why it weighs almost nothing. Hempel's logic is right. The intuition "the apple is irrelevant" is right about the size, but wrong about zero.
This is one of the best-known paradoxes in the philosophy of science. It asks what should count as confirming a rule at all. To answer with a number rather than words, take a small imaginary park.
"All ravens are black." The natural way to check it: look at ravens. Every black raven is an example that fits the rule, and it seems to confirm the rule a little.
"Everything that is not black is not a raven." This is the same rule said the other way round: if one is true, so is the other, and a single white raven breaks both. Logicians call such a pair a contrapositive.
You pick a random non-black thing, and it is a red apple, not a raven. It fits the second rule just as a black raven fits the first. If an example that fits a rule confirms it, then the apple confirms "all ravens are black" too.
The park has 100 ravens and 10,000 non-black things: apples, leaves, flowers, benches. You don't know whether all the ravens here are black. Maybe all 100 are black, or maybe 99 are black and one is white. Does the red apple confirm that all ravens are black, and by how much compared with one black raven?
"An apple says nothing about ravens"
Almost. If a white raven existed, it would be one of the 10,000 non-black things. A random non-black thing could have turned out to be it with a chance of 1 in 10,000, and it turned out to be an apple. This is a check that could hardly fail, so the confirmation it gives is almost zero. Almost, but not zero.
A park with 100 ravens and 10,000 non-black things. A random non-black thing turned out to be a red apple. Pick an answer.
A breakdown of your answer will appear here.
Hard to decide? Open the hints one at a time: the easy one first, then stronger ones.
Only a white raven breaks the rule. Think about where it could turn up: among the ravens or among the non-black things.
If there is a white raven, a random raven turns out to be it with a chance of 1 in 100. What about a random non-black thing?
A check confirms a rule the more strongly, the more easily it could have failed. Compare 1 in 100 with 1 in 10,000.
The next rounds show how much the red apple weighs: a secret park, a million simulated parks and the explanation. Answer round 1 first, or go on?
You chose to always show all rounds.
The game has secretly decided which park this is: all 100 ravens black, or one white raven among them, each with a 50% chance. Look at random ravens or random non-black things. The bar shows how likely it is that all ravens are black, given everything you have seen. The scale shows how many "apples" your observations weigh. Try to find out the truth with as few looks as possible.
It gives the solution away, so make your own guess first.
The simulation builds park after park. Every other one (at random) has a white raven. In each park it makes k random looks of one kind. As in the puzzle, whatever is looked at goes back: 100 looks at ravens do not mean you have seen all 100, some you will see twice and some not at all. For checks without repeats see round 5. Then it keeps only the parks where no look showed a white raven and counts what share of them really have no white raven. That is the chance of the rule after k such observations. The formula gives the same number exactly.
It shows the answer right away.
Across: how many observations; up: the chance that all ravens are black. The black line shoots up to 100% within a few hundred ravens. The pink one needs tens of thousands of things to move visibly away from 50%.
Both statements are one and the same rule. The difference is not in the logic but in where we look for the culprit.
“In plain words” is hidden on all puzzles.
Only one thing breaks the rule: a white raven. If it exists, it is both a raven and a non-black thing. So you can look for it in either place. But among 100 ravens it stands out, while among 10,000 non-black things it gets lost, like keys in a big warehouse.
Looking at a raven is like looking for your keys in the drawer where they should be: if it is empty, you have learned something. Looking at an apple is like looking for them in a random corner of the warehouse: nothing there, but you did not expect to find them there anyway. Every check that passes confirms the rule a little. The more easily it could have failed, the more it weighs. This card does not say why the difference is exactly 100-fold. The next one does.
A raven check could fail 100 times more easily than an apple check. That is why a hundred apples together weigh about as much as one black raven. And 100 and 10,000 are simply the numbers of ravens and of non-black things in the park. In the real world there are billions of times more non-black things than ravens, so a single apple weighs so little that intuition rounds it to zero.
“Rigorous” is hidden on all puzzles.
odds after = odds before × P(E | H) / P(E | ¬H)
H is "all ravens are black", E is what we saw. The likelihood ratio says how many times more likely the observation is if the rule is true. If all ravens are black, a random raven is surely black; if one is white, it is black with a chance of 99/100. The ratio is 100/99 ≈ 1.0101. For the apple: surely against 9,999/10,000, a ratio of ≈ 1.0001. Odds of 1 : 1 become 100 : 99 after a raven and 10,000 : 9,999 after an apple.
After k apples the odds are (10,000/9,999)k. A hundred apples give 1.0001100 ≈ 1.01005, almost as much as one raven with its 1.0101. A thousand apples raise the chance of the rule only to 52.5%, while a thousand black ravens raise it to 99.996%. Apples do confirm the rule, but to match the ravens you need a hundred times more of them.
H: all R ravens are black; ¬H: one of them is white. There are N non-black things, and the white raven, if any, is among them. Bayes' theorem in odds form:
A random raven is black, and a random non-black thing is not a raven:
How many apples weigh as much as one raven: the odds after k apples are Λapplek, so compare logarithms.
In our park R = 100, N = 10,000:
The weight of one apple relative to a raven is ≈ R/N: the number of ravens divided by the number of non-black things. If there were as many non-black things as ravens, an apple would weigh as much as a raven. The more non-black world there is, the less each piece of it weighs.
“Deeper” is hidden on all puzzles.
The same red apple may or may not be evidence, depending on how you came to it. If you picked a random non-black thing and checked whether it is not a raven, the check could have failed, and the weight of 1.0001 is real. But if you picked a random apple and looked at whether it is red, an apple's colour has nothing to do with ravens: the chance of seeing red is the same in both parks, the ratio is exactly 1, and there is no evidence at all. As early as 1960 the statistician I. J. Good stressed that the weight of an observation depends on how the check was made.
Good showed that even a black raven does not always confirm the rule. Imagine two possible worlds, each with a million other birds. In the first there are 100 ravens, all black. In the second there are 1,000 black ravens and one white one. You pick a random bird, and it is a black raven. In the second world this happens about ten times more often: it has ten times more black ravens. So if the worlds were equally likely beforehand, the chance that all ravens are black falls from 50% to ≈ 9.1%. The principle "an example confirms the rule" only works when other knowledge does not change the picture.
Hempel himself did not think his logic was wrong. In his view the paradox is a psychological illusion: the apple seems irrelevant because we know, without saying it, that ravens are few and non-black things are countless. The Bayesian answer, first given by the Polish logician Janina Hosiasson-Lindenbaum in 1940 and later developed by Good and others, put a number on this idea: the confirmation is there, but its weight is roughly the ratio of ravens to non-black things. In 2010 the philosophers Branden Fitelson and James Hawthorne worked out exact conditions under which a black raven weighs more than a non-black non-raven.
Mathematics: when checking without repeats, ravens settle the question in 100 steps, and things only in 10,000. Logic: a check confirms exactly as much as it could have failed. Practice: tests that cannot fail confirm nothing.
Mathematics
Now we don't put anything back: every next raven or thing is a new one. The same park: 100 ravens, 10,000 non-black things, 50-50 before any check.
If a white raven exists, after n different black ravens it hides among the remaining 100 − n. The chance that all n turned out black is (100 − n)/100, so the odds of the rule are 100/(100 − n), and the chance is 100/(200 − n). Each next raven weighs more than the one before, and the hundredth settles the question completely. With things the formula is the same, only 10,000 replaces 100: to settle the question with apples you would have to look at every non-black thing in the park.
In general: an observation from a class of size M weighs roughly 1/M. A black raven and a red apple fit the same rule, but they come from classes of different size: R ravens and N non-black things. Hence an apple weighs ≈ R/N of a raven.
Logic
q is the chance that one check would fail if the rule were false. The check passed. How many such checks does it take to raise the chance of the rule from 50% to 95%?
People say: "absence of evidence is not evidence of absence". The raven paradox shows when that is wrong. If you looked where the culprit should have turned up and found nothing, that is evidence it is not there, and the stronger the more likely it was to turn up there. But if you looked where it would almost certainly not have been anyway, the silence says almost nothing. This is just Bayes' theorem: the weight of a check is the ratio 1/(1 − q).
The same logic from the other side is in the prosecutor's fallacy: there a rare match looks like proof, here an ordinary observation looks like nothing. And Bayes' theorem in the glossary shows how to work out such updates in general.
Practice
In programming, a test that passes whatever the code does is a red apple: it "fits the rule" but checks nothing. This is exactly why mutation testing was invented (DeMillo, Lipton and Sayward, 1978): small bugs are put into the code on purpose to see whether any test fails. If none does, the tests could not have failed, and their green colour means little.
In medicine, a cold usually clears up by itself within a week or two. "I took this remedy and recovered within a week" is a check that could hardly fail: people recover without the remedy too. That is why drugs are compared with a placebo in controlled trials: only there could the result have come out differently if the remedy did not work.
Carl Hempel described it in "Studies in the Logic of Confirmation", published in two parts in the journal Mind in 1945. Early versions of the idea appeared in his work from 1937.
The philosopher Nelson Goodman, in his book "Fact, Fiction, and Forecast" (1955), wrote wryly about "the prospects for indoor ornithology": if an apple confirms a rule about ravens, you can study birds without leaving the house.
Even before Hempel's paper in Mind, the Polish logician Janina Hosiasson-Lindenbaum analysed the paradox with probabilities in "On Confirmation" (Journal of Symbolic Logic, 1940): a non-raven confirms the rule more weakly because non-ravens are far more numerous.
That is what Good called his short paper in the British Journal for the Philosophy of Science with the example where a black raven weakens the rule. A pun: a white shoe is the classic example of a non-black non-raven, and a "red herring" is a false lead.
Branden Fitelson and James Hawthorne, in the chapter "How Bayesian Confirmation Theory Handles the Paradox of the Ravens" (2010), set out exactly which assumptions make a black raven confirm the rule more strongly than a non-black non-raven.
In our park, 293 black ravens are enough to raise the chance of the rule from 50% to 95%. With apples (putting them back) you would need to look at 29,443 non-black things, almost three times more than the whole park holds.
Share your own explanation, an objection or a variant of the puzzle. You need a GitHub account to comment.
Is “School maths” the right level for this puzzle?
Thank you! This helps set the levels right.
I keep adding new puzzles, new ways to explain them and write-ups of the discussions they start. Leave your email and I'll write when something new is out. No spam, one-click unsubscribe.
The sign-up form will appear here soon.