Ten bidders look at the jar and honestly estimate how much money is inside. Each bids their estimate, and the highest bid takes the jar. Nobody is greedy and nobody cheats. On average, how much does the winner make or lose? Lock in your answer, then check it with a game, a simulation and the explanations.
The estimates are honest, but the jar goes not to the most accurate one but to the highest. The winner is whoever overestimated the most, so for a jar worth 100 the winner pays about 131 on average. Winning is itself bad news about your estimate. Check it with a game, a simulation and the explanations.
Max Bazerman and William Samuelson ran this in MBA classes in 1983. We simplify their experiment to a clean model.
A glass jar holds coins worth exactly 100 dollars. The bidders don't know that: they can only look at the jar.
Each of the 10 bidders estimates the contents alone, without seeing the others' estimates. The estimates are honest: on average they hit 100, but each one is off up or down, typically by about 20 dollars.
Each writes a bid equal to their estimate in a sealed envelope. The highest bid gets the jar and pays what it bid.
On average, how much does the winner make or lose if the auction is repeated over and over?
"The estimates are honest, so on average the winner pays a fair price"
Each estimate taken alone is fair on average. But the winner is not a random bidder: it is the one whose estimate is highest. And the highest of ten estimates is almost always too high, by 31 dollars on average.
A standard deviation of 20 means about two thirds of the estimates fall between 80 and 120.
Ten bidders honestly estimate a jar worth 100 dollars and bid their estimates. On average, what does the winner end up with?
Pick an answer. A simulation of 10,000 such auctions will appear here.
Hard to decide? Open the hints one at a time: the easy one first, then stronger ones.
Think about who actually gets the jar: the bidder with the most accurate estimate, or the one with the highest?
Each estimate on its own is honest: on average it hits 100. But the winner is not picked at random. What is the average of the largest of ten estimates, if each is off up or down?
Imagine ten rolls of a die instead of ten estimates. One roll averages 3.5. What does the largest of ten rolls average? Now apply that to the estimate errors.
Now the jar's value is random, between 60 and 140, and nobody knows it, not even you. You see only your own estimate, with the same error as the bots'. Choose how far below your estimate to bid, and place the bid. The bots are either honest (they bid their estimate) or careful (they bid their estimate minus 31).
For each number of bidders the simulation runs 2,000 auctions where everyone bids their honest estimate, and records how much the winner overpaid. The line is the exact value, the dots the simulation. The slider sets how imprecise the estimates are.
The first layer has no formulas, the second has a table, formulas and a chart, the third covers history and auction theory.
“In plain words” is hidden on all puzzles.
All ten look at the same jar. Some underestimate a little, some overestimate a little. The auction does not ask whose estimate is most accurate; it asks whose is highest. So the winner is the one who most "saw" more money than there is.
If you won, it means nine others, looking at the same jar, decided there was less money in it. The win itself is news: your estimate is most likely too high. This card explains why the winner overpays, but not by how much. The next layer gives the number.
Numbers from 10,000 simulated auctions. The crowd as a whole estimates correctly, yet the winner almost always overpays. In Bazerman and Samuelson's experiment it was even more striking: the students' average estimate was below the jar's true value ($5.13 against $8), while the average winning bid was above it ($10.01).
Of two estimates, the higher is too high by 11 on average; of ten, by 31; of a hundred, by 50. The more people estimate the same thing, the more extreme the highest estimate. So a crowded auction calls for more caution, not less, though instinct says the opposite.
Here nobody overestimates on purpose and nobody gets carried away. Each estimate on its own is honest. The overshoot is created by the rule "the highest wins". It picks the estimate with the biggest upward error, however sensible the people are.
“Rigorous” is hidden on all puzzles.
| Bidders n | E[max] in σ | Overpay, σ = 20 | P(overpay) |
|---|
The average of the largest of n independent normal errors, in standard deviations σ. The winner's overpayment is σ times this number. The chance that the winner overpays is 1 − ½ⁿ: there is no overpayment only if all n estimates are below 100.
Bidder i's estimate: the true value V plus an error σZi, with Zi independent standard normals. The winner pays the highest estimate.
The average overpayment: σ times the mean of the maximum of n standard normals.
There is no overpayment only when every estimate is below V.
If everyone bids their estimate minus the same discount s, the winner gets s − σ·E[max] on average. The discount at which the winner breaks even grows with both the number of bidders and the error.
“Deeper” is hidden on all puzzles.
The name comes from a paper by three engineers at the oil company Atlantic Richfield, Capen, Clapp and Campbell, "Competitive Bidding in High-Risk Situations" (Journal of Petroleum Technology, 1971). Companies bought drilling rights on the US continental shelf at auction and year after year earned unexpectedly low returns.
The explanation was the one on this page: the oil under a tract is the same for everyone, the geologists' estimates are imprecise, and the tract goes to whoever valued it most optimistically.
A jar of coins, oil underground, the rights to a film: their value is the same for everyone, but nobody knows it exactly. That is a common-value auction, and in it winning carries bad news.
If you buy a painting that you personally love, your value is private: the other bidders know nothing about what it is worth to you. Then winning only means you value it most, and there is no curse. Most real auctions sit somewhere in between.
Auction theory says to ask not "what do I think the jar is worth?" but "what is it worth if my estimate turns out to be the highest of ten?". The second number is lower, and the gap grows with the number of rivals and with the imprecision of the estimates. A bid worked out this way does not lose on average, even when it wins.
People learn this with difficulty. In John Kagel and Dan Levin's laboratory auctions, bidders who had learned to avoid the curse in auctions of four fell into it again in auctions of seven: the experience did not carry over well to new conditions.
First the maths (how the curse grows with the number of bidders), then the logic (any pick of the best by a noisy estimate), then practice (the winner among ad creatives).
For large n the highest of n normal estimates is too high by about σ·√(2 ln n). The formula overshoots for small n: at n = 10 it gives 2.15 instead of the exact 1.54, and it closes in only very slowly: with a million bidders it is 5.26 against 4.86.
The key is the square root of a logarithm. Going from 10 to 100 bidders adds 0.97σ, from 100 to 1,000 another 0.73σ, from 1,000 to a million 1.62σ. The first rivals add the most curse. But it never stops, and for normal errors it has no ceiling.
10 projects, each with a random true value (mean 100, spread 20). The estimates are honest, with error 20. We pick the project with the highest estimate.
An auction is just one case of a general rule: if you pick the best of several options by a noisy estimate, the estimate of the pick is too high. In 2006 the decision analysts James Smith and Robert Winkler called this the optimizer's curse: even with honest estimates, disappointment awaits on average after the choice.
The cure is to shrink the estimate towards the average. Here the real differences between projects and the errors are equally large, so only half of the deviation from 100 deserves trust: read an estimate of 140 as 120. It is the same regression to the mean, applied to your own forecasts.
10 ad banners with the same true CTR of 2%. Each is shown the same number of times, and the banner with the highest CTR is declared the "winner".
Picking the best of 10 banners is an auction in which each banner's "bid" is its measured CTR. Even when all banners are the same, with 5,000 impressions each the winner looks 15% better than the true 2% on average. Once rolled out to everyone this "lift" disappears. The more variants, the stronger the curse, and it is the same mechanism as in multiple testing.
Even a real winner of an A/B test that crossed the significance threshold usually shows a bigger effect than it has: see the "winner's curse" card on "Two identical pages". Hence the habit of discounting a measured lift and confirming the winner with a repeat test. The same is suspected in baseball's free-agent market, where the team with the rosiest view of a player signs him at the highest price, and in IPOs, where those who know less are more likely to get their full order filled in exactly the weaker issues.
Capen, Clapp and Campbell, engineers at Atlantic Richfield, explained why the companies that won auctions for oil tracts on the US continental shelf earned less than expected year after year. Their 1971 paper introduced the phrase "winner's curse".
Bazerman and Samuelson ran 48 auctions in 12 MBA classes at Boston University. Jars of coins and paper clips were worth $8 each. The average estimate was $5.13, yet the average winning bid was $10.01: winners lost $2.01 on average.
Richard Thaler devoted one of his "Anomalies" columns in the Journal of Economic Perspectives to the winner's curse, and in 1992 he gave the name to his book on the paradoxes of economic life. In 2017 Thaler won the Nobel prize in economics.
In Kagel and Levin's laboratory auctions, experienced bidders learned not to overpay in auctions of four. But in auctions of seven, where the curse is stronger, they overpaid again.
In 2000 the British government sold five third-generation mobile licences for £22.5 billion, against an official forecast of £1–3 billion. Economists still argue over whether the companies overpaid, and the auction became one of the best-known cases in debates about the winner's curse.
James Smith and Robert Winkler showed in Management Science that the same mechanism disappoints anyone who picks the best option by estimates: a project, an investment, a candidate. Their cure is Bayesian shrinkage of the estimates, which they called "disciplined skepticism".
Share your own explanation, an objection or a variant of the puzzle. You need a GitHub account to comment.
Is “Advanced” the right level for this puzzle?
Thank you! This helps set the levels right.
I keep adding new puzzles, new ways to explain them and write-ups of the discussions they start. Leave your email and I'll write when something new is out. No spam, one-click unsubscribe.
The sign-up form will appear here soon.