Math paradoxes

The 37% rule: the secretary problem 100 candidates. One choice. A 37% chance of the best.

Candidates arrive one at a time, and you must decide on the spot: hire now or turn them away for good. It seems you could pick the best of 100 with about a 1% chance. A simple optimal stopping rule gives 37%: let the first 37 go, then take the first one who beats everyone before. Check it with a game, a simulation and five explanations.

1%at random
37%with the 37% rule
The puzzle

Picture this: you are hiring a secretary

This is the classic optimal stopping problem. It is known as the secretary problem, the marriage problem or the game of googol. The core is the same: options appear one by one, you can't go back to one you turned down, and you must decide when to stop.

  1. 1

    The candidates

    There are n candidates for one job. You know how many there are, but they come to the interview in random order.

  2. 2

    The interview

    After each interview you know only one thing: how this candidate compares with the ones you have already seen. You know nothing about those still to come.

  3. 3

    Decide on the spot

    You can hire the candidate, and the search is over. Or you can reject them, and they are gone for good. If you reach the last one, you have to take them.

  4. 4

    The goal

    There is only one way to win: hire the best of all n. The second best counts as a loss. Which strategy gives the highest chance, and how high is it with 100 candidates?

Intuition says

"The best is one in a hundred, so the chance is about 1%. It's a lottery"

In fact the 37% rule finds the best 37% of the time, and that chance barely depends on how many candidates there are. Even with a million candidates it stays at 36.8%.

Model assumptions

  • The number of candidates n is known in advance.
  • They arrive in random order.
  • Any two candidates can be compared, with no ties.
  • A rejected candidate can't be called back.
  • Only the very best counts.

Round 5 shows what happens when these rules change.

Round 1 · Guess

What chance of hiring the best of 100?

Imagine the smartest strategy anyone could come up with. How often would it find the best of 100 candidates? Lock in your answer before reading on.

5%
0%25%50%75%100%

Your answer will be compared with the right one here.

Round 2 · Hire

20 candidates, one decision

Candidates arrive one at a time. The height of a bar shows how a candidate compares with those seen so far, and orange means "best so far". You see the true scores only at the end. Play a few times: the panel on the right shows how the 37% rule would have done on the same lineups.

Candidate1 / 20
Among those seen1

Your score

You0 / 0
The 37% rule0 / 0

With 20 candidates the best rule is to skip 7, then take the first one who beats everyone before. It finds the best in 38.4% of lineups. Picking at random would give 5%.

Round 3 · Simulation

How many candidates to skip

A strategy with cutoff r: skip the first r candidates, then hire the first one who beats everyone before. The computer plays thousands of lineups for each r. The line is the exact formula; the dots are the simulation. For 100 candidates the peak is at r = 37.

exact formula simulation
–
–
Round 4 · Five explanations

Why 37% exactly

The first part of the candidates is not for choosing. It is for learning what counts as good. The only question is how long to spend on it.

Look, then leap

The first 37 set the bar

With no information you can't tell whether a candidate is good. So you only assess the first 37 and remember the best of them. That person becomes the bar. After that you take the first candidate who clears it.

The skipped candidates are not wasted: they are what tells you the level. The price of that knowledge is that the best might be among them.

Too early and too late

Both mistakes are costly

Skip only 5 of 100the bar is low, you settle too soon
Skip 80 of 100in 80% of lineups the best is already gone

Stop looking too early and the bar is weak, so an average candidate clears it. Look for too long and the best was probably among those you skipped, so nobody later beats the bar. A cutoff of 37 balances the two risks.

A tiny example

Three candidates, six orders

The number is the candidate's true place: 1 is the best. Faded means skipped; dark means hired.

Where the other 63% go

Even the best strategy loses more often

With a cutoff of 37 there are three outcomes. In 37.1% of lineups you hire the best. In another 37% the best was among those skipped: nobody clears that bar, and you reach the end. In the remaining 25.9% someone other than the best clears the bar first.

For formula lovers

The exact formula and where 1/e comes from

Say the best stands at place i, with i greater than r. That happens with probability 1/n. You hire them if nobody cleared the bar before, that is, if the best of the first i−1 candidates was among the first r. The chance of that is r/(i−1). Add it up over all i:

P(r) = (r/n) · (1/r + 1/(r+1) + … + 1/(n−1))
x = r/n:   P ≈ x · ln(1/x) = −x · ln x
(−x · ln x)′ = −ln x − 1 = 0  →  x = 1/e ≈ 0.368,   P = 1/e ≈ 36.8%
n = 100:   r = 37,   P(37) = 37.1%

The sum of reciprocals from r to n−1 is about ln(n/r). So the chance depends only on the share x = r/n, not on n itself. The function −x · ln x peaks at x = 1/e, and its value there is also 1/e. That gives both numbers: skip 37% and win 37% of the time.

Round 5 · Changing the rules

Other n, other goals, a second chance

The 37% figure doesn't care how many candidates there are, but it does care about the goal. If anyone from the top three will do, or you can go back to someone you rejected, the share to skip is quite different.

The best cutoff for different n

Skip the first
Chance of the best
n/e ≈

Share and chance both tend to 1/e

best chance share skipped r/n

The n axis is logarithmic, from 3 to 10,000. The dashed line is 1/e ≈ 36.8%. For small n the chance is even higher: 50% with three candidates.

You may go back to someone you rejected

Suppose that at the end, if you hired nobody, you may call back the best person you saw. They say yes with probability q. 100 candidates.

with a call-back classic problem
Skip the first
Chance of the best

Other goals and an unknown number of candidates

Same rule everywhere: skip the first r, then take the first one who beats everyone before, and if nobody does, take the last. The best cutoffs were found by computer. Press "Check" to rerun 10,000 simulated lineups.

Goal (100 candidates)Best cutoffResult

The last row is about a search with a time limit, when you don't know how many candidates will come. Then you count time instead of people: you only look during the first 37% of the period. The chance stays at least 1/e for any number of candidates.

Bonus · Fun facts

Things to tell your friends

11

Kepler looks for a wife

After his first wife died in 1611, the astronomer Johannes Kepler considered 11 possible brides. In the end he married the fifth, Susanna Reuttinger. The story is often told as an example of the secretary problem, including in the book Algorithms to Live By.

1960

The game of googol

Martin Gardner brought the problem to a wide audience in his Mathematical Games column in Scientific American in 1960. In his version, people write any numbers on slips of paper, as large as a googol, and you have to stop at the largest.

1% → 37%

37 times better than guessing

Pick one of 100 candidates at random and the chance of the best is 1%. The 37% rule gives 37.1%. With 1,000 candidates a random pick gives 0.1%, and the rule still gives 36.8%.

2016

Flats and dating

Algorithms to Live By (2016) by Brian Christian and Tom Griffiths opens with the 37% rule and applies it to flat hunting and dating. If you search from age 18 to 40, the first 37% of that span ends at about 26.

e ≈ 2,718

The same number elsewhere

The number 1/e ≈ 36.8% also turns up in the Secret Santa problem: when many people draw names at random, the chance that nobody draws their own name also tends to 1/e.

3,87

An average rank below 4

If the goal is not "the best or nothing" but the best average rank, there is a cleverer strategy with several thresholds. Even for a huge number of candidates it gives an average rank of about 3.87. The result was published in 1964.

Discussion

What do you think?

Share your own explanation, an objection or a variant of the puzzle. Commenting needs a GitHub account.

Newsletter

New puzzles, explanations and discussions

I keep adding new puzzles, new ways to explain them and write-ups of the discussions they start. Leave your email and I'll write when something new is out. No spam, one-click unsubscribe.

The sign-up form will appear here soon.