Math paradoxes
Course «A/B testing paradoxes» · chapter 19 of 21 →

The Will Rogers phenomenon One person switches teams. Can both teams get stronger?Both averages rise, yet nobody improved.

Two teams, each with its own average. The coach moves one member from the first team to the second. Surely one team has to get weaker. Lock in your answer, then check it with a game, a hospital simulation and a few explanations.

Move someone who is below their own team's average but above the other team's, and both averages go up. The overall average does not change: nobody got better, only the boundary between the groups moved. The same thing "improves" cancer survival when sharper scans move patients from an early stage to a late one.

Make a guess
7 → 7.5strong team's average
2.5 → 3weak team's average
5 = 5overall average
The puzzle

Picture this: two teams at a maths contest

The name comes from a joke credited to the American humorist Will Rogers: when the Okies left Oklahoma and moved to California, they raised the average intelligence in both states. Whether he ever said it is unknown; more on that in the facts. First, the puzzle.

  1. 1

    Two teams

    At a school contest nine players scored from 1 to 9 points, one score each. The strong team A: 5, 6, 7, 8, 9. The weak team B: 1, 2, 3, 4.

  2. 2

    The averages

    Team A's average score is 35 / 5 = 7, team B's is 10 / 4 = 2.5. All nine together average 45 / 9 = 5.

  3. 3

    The transfer

    The coach may move exactly one player from team A to team B. Their score stays the same.

  4. 4

    The question

    Can one player be moved so that the average score rises in both teams? If so, whom?

The line-ups

Team A · average 7
56789
Team B · average 2.5
1234
  • Players' scores do not change.
  • A team's average is the sum of scores divided by the number of players.
  • "Rises" means "becomes strictly larger".
Intuition says

"Whatever one team gains, the other loses. Both cannot rise"

That would be true of the total score. But an average also depends on the head count. The player with 5 points is the weakest in team A, so A is stronger without them. And they are stronger than everyone in team B, so B is stronger with them too.

Round 1 · Guess

Whom do you move?

Team A: 5, 6, 7, 8, 9 (average 7). Team B: 1, 2, 3, 4 (average 2.5). After one player moves from A to B, the average score must rise in both teams.

Your answer

Pick an answer. This is where you will see what happens to both averages after each of the five possible moves.

Hints

Hard to decide? Open the hints one at a time: the easy one first, then stronger ones.

Round 2 · The game

Move the players yourself

Tap a player to move them to the other team. The goal: as many moves in a row as you can, each one raising the average of both teams. Keep an eye on the overall average.

team A and its average team B and its average overall average

Your moves

    Hint: look at where the player sits on the line relative to the two average ticks.

    Round 3 · Simulation

    A new scanner and 2,000 patients

    Here is where the trick happens for real. A hospital splits cancer into an early stage (no metastases found) and a late stage (metastases found). Each patient has a hidden measure of how far the disease has spread, from 0 to 100. The higher it is, the lower the chance of living 5 years: from 100% at 0 down to 0% at 100. The scanner only sees metastases once the spread reaches a threshold. The old scanner sees them from 60, the new one already from 40. The treatment is the same, and so are the patients.

    early stage late stage moved between the old and the new labels

    Each dot is a patient (400 of 2,000 shown). A filled dot lived 5 years, a hollow one did not. The vertical line marks the spread from which the scanner sees metastases.

    early stage: exact and simulated late stage all patients
    Lived 5 years: early stage
    –
    Lived 5 years: late stage
    –
    Lived 5 years: all patients
    –

    Round 4 · Explanations

    Why both averages rise while nothing changed

    A group's describes not only its people but also where the line between the groups is drawn. Move the line and the averages change, even if no single person changed.

    In plain wordspictures, analogies, small numbers

    “In plain words” is hidden on all puzzles.

    The core idea

    The weakest of the strong team, the strongest of the weak one

    The 5-point player against A's average5 < 7
    The same player against B's average5 > 2.5

    The 5-point player was dragging team A down: they are below its average. When they leave, A gets stronger. In team B they are the best: above its average. When they arrive, B gets stronger too. That is enough to see the trick. Why nobody actually improved is the next card.

    The check

    The overall average stays put

    All 9 before the move45 / 9 = 5
    All 9 after the move45 / 9 = 5

    The same nine people, the same scores, the same sum. Only who sits in which team changed. Both teams "got stronger" because the strong team got smaller and even stronger, and the weak one got bigger. The weak team's weight in the overall average grew by exactly enough to cancel both gains.

    In the hospital

    The borderline patients

    Early stage, old scanner70%
    Patients with spread 40–6050%

    The new scanner moves patients with spread from 40 to 60 into the late stage. They live 5 years 50% of the time: worse than the rest of the early stage (70%), but better than the late stage (20%). They are the "5-point players". Both stages look better, though no patient lived a single day longer.

    Where intuition slips

    An average is not a property of the people

    We are used to reading "the group's average went up" as "the people in the group got better". But an average depends on who is in the group. When the membership changes, the average changes without anyone improving. So whenever a group's average goes up, ask: did the rule for who counts as a member change?

    Rigoroustables, charts, formulas

    “Rigorous” is hidden on all puzzles.

    For formula lovers

    The transfer condition and the weighted average

    Group A has nA people and average a, group B has nB people and average b. A person with score x moves from A to B. This is how the averages change.

    a′−a=a−xnA−1 b′−b=x−bnB+1

    Both differences are positive exactly when the score lies between the two averages.

    b<x<a 2.5<5<7:7−54=+0.5,5−2.55=+0.5

    The overall average is the weighted average of the two groups. The sum of scores and the head count do not change, so neither does it.

    m=nAa+nBbnA+nB 5·7+4·2.59=4·7.5+5·39=5

    Both averages rose by 0.5 and the overall one did not move, because group B's weight grew from 4/9 to 5/9. Group B is still weak; it is just bigger now. If x equals one of the averages, that average does not change, so the condition is strict.

    Chart

    How the averages change depending on whom you move

    change in A's average change in B's average both rise

    Across is the score of the player moved from A to B. Line A crosses zero at 7 (A's average), line B at 2.5 (B's average). Between them both lines are above zero. In team A the players with 5 and 6 points fall there.

    The hospital in numbers

    5-year survival in the round 3 model

    Old scanner (60)New scanner (40)
    Early stage70%80%
    Late stage20%30%
    All patients50%50%
    Share in the late stage40%60%

    Spread is uniform from 0 to 100, the chance of living 5 years is 1 − spread/100. If the scanner sees metastases from a threshold t, the early stage survives on average 100 − t/2 percent, the late stage 50 − t/2, and everyone together always 50. Every step the threshold drops lifts both stages by half a step. Patients with spread 40–60 (20% of all) survive 50% of the time.

    Deepergeneralisations, theory, history

    “Deeper” is hidden on all puzzles.

    A bit of history

    Feinstein and lung cancer

    In 1985 Alvan Feinstein and two co-authors published "The Will Rogers phenomenon" in the New England Journal of Medicine. They compared lung cancer patients first treated in 1977 with patients from 1953–1964 at the same institutions. The newer group had higher six-month survival in every TNM stage.

    The newer group had gone through many new imaging tests, which found metastases that used to be silent. Some patients who would once have been put in a "good" stage were now put in a "bad" one. When the authors split both groups by symptom stages, which the new tests could not change, survival came out similar.

    How to guard against it

    Compare by the same rules

    Comparing survival by stage across decades, countries or hospitals is risky when the diagnostics differ. There are three honest ways out. Compare all patients together, like the "all" row of the table. Re-stage the old patients by the new rules, so the line is drawn in the same place. Or split by a feature the new technology does not change, as Feinstein did with symptom stages.

    This trap differs from another one doctors know well: early detection lengthens survival "from diagnosis" because the clock simply starts earlier. Here nobody even has to live longer or be found earlier: moving the line between the stages is enough.

    Round 5 · Scaling up

    Chains of moves, group boundaries and product metrics

    First the maths (how many times in a row both averages can rise), then the logic (what really changes when a boundary moves), then practice (how the trick gets into analytics reports).

    1 · Maths: a chain
    A's average B's average overall average

    How many times in a row?

    The same holds for a whole group: move several people at once and both averages rise when the movers' average lies between the two group averages. That is how the "Okies" of the joke could raise the average of both states, even if some of them were bright and some were not.

    That no chain is longer than n − 2 was checked by brute force over every split for n from 3 to 11.

    2 · Logic: the boundary moves, not the people

    Change a definition and the averages change on their own

    Every split into groups has a boundary: a cancer stage, a "top student", "tall", an "active user", a "premium customer". Move the boundary so the borderline cases cross it, and the average can rise on both sides. There is no real improvement: the same people, the same results.

    The rule against it is simple: when the averages of all the parts go up, check the whole. If the overall average stays put, only the labelling changed. A related group trap is Simpson's paradox, where the parts and the whole point in opposite directions because the groups carry different weights.

    Where else it shows up

    Better diagnostics. More sensitive tests find milder forms of a disease. The mild cases join the sick and improve their numbers, while the healthy look healthier without them.

    Ratings and tiers. Redrawing the line between "A-class" and "B-class" hotels, schools or suppliers changes the average scores of both tiers without any change in the hotels themselves.

    3 · Practice: "active users"

    A product has 1,000 users. The table shows how many sessions a week they have and how much they pay a month on average. Choose from how many sessions a user counts as active.

    SessionsUsers$ per userSegment

    The monthly report

    A/B tests are no different: if the new variant changes who counts as "active", "engaged" or as having seen the change, comparing metrics inside those segments between variants can show a gain in both segments with no gain overall. Make the main decision on the metric over all users of each variant, as randomisation assigned them. How to read the test itself is on the page "Two identical pages".

    Bonus · Fun facts

    Things to tell your friends

    1970

    Rogers probably never said it

    Will Rogers died in 1935. According to Quote Investigator, the joke about the Okies was first credited to him in 1970, when the Los Angeles Times quoted the Oklahoma governor Dewey Bartlett. The line has not been found in Rogers's own writings.

    1939

    The joke is older than the name

    The earliest version found ran in 1939 in the McComb Daily Journal of Mississippi: one man moved from one county to another and "raised the intelligence level of both". The version about Okies and California turns up in 1955.

    NZ → AU

    The New Zealand version

    The New Zealand prime minister Robert Muldoon is credited with a version: New Zealanders who emigrate to Australia raise the IQ of both countries. In 1990 a Sydney Morning Herald journalist wrote that Muldoon told it at a dinner, and added that the joke was very old.

    1985

    A doctor gave it the name

    The term "Will Rogers phenomenon" was coined by the Yale clinical epidemiologist Alvan Feinstein in a 1985 paper on lung cancer. He took the joke from a friend who credited it to Rogers. Since then researchers look for "stage migration" in studies of cancer and other diseases.

    28%

    Same slides, new grades

    In 2005 Peter Albertsen and colleagues had an experienced pathologist re-grade the slides of 1,858 men diagnosed with prostate cancer in Connecticut in 1990–1992. The mean Gleason score rose from 5.95 to 6.8. With the new grades, mortality within each score looked 28% lower, though the patients' outcomes were the same.

    1939

    Who the Okies were

    "Okies" was the name for migrants from Oklahoma and nearby states whom the Dust Bowl and the Great Depression of the 1930s drove west to California. John Steinbeck told their journey in "The Grapes of Wrath" (1939), which won him the Pulitzer Prize.

    Discussion

    What do you think?

    Share your own explanation, an objection or a variant of the puzzle. You need a GitHub account to comment.

    Is “School maths” the right level for this puzzle?

    Thank you! This helps set the levels right.

    Newsletter

    New puzzles, explanations and discussions

    I keep adding new puzzles, new ways to explain them and write-ups of the discussions they start. Leave your email and I'll write when something new is out. No spam, one-click unsubscribe.

    The sign-up form will appear here soon.