Frederick's paper
Shane Frederick described the test in "Cognitive Reflection and Decision Making", published in the Journal of Economic Perspectives in 2005. He compared test scores with how people make choices about money, risk and waiting.
A bat and a ball cost $1.10 together. The bat costs a dollar more than the ball. The answer "10 cents" pops into your head at once, and it is wrong. The ball costs 5 cents. This is the first of three questions in the cognitive reflection test. Take the test, check the answers by hand and see why the fast thought is so convincing.
In both receipts the bat costs a dollar more than the ball. Only one adds up to $1.10.
The cognitive reflection test was published by the psychologist Shane Frederick in 2005. It has three problems. Each has a fast answer that comes to mind on its own and a right answer you have to work out. The maths is primary-school level. The test checks something else: whether you stop to check your first thought.
A bat and a ball cost $1.10 in total. The bat costs $1.00 more than the ball. How much is the ball?
Five machines make five widgets in 5 minutes. How many minutes do 100 machines need to make 100 widgets?
A patch of lily pads on a lake doubles in size every day. It takes 48 days to cover the whole lake. How many days does it take to cover half of it?
Answer the way you normally would. There are no tricks in the wording: each question means exactly what it says.
"10 cents, 100 minutes, 24 days"
The right answers are 5 cents, 5 minutes and 47 days. The fast answers are not random. Each one fits the numbers in the question but not the question itself.
Take the test in round 1 first, then read the explanations.
The timer starts when you begin typing. Don't rush and don't slow down on purpose: answer what seems right to you. After the check you'll see your answers next to the fast ones and the right ones.
A bat and a ball cost $1.10 in total. The bat costs $1.00 more than the ball. How much is the ball?
Five machines make five widgets in 5 minutes. How many minutes do 100 machines need to make 100 widgets?
A patch of lily pads doubles in size every day. It takes 48 days to cover the whole lake. How many days does it take to cover half of it?
Pass the phone around: every attempt goes into this session's statistics in round 3. Nothing is sent anywhere.
Every fast answer can be checked by plugging it back in. Move the sliders and see where the conditions really hold.
It is the same idea as the duckweed lake puzzle. There you can grow the duckweed on a lake day by day.
On the left is a model: on each question, each person stops to check with probability c, and otherwise gives the fast answer. The line shows the exact score distribution; the dots are the simulation. On the right are the real answers of the people who took the test on this page in this session.
The mistake is not in the arithmetic. The brain answers an easier question in place of the one asked and does not check the result. Here are five ways to see how that happens.
In Thinking, Fast and Slow, Daniel Kahneman describes thinking as the work of two systems. System 1 produces an answer instantly. System 2 could check it, but it is "lazy" and often just agrees. The cognitive reflection test catches exactly that moment.
The total $1.10 splits on its own into one dollar and 10 cents. The phrase "a dollar more" nudges you to take that split. But the difference between $1.00 and $0.10 is 90 cents, not a dollar. The fast answer fits the numbers but not the condition.
The pattern "5, 5, 5" suggests "100, 100, 100". But the question gives the rate of one machine: 5 machines make 5 widgets in 5 minutes, so each machine makes one widget in 5 minutes. The 100 machines work in parallel, and each makes its own widget in the same 5 minutes.
The time depends only on how many widgets each machine has to make. Here it is one widget, so 5 minutes.
Intuition halves 48 because it is used to linear growth. But the patch doubles every day. So if the whole lake is covered on day 48, half was covered the day before. On day 24 the patch covered just 1/16,777,216 of the lake, about one sixteen-millionth.
Each equation takes one line and is easy to solve. The hard part is writing it down at all instead of going with the first thought. The general rule for the ball: if the total is T and the difference is D, the ball costs (T − D) / 2, and the fast answer T − D is always exactly twice that.
The same trap can be built from any numbers. First change the prices in the ball problem, then try five new puzzles in the same spirit. We wrote them for this page; they are not in the original test.
Type a number and press "Check". You'll see the fast answer, the right one and a short explanation.
Shane Frederick described the test in "Cognitive Reflection and Decision Making", published in the Journal of Economic Perspectives in 2005. He compared test scores with how people make choices about money, risk and waiting.
The whole test is three questions and takes a few minutes. It needs almost no knowledge. It measures the habit of stopping to check the answer that came to mind first.
In Frederick's data, MIT students had the highest average score of the groups he tested. Yet there too, as at Harvard and Princeton, many students gave the fast wrong answers.
Daniel Kahneman discusses the bat and ball in his 2011 book as an example of System 2 failing to check System 1. Kahneman won the Nobel prize in economics in 2002. He borrowed the terms "System 1" and "System 2" from the psychologists Keith Stanovich and Richard West.
The third question uses the same idea as the duckweed lake puzzle on this site: with daily doubling, the second half fills in the last day.
In any problem of the form "together T, one costs D more", the fast answer T − D is exactly twice the right answer (T − D) / 2. With $100 and $99, the ball costs 50 cents, not a dollar.
Share your own explanation, an objection or a variant of the puzzle. Commenting needs a GitHub account.
I keep adding new puzzles, new ways to explain them and write-ups of the discussions they start. Leave your email and I'll write when something new is out. No spam, one-click unsubscribe.
The sign-up form will appear here soon.