If you have taken an undergraduate psychology class, you probably have heard about the experiment where monkeys, having previously shown little or no preference between red, blue, and green M&Ms are offered them in pairs. First blue and red is offered, and the one not chosen (say blue) is offered against green. About 2/3 of the monkeys choose green. Psychologists claim this is a sign of cognitive dissonance: information is ignored or used selectively to confirm biases. If blue is not chosen in the first round, it must be “bad”, so it is less likely to be chosen in the second round. Not so fast! If the monkey does have a preference among the colors, then perhaps the monkey is acting consistently. There are six choices for an ordering of the colors:
-
R G B
-
R B G
-
G B R
-
G R B
-
B R G
-
B G R
Choosing Red over Blue means that cases 3, 5, and 6 do not occur. In 2 of the three remaining cases, Green is preferred to Blue!
This is reminiscent of the infamous Monty Hall problem, as explained in the New York Times, in an article covering the cognitive dissonance issue:
Here’s how Monty’s deal works, in the math problem, anyway. (On the real show it was a bit messier.) He shows you three closed doors, with a car behind one and a goat behind each of the others. If you open the one with the car, you win it. You start by picking a door, but before it’s opened Monty will always open another door to reveal a goat. Then he’ll let you open either remaining door.
Suppose you start by picking Door 1, and Monty opens Door 3 to reveal a goat. Now what should you do? Stick with Door 1 or switch to Door 2?
…
You should switch doors.This answer goes against our intuition that, with two unopened doors left, the odds are 50-50 that the car is behind one of them. But when you stick with Door 1, you’ll win only if your original choice was correct, which happens only 1 in 3 times on average. If you switch, you’ll win whenever your original choice was wrong, which happens 2 out of 3 times.
Unlike the Monty Hall problem, which is really not much more than a parlor game, the effect of not seeing the monkey issue is more far-reaching. A Yale economist, M. Keith Chen, who noted the issue, believes that this goes to heart of a lot of testing in psychology:
Dr. Chen remains convinced it’s a broad problem. He acknowledges that other forms of cognitive-dissonance effects have been demonstrated in different kinds of experiments, but he says the hundreds of choice-rationalization experiments since 1956 are flawed.
Even when the experimenters use more elaborate methods of measuring preferences — like asking a subject to rate items on a scale before choosing between two similarly-ranked items — Dr. Chen says the results are still suspect because researchers haven’t recognized that the choice during the experiment changes the odds. (For more of Dr. Chen’s explanation, see TierneyLab.)
It is striking that such an obvious point, and one that relies on only the most rudimentary understanding of probability, took decades to see. I wonder if ten years from now, we will be wondering about our current conundrums: “P=NP? Yeah, isn’t it amazing that the field missed the obvious fact that …” Now fill in the “…”.