There is nothing more grating than having a journalist toss around numbers showing no understanding of conditional probability (actually, there are 12 more grating things, but this ranks right up there). In a nice story from NBC Chicago, journalists Dick Johnson and Andrew Greiner write about an autistic teen who has picked the first two rounds of the NCAA tournament correctly:
An autistic teenager from the Chicago area has done something almost impossible.
Nearly 48 games into an upset-filled NCAA tournament, 17-year-old Alex Herrmann is perfect.
“It’s amazing,” he says. Truly.
Yes it is amazing. But the writers get tripped up when trying to project the future:
There are still four rounds remaining, so it could fall apart — the odds of a perfect wire to wire bracket is about 1 in 35,360,000 by some measures or 1 in 1,000,000,000,000 by others.
Aaargh! Let’s let pass the factor of 28,000 or so difference in estimates. THIS IS NOT THE RELEVANT STATISTIC! We already know that Alex has picked the first two rounds correctly. We are interested in the probability he has a perfect bracket given he picked the first 48 games correctly. This is about the simplest version of conditional probability you can get.
If all he did was flip a coin for each of the remaining 15 games, he would have a one in 32,768 chance of having a perfect bracket, given where he is now. Not great odds, certainly but nothing like the probabilities given in the quote. You can argue whether 50/50 on each the fifteen remaining games is the right probability to use (Purdue as champion?), but there is absolutely no justification for bringing in the overall probability of a perfect bracket. By quoting the unconditional probability (and who knows where those estimates come from), the writers vastly underestimate Alex’s chance of having a perfect bracket.
I swear I see the confusion between unconditional probabilities and conditional probabilities twice a day. I suspect the stroke that will finally get me will be caused by this sort of error.
Edit. 9:06PM March 23. From the comments on the original post, two further points:
- The writers also seem confused about joint probabilities:
- It appears that if you manage a pool on cbs.sportsline.com, you can edit the picks after the games. That might be a more reasonable explanation for picking 48 games, but it is hard to tell.
One in 13,460,000, according to BookofOdds.com. It’s easier to win the lottery. Twice.
No… not unless your lottery has the probability of winning of one in the square root of 13,460,000, or one in 3669. While there are “lotteries” with such odds, the payoffs tend to be 1000 to 1, not millions to 1. I bet they thought winning one lottery might be 1 in 7,000,000 so two lotteries “must be” 1 in 14,000,000. No, that is not the way it works.
So, to enumerate what journalists should be tested on, lets go with:
- Conditional Probability
- Joint Probabilities
- Online editing possibilities
You are welcome to add to the certification test requirements in the comments.
Your final comment is too good to let pass.
Anyone who has ever taught introductory statistics knows that conditional probability concepts are seriously counter-intuitive. There is actually a whole literature in the psychology of judgment and decision making on problems people have with conditional probabilites, including a reasoning error known as “confusion of the inverse” (see http://en.wikipedia.org/wiki/Confusion_of_the_inverse for a pretty good description). And it is not just journalists that screw this up. I recommend the chapter by David Eddy cited in the wikipedia article. People, including people with the letters “MD” after their name, often get confused about the difference, say, between the probability of a patient having a stroke conditional on a positive test result versus the probability of a positive test result conditional on the patient having a stroke. You want your doctor to think about the former, but the scientific literature tends to report the latter. So heaven forbid this happens to you, but if they ever do wheel you into the ER, let’s hope your doctor doesn’t try to compute Bayes’ Theorem in his or her head! They are not that good at it!
Amen Mike. Another “proof” are the casinos in Las Vegas, which make big profits thanks to the number of people who believe in “luck” and “streaks” rather than probability.
Re Don Kleinmuntz’s comment, I’ve fought the conditional probabilities battle in both stats classes and non-mathematical classes (introductory org behavior, where the topic is managers making decisions based on faulty understanding of stats). One device I’ve had some success with is the pregnancy test. I conjure up a hypothetical pregnancy test with false positive and false negative rates. Then I (hypothetically) administer it to one of the male students, get a (hypothetical) positive result, and ask what the probability is that he’s actually pregnant. I’m not sure it gets the entire concept across, but it at least convinces the students that what seems obvious to them is not correct.
Some nice remarks about general confusions related to probability statements:
http://research.microsoft.com/en-us/um/people/minka/papers/nuances.html
Still more interesting commentary here [sciencenews.org].