I have previously written on how decision makers (and journalists) need to know some elementary probability and statistics to prevent them from making horrendously terrible decisions. Coincidentally, Twitter’s @ORatWork (John Poppelaars) has provided a pointer to an excellent example of how easily organizations can get messed up on some very simple things.
As reported by the blog Bad Science, Stonewall (a UK-based lesbian, gay and bisexual charity) released a report stating that the average coming out age has been dropping. This was deemed a good enough source to get coverage in the Guardian. Let’s check out the evidence:
The average coming out age has fallen by over 20 years in Britain, according to Stonewall’s latest online poll.
The poll, which had 1,536 respondents, found that lesbian, gay and bisexual people aged 60 and over came out at 37 on average. People aged 18 and under are coming out at 15 on average.
Oh dear! I guess the most obvious statement is that it would be truly astounding if people aged 18 and under had come out at age 37! Such a survey does not, and can not (based just on that question), answer any questions about the average coming out age. There is an obvious sampling bias: asking people who are 18 when they come out ignores gays, lesbians, and bisexuals who are 18 but come out at age 40! This survey question is practically guaranteed to show a decrease in coming out age, whether it is truly decreasing, staying the same, or even increasing. How both the charity and news organizations who report on this can’t see this immediately baffles me.
But people collect statistics without considering whether the data address the question they have. They get nice reports, they pick out a few numbers, and the press release practically writes itself. And they publish and report nonsense. Bad Science discusses how the question “Are people coming out earlier” might be addressed.
I spent this morning discussing the MBA curriculum for the Tepper School, with an emphasis on the content and timing of our quantitative, business analytics skills. This example goes to the top of my “If a student doesn’t get this without prodding, then the student should not get a Tepper MBA” list.
Added December 2. Best tweet I saw on this issue (from @sarumbear):