06 February 2014

Random rant about denominator-free statistics

Not particularly relevant to anything, but offering a chance for a random science lesson, I got really annoyed at a couple of items in my twitter feed over the last few minutes.

The first (in which the tweet was ridiculing the claims) linked to this article about a new journal paper which claimed:
Cat bites to the hand are so dangerous, 1 in 3 patients with such wounds had to be hospitalized, a Mayo Clinic study covering three years showed. Two-third of those hospitalized needed surgery.
Regular readers will know my lack of respect for Mayo's attempts at research on anything other than medical treatments. But this is worse than usual. Contrary to the headlines that this claim is designed to generate, obviously only a tiny fraction of cat bites result in hospitalization. Their error, of course, is using "patients" as the denominator, a completely useless population to base the statistic on. Who becomes a "patient"? Obviously it is people who are suffering serious medical problems. In the case of cat bites, this consists of people who, after being bitten by their cat many times before, and suffering no serious problems from it, develop an infection that will not go away. Does this represent 1/1000th of all cat bites? 1/10,000th? Fewer? We have no way of knowing, so the statistic is worthless. More important, we do not know what degree of seriousness typically makes someone a patient. Someone with a non-trivial cat bite injury has no way of guessing her chance of needing hospitalization or surgery because there is no way of knowing how serious the condition of the average patient was in comparison.

(Note: People bitten by stray cats of unknown rabies status might also end up as patients and be hospitalized on spec, which further biases the number away from how it is being interpreted.)

Note that this is exactly the same error that infection disease "experts" always make when there is an exciting disease like bird flu.  They claim things like "half of those who get it die." Um, no. Half of those who get it and get so sick from it that they are willing to go to the almost-certainly-low-quality hospital that they cannot afford (think about the populations in question) end up dying. A bit different.

(If you are particularly interested in that observation, I have written about it at some length in this blog, but I am feeling too lazy to go look for the link.)

The other version of denominator-challenged innumeracy was this tweet from a group calling itself the Center for Priority Based Budgeting, which was naively retweeted by the Robert Wood Johnson Foundation's public health feed.  It included a coded map of the USA and the observation "Half of the nation's uninsured live in just 116 counties".  I am not going to go check this, but eyeballing the map, I would guess that half of the nation's people live in those 116 counties, so this is not exactly exciting news.

Seriously, did everyone who understands numbers from the last couple of generations go to Wall Street, Google, and the NSA, leaving the rest of the world innumerate?

No comments:

Post a Comment

Note: Only a member of this blog may post a comment.