A recent story in The Washington Post’s Wonkblog claims that that, by looking at people who wind up at the Emergency Room, you can figure out how often drinking leads to injury. The Post’s headline—”Study: Having just one drink doubles your risk of getting injured”—suggests that all drinking is inherently dangerous, even at surprisingly moderate levels: one drink doubles your risk, and three drinks increase your risk by a whopping factor of five.
But are those who stumble into the emergency room after drinking telling us something generalizable about everyone who drinks, or the risks to anyone who drinks? Is teetotalism now to be thought of as the equivalent of wearing a helmet while cycling, a habit everyone should adopt?
Let’s take a look at the study’s claim and the methods that generated this data. First of all, the claim is not about the absolute risk of ending up at the ER if you have been drinking; it’s about comparative risk. The problem is this: when you only have people who have been injured in the hospital to look at, to whom do you compare them?
The study, published in advance online by the journal Addiction is a matched case-crossover study using data from 18 countries and 37 emergency departments. And it is designed to compare people who were injured to… themselves.
People in the ER were interviewed about their drinking. Specifically, they were asked if they drank in the six hours prior to arrival at the ER and, if so, how many drinks did they consume. Then they were asked how much they drank in the same period the previous week.
Ignoring, for a moment, the seemingly vital question of how much an injured person drank, there are four possibilities for each individual:
- He/she could have drunk alcohol before the injury, and also the week before;
- He/she could have drunk alcohol before the injury, but not the week before;
- He/she could have not drunk alcohol before the injury, but did drink the week before; and
- He/she could have not drunk alcohol either before the injury nor the week before.
After collecting this information for each person, the data is sorted according to the number of drinks consumed, and a comparison is made to get what’s called an odds ratio of risk of injury for each drinking level. In this case, the odds ratio is the number of people who drank within six hours of coming to the ER, but did not drink the previous week, divided by the number of people who did not drink before their injury, but who did drink the previous week.
In theory, designing a study in this way has its advantages. Because people are matched to themselves, you don’t need to worry that you didn’t control for confounders like income or race. But there are some notable disadvantages too: people are asked to remember whether they had drunk a week prior in a specific six-hour window. All of the people being asked have just suffered an injury that brought them to an ER. And some of the people being asked are drunk.
That leaves us to wonder how accurate the data are, and whether better measurements would make the odds ratio higher or lower.
But suppose we treat the data as if it were accurate and not skewed by asking drunk and injured people what they drank the previous week. What do we learn about ER injuries and drinking? The study’s conclusions imply that drinking is correlated with an increased risk of injury, but quantifying that risk is not quite as easy as establishing an odds ratio.
The report calculated the odds ratio using two different models, and resulting in similar levels of increased odds. For the sake of argument, we use the higher odds ratio of 2.7, which can be understood as follows: Count the number of people who had a drink and subsequently ended up in the ER, but did not drink at the same time the previous week. Divide this by the number of people who did not have a drink six hours before their accident, but did have a drink at a similar time the week before. This is the odds ratio in the context of this study, in which the authors want to directly compare the relationship of drinking to ending up in the ER by eliminating the data about injuries of people who always drink or people who never drink.
If it were equally likely to be injured whether you are drinking or not, you would expect the number of people who drank previously but not the night of the injury to be similar to the number who drank on the night of the injury but not previously. The ratio 2.7 implies that, among these injured people, it was much more likely that they were drinking the night of the injury.
To give the numbers some context, suppose that there were 2700 people who had one drink before their injury but not the previous week, and 1000 people who drank the previous week, but not before their injury. This results in an odds ratio of 2.7 (2700 divided by 1000). This does not directly say that drinking causes injury, but it does suggest that, if you’re caught in the ER (and you do occasionally drink), then you were more likely to have been drinking six hours prior to ending up in the ER.
The authors of this study, of course, don’t know the actual risks of injury associated with having a single drink; the beauty of the study design is that they don’t have to know it. How we might understand the actual risk of injury due to drinking in the context of this study? We cannot. We can only talk in comparative terms. It may be that every 1 in 100 people who knocks back a drink ends up in the ER, or it could be that 1 in 10,000 people have this experience.
Some might ask what the difference between the odds ratio and a relative risk is, in the context of this type of study. In this case, it’s tricky and not the least bit confusing for the language in the literature. In the case of a pair-matched case control study of people who end up in the ER, we have formed the ratio of those who drank the fateful night, but not the week before, and the number of folk who didn’t drink the fateful night, but did the week before. This is comparable to the relative risk of ending up in the ER after drinking the fateful night (if you’re an occasional drinker) and ending up in the ER after not drinking the fateful night (but drinking the previous week). Sometimes researchers call this a relative risk, but it is not the “usual” relative risk that would be the first to come to mind.
The usual relative risk would be calculated also using data about those who drink a lot or who don’t drink at all. It would be the absolute risk of ending up in the ER six hours following drinking, divided by the absolute risk of ending up in the ER if you didn’t drink six hours prior to the accident time (and no one cares about what you did last week). In this scenario, the risk of injury following drinking would include the injuries of people who drink all the time, and the risk of injury following not-drinking would include the injuries of people who never drink. As we said before, we don’t know the actual risk of incurring an injury while drinking, so this particular risk level cannot be computed using the data collected.
Even with the caveat about odds ratio as opposed to relative risk, an important distinction should be made between an increase in odds and the factor change of odds. If the odds of injury after consuming one drink are 2.7 times as high, then the increase in odds is just 1.7 times the original odds.
One can see how this point escaped journalists. As the Post reported, “With three glasses of wine, your risk of hurting yourself and going to the E.R. increases by a factor of five.” Presumably the Post went for the more aggressive estimate of the two models reporting on three drinks, resulting in an odds ratio of 4.6 at three drinks. A bit of fine print points out that this bigger odds estimate is based on a novel approach to analysis that narrows confidence intervals, and it includes any number of two to three drinks. Nonetheless the Post reported an increase in a factor of 5, when at worst it’s an increase in a factor of 3.6. According to the other, more traditional estimate, the odds are 4.1 for injury following drinking 2-3 drinks. So, drinking more increases your odds of injury by a factor of 3.1, not five. This is because the initial odds are 1.0
Notwithstanding the question of how the statistics are reported, the bigger problem with the study is who ends up in the ER. A better way of describing this study would be to say that it demonstrates that if you’re likely to end up in the ER anyway, drinking is correlated with an increased likelihood.
Why the ER isn’t necessarily a microcosm of the world
The study is unlikely to apply to wide swaths of the population: people who just don’t take that many risks that could lead to injury, people who drink quietly or outside the context of big parties or activities, people who don’t get into cars after drinking. Perhaps violent people get more violent when drinking, leading to the violent injuries reported in this study—but nonviolent people don’t have any resulting problems when drinking.
Finally, there is the more mundane criticism that the Wonkblog takes an association and suggests causality. Sure, drunk driving is real and leads to serious injury; but some activities that are risky are undertaken more often when drinks are also present, and may not be directly related to the drinking. Drinking correlates with social activity, and social activity itself correlates with injury.
The data seen in this study of ERs could have emerged from a situation in which risk is entirely associated with going out. To illustrate this point, here’s a hypothetical data set in which you see the numbers the hospital sees, but alcohol, by assumption, plays no role.
Suppose a community is completely uniform in terms of risk of injury and drinking habits. In this theoretical world, the only risk of injury is due to going out in the evening, which results in a 10 percent, risk of injury, regardless of drinking status.
Everyone drinks 10 percent of the time. When you are drinking, you go out ½ the time, stay in the other ½ the time. When you’re not drinking, you go out about 20 percent of the time.
Now we do the numbers: if you’re drinking, the chance you’re going out is 0.5 and the chance of injury is 0.1, resulting in a .05 chance of injury. If you don’t drink, you will go out with a probability of 0.2 and you will get injured with probability 0.1, resulting in a net probability of ending up in the ER .02. The ratio of drinking to not drinking is .05/02, or 2.5. Yet by assumption, going out is the risky activity and drinking is simply correlated with it.
In the specific conditions of people arriving in the ER, drinking may be correlated with injury; however, the increase in risk was exaggerated by The Washington Post. Attention needs to be paid to describing the increase in odds more clearly and not confusing it with the factor of change.
Please note that this is a forum for statisticians and mathematicians to critically evaluate the design and statistical methods used in studies. The subjects (products, procedures, treatments, etc.) of the studies being evaluated are neither endorsed nor rejected by Sense About Science USA. We encourage readers to use these articles as a starting point to discuss better study design and statistical analysis. While we strive for factual accuracy in these posts, they should not be considered journalistic works, but rather pieces of academic writing.