Hundreds, possibly thousands of health studies may be wrong because researchers ignored human fallibility
You may have never heard of the Energy Balance Working Group, but this collection of 45 experts on nutrition, exercise, biochemistry, and other related disciplines have collectively thrown a “House-like” wrench into the research literature on everything from obesity to cancer and heart disease. Gregory House, the fictional and fantastically brilliant physician played by Hugh Laurie in the eponymous TV show frequently found his patients wanting in the court of self-reported truth: “I’ve found that when you want to know the truth about someone that someone is probably the last person you should ask.”
This is more or less what the Energy Balance Working Group have concluded in an “expert report” recently published in the International Journal of Obesity. If you want to know the truth about how much someone eats and exercises that someone is probably the last person you should ask.
“There are at least two categories of solutions on the horizon. In one category, there are wearable monitoring devices that can collect objective, real-time data. Examples in the works or in use include photographic food diaries, records of chewing and swallowing behavior, and evaluating the time and intensity of movement using accelerometers and GPS, among others. It is important to note that there are still challenges converting these measurements into reliable estimates of energy intake and expenditure, but work is ongoing. A second category includes biochemical and physical techniques that estimate the amount of energy a body has, such as the amount of body mass stored as muscle or adipose, and the amount of energy flux through the body, such as the doubly labeled water method. These methods are more accurate but also tend to be more expensive and difficult to use on a very large scale. Other methods are being proposed as technology advances, and it will be exciting to watch the methods emerge and evolve from fields such as physics, engineering, and basic chemistry. Such methods can only improve if they are adequately funded and if we can collectively admit that science needs something better than just asking about an individual’s energy balance” — David Allison, Distinguished Professor,
Quetelet Endowed Professor of Public Health, University of Alabama
In the technical language of the report “Energy Intake” (EI) and Physical Activity Energy Expenditure” (PAEE) are measures that are routinely misremembered when people are asked in health surveys to recall what they have eaten and how much they have exercised. For example, when self-reported estimates of energy intake are compared with the “gold standard” laboratory technique for estimate energy expenditure in humans (doubly labeled water), people systematically underestimate what they eat by hundreds of calories a day. When it comes to physical exercise, people tend to over-report their exertions.
And yet, even though these limitations are widely known, much research continues to accept the gospel of memory as a basis for scientific conclusions about health:
“Subjective methods of consumption are not only used for studies related to energy balance and body weight, but also to draw conclusions that inform clinical practice and dietary recommendations for conditions such as chronic kidney disease, cancers, heart disease, diabetes and Alzheimer’s disease. The lack of validity of some messages intended for clinical application is highly disconcerting. Inaccurate dietary data collection can result in spurious associations and effects that can directly impact patient care.”
How many findings are now presumptively invalid due to such self-reported data? We don’t know, says the University of Alabama’s David Allison, one of the lead authors of the expert report. “We can say that all conclusions based solely on self-reported energy intake or expenditure cannot be considered substantiated.”
The problem is that self-reporting is a cheap and convenient source of data for research, while more accurate alternatives are either expensive and challenging or, as yet, more promise than reality (see sidebar). But this shouldn’t, the report says, provide a rationale for continuing to fund studies based on inadequate measurement tools. Funding needs to be directed to solve this problem now, given the clinical and societal need for accurate measurement of energy intake and expenditure. As the authors note, “in this case, the adage ‘something is better than nothing’ must be changed to ‘something is worse than nothing.’”
Recent Comments