We’ve talked about the issue with ‘single study syndrome’– promoting the results of a single study without including the context of the full body of science. We’ve talked about when headlines are grossly out-of-step with a study’s actual conclusions. Sometimes, you get all these Science Sins wrapped into one.
Today, Daily Mail told us to ‘forget counting calories, just cut out sugar’ (forget decades of research on energy balance and weight loss), while Time said ‘Sugar Is Definitely Toxic, a New Study Says’ (Wow, this must have been a very comprehensive study…oh, wait). And don’t forget the lead author chiming in on the Guardian stating ‘This study establishes that all calories are not the same.’ Really? Why must that myth come up EVERYWHERE? Here’s what’s getting missed about this new study:
Length of Study: Short-term studies (and this is an extremely short one, at 9 days) can provide misleading results. While this is an important topic, a study of longer duration would provide us more conclusive “answers” that are more applicable to long-term health decisions. Dr. Andrew Brown shed light on how short study length can create some problems:
It’s important to remember that these short term studies (which I myself have conducted) can lead us in the wrong direction sometimes. Take for example cholesterol: blood cholesterol can go up with short term increased intakes of cholesterol (within reasonable limits), but tends to stabilize after a couple weeks, leading to negligible perturbation of blood cholesterol.
No Control Group: The study authors state, “each participant served as his or her own control.” We’ve talked before about how control groups are essential to avoid confounding factors. This study looked specifically at children (n=43) who were enrolled in an obesity intervention program – a pretty niche population – without having a control group, so a number of factors may alter the findings of this study.
No Baseline Measurement: Dr. Brown highlighted that the study had no control group; no actual measurement of baseline diet; and no comparison between sugar and starch. Subjects filled out a food frequency questionnaire (FFQ) about their eating habits, which often under-report food intake. In fact, because of inaccuracies reported in the FFQ, authors had to make mid-study dietary adjustments to compensate for weight loss in subjects, indicating that basal calorie predictions based on the FFQ responses were not accurate.
The biggest concern?
The study title and resulting headlines are curious given the data and protocol described in this paper. They don’t match up. We appreciate the focus of study — it’s an important conversation to have. Some people do over consume sugars (along with many other things in their diet), and this short-term study may provide insight into dietary modifications that spurs future research that is helpful to such populations. However, break-through scientific “answers” can’t be concluded from studies of this design or duration. As Dr. Brown expressed, “everything [in the study] is confounded, and although interesting as a starting point for more research, the conclusions far overstate the inferential value of the study.”
The path to good health is a much longer, more rigorous road than this study purports.