I am in a foul mood today. I woke up, at 4:45 am courtesy of my baby, generally ticked off and my day only got worse. So I did what I always do when I get in a funk: call my sister and whine. And then I went around to various health & fitness blogs making comments pointing out the errors in their research, a favorite bad-day pasttime of mine. (If your blog was one that ended up in my cross-hairs, I do apologize. Like I said, bad day.)
I don’t know what it is but the health and fitness industry seems particularly liable to either a) use a study inappropriately (make conclusions that neither the researchers nor the data support) or b) use poorly designed studies (those funded by drug companies, say, with a sample group of 3). Nothing incurs my wrath more than bad research or the willful misrepresentation of good research.
The Lesser Evil
Of course the worst end of the spectrum are the dubious weight-loss supplement/diet plan ads. They always feature a good-looking, slim “doctor” (perhaps a PhD of liberal arts? Or an actual MD that they managed to buy off?) in an official looking white coat. The doctor gives wildly fantastic promises about said product.
Take this one for Zantrex-3: “546% more weight loss!” 546% huh? That’s a pretty specific number. How do they know it’s not actually 547%? Either way it is a ridiculous, meaningless number. What exactly does 546% more weight loss look like? Do you shrink down until you are a dot? Negative space? Should I be scared? The fine print at the bottom of the page says “The “546% more weight loss” claim is based solely on Zantrex-3’s active weight-loss component. However, Zantrex-3’s Super Stimulant® has been shown to produce additional weight loss in some studies.” Wow – did you catch that? Their special proprietary secret ingredient (that they won’t say what it is) has been in “some studies.” What happened to the “other” studies? Did they, oh, not support the ludicrous claims? Quick, somebody call the doctor! Or anyone wearing a white lab coat – they’re the same thing, right? And of course the ad ends with the ubiquitous “Individual results may vary.” You think?
The Greater Evil
Everyone knows the diet pill/diet ads are ridiculous though. My real beef is when fake research masquerades as real research, like when it was recently discovered that Prozac and some other anti-depressants had specifically only published studies that showed them in a favorable light, suppressing all others. Or when you read ads extolling the virtues of dairy – that are funded, researched and published by the National Dairy Council. (Not that I have anything against dairy, per se, just pretend research)
The Evilest Evil
The most egregious sin in my book though is when people take legitimate research and extrapolate erroneous or even fallacious conclusions. A perfect example is when people take mice studies and apply them to humans without mentioning the fact that the research was done on mice. Another great example is this study (a well designed and interesting study, I might add) about what happens to runners when they stop exercising and then what happens to them when they start up again. The study found two major conclusions:
1. Consistency is best. Stopping running will cause you to gain weight.
2. Your fitness level makes a difference though. People with a high level of fitness easily shed the gained weight after returning to their running. People with a low level of fitness not only gained 4 times as much weight during their off time but had a much harder time losing it again once they resumed running.
See? Interesting! Informative! And yet one widely-read blog, that specializes in promoting weight lifting and interval bursts over endurance cardio, spun it as: Don’t start running because if you ever stop, you’ll gain weight! Runners who quit exercising gained more weight than their sedentary peers!! AAAAGGGGHHH! Another blog, through a popular running magazine, spun it this way: Runners should never stop a running program and should keep adding miles to up their fitness level, or at least not reduce their mileage, lest they get fat. Aaaaagghhhh! Same study, completely contradictory conclusions.
And of course the comments on both these sites reflect what the bloggers intended: to spin the research to support their pet theories. The research, being much less sensational, doesn’t support either one of these claims.
Will No One Think Of the Children?
Save the research! Save the studies! Campaign for truth in blogging! Oh, wait… ahem.
Like I said, I’m in a foul mood. (But you know what made me feel a teensy bit better? That mock ad!! Har! It’s totally worth all the squinting you have to do to read it.)
amen! as a scientist, it is so irritating to be reading about some study and then the article/post wraps up with the “lesson” and it totally misrepresents the findings or has nothing to do with them! ugh! bad day here, too. 🙂
You see, though, each person will interpret a given set of data in his or her own way, erroneous or not. You can’t fault a person for drawing conclusions different than those you would draw.
Additionally, the entire point of mouse work in science is to be able to extrapolate the results to another mammal. As humans aren’t lining up to take part in studies, researchers must make do with the reagents at their disposal.
Lastly, whenever you see an ad for a drug or particular food product, you can rest assured that the “research” they are touting was funded by the company, and only the good parts are shown. No matter what company and what product.
Gena – you bring up some great points! I like how you think – you sound like me:)
1. I would never fault anyone for stating their opinion. But there is a difference between saying “Here is the data – this is what this study means to me” and saying “See? The numbers prove me right!” whilst actively misreporting or selectively reporting the study. I do agree that no research can be done without human bias entering the equation but a good researcher and analyst will do their best to limit it.
2. You kinda missed my point with the mouse studies. I have no problem at all with scientists using mice in studies or even attempting to extrapolate the data to humans and other mammals. My problem comes when people cite these studies WITHOUT mentioning they were done on mice. Because in the end, while mice can give us clues to our own biology, they can’t give us all the answers (the hoopla around leptin being the most famous example I can think of).
3. You are right on this point, sadly. There are a few national and international bodies that provide funding for nonpartisan research – the most notable being educational institutions which are classified on a research scale that has stringent standards that must be met. But quite a lot of research is done by parties that have a vested interest in the end result. Again, my problem isn’t with The National Dairy Council funding research – my problem is when a) they fund it, research it & publish it themselves. At the very least their methodology should (and could be) peer reviewed. and b)when they are sneaky about hiding their ties to a particular study.
Hope this helped clear up some points! Keep posting Gena – I love another analytical mind to bandy with.
Interesting post! As a health writer / magazine “service” journalist, I am faced with these conflicts all the time. So often the editors just want to extract the “news you can use” from any study … no matter how preliminary or unpractical! Even university press releases can be completely misleading at times … The media industry needs more responsible journalists (and more space/time to cover all the important details) in order to convey things accurately.
I’m intrigued by the topics on your blog … as I’m just getting into fitness myself, I will certainly stop back to visit 🙂
Oooh Amanda! You are what I want to be when I grow up!! Does Health magazine need any fitness writers?
Thanks for your thoughts – I do hope you stop back often:)