In my last article I was very critical of the use of before/after pics as a method for proving the effectiveness of fitness and diet products/services. My criticism begs the question, however, how can one judge the effectiveness of fitness or diet related claims? That’s a huge subject that can’t be handled adequately here. In the space of this short article, however, I would like to address the most common error people make regarding fitness and diet claims. If there is only one caveat you should pay attention to let it be this: anecdotal evidence is neither proof of causation nor correlation.
Judging the effectiveness of a fitness or diet product cannot be done by reference to anecdotal reports. That’s why no matter how many success stories they throw at you, or how many of your well meaning friends or neighbors tell you about their amazing results, you must be wary of the grandiose claims made by exercise and diet product sellers. As the sellers often admit, albeit in fine print, results with their products may vary.
We each have easy access to more information then ever before. The fact that we are not all exercise physiologists or registered dieticians does not absolve us from applying some common sense.
Take claims of “miraculous” and “effortless” fat loss for instance. Given that a pound of fat represents 3,500 calories of stored energy, how realistic is it that any one of us can lose more then 2-3 lbs. of fat per week? And yet products that make the claim that they can help users lose much more then this are ubiquitous. If you see or hear of a product that claims to melt many pounds away per week, ask yourself “pounds of what?”, because you can rest assured that the basic laws of physiology and physics have not been suspended for the product in question.
A properly examined fitness or diet product will have numerous scientific studies behind it. I say numerous because even good scientists can get it wrong and being able to replicate another scientist’s results is a crucial check on their objectivity.
Scientific studies will control for a number of variables that may affect the results of an exercise program, diet or supplement. Without these controls, we are just left with opinion, and as my father used to crudely put it, “opinions are like *@%*#, everyone has one”.
PubMed is an excellent online resource for those who are interested in what science has to say about various exercise programs, exercise movements, diets or supplements. But your interest must extend beyond just reading the study’s abstract. That just gives you the headline and most bear details of the study, much like a media report would. In order to understand whether the study was done well, you have to delve beyond the headline and conclusion, paying particular attention to the study’s methodology or the “how” of how the scientists arrived at their results.
When you look at whole studies you get an appreciation for how painstakingly scientists qualify what they claim to know or have proven. They will tell you that they only studied college age men for example and that further study needs to be done to both replicate and extend their results. That is the beauty of science: it doesn’t pretend to know what it doesn’t know. We would all do well to do the same for our own beliefs.