On Saturday mornings, my wife and I take turns getting the paper and the morning coffee, and we relax with it for a half hour before starting the weekend routine. My spouse has become used to my reaction when I turn to the magazine that comes with our paper: USA Weekend. Or more accurately, she has become used to covering her ears. When I put my Evidence-Based Living hat on, I believe that USA Weekend’s science reporting could at least enter any “worst of the year” contest.
But then I realized: This may be a “teachable moment” for me and others! On the positive side, it’s nice that relatively heavy coverage is given in USA Weekend to scientific findings. Their health and lifestyle articles are filled with “a recent study shows…” And they make many recommendations regarding nutrition, much of it supposedly based on science. There is even a celebrity panel called “The Doctors” who purport to answer your health questions.
Ah, but the road to you-know-where is paved with good intentions. And what you get from USA Weekend is almost the opposite of good evidence-based advice: It’s a mish-mash of simplistic inferences from individual studies mixed in with folk wisdom and anecdote – and it’s nearly impossible for the lay reader to tell the difference. As such, it’s a great example of exactly the kind of “science journalism” you should avoid taking too seriously. Let me give you a few examples of where the scientific advice provided in USA Weekend should have the label “Let the Reader Beware.”
1. No access to the original research. I am willing to be corrected on this, but nowhere on the USA Weekend site could I find any citations to the original studies. Evidence-Based Living always recommends you go back to the original scientific articles before believing the media, but so little information is given in a typical USA Weekend story that I couldn’t even determine what research was being referred to. If you can’t find the article, how do you know if the finding is real or not?
2. Reliance on a single study (or two). Regular readers of Evidence-Based Living know one cardinal rule: Never believe a single study (or a couple studies). Very often, articles in USA Weekend state: “Swedish scientists have found…” “New Research Shows…” “Two studies found,” “According to research presented at the American Chemical Society.”
What do we really need? All together now, EBL-ers: Systematic reviews of all available research leading to evidence-based practice recommendations. We need to see a finding replicated over and over, using rigorous scientific methods. We want those findings peer-reviewed by other scientists. And we want to know that they work outside of a controlled study. A couple of studies never prove a point, so we should not base our health-related behavior on the findings of a single study (and that’s what almost all scientists tell you at the end of their articles).
Just to give one example, USA Weekend reports that snoring is related to metabolic syndrome. In the closest article to this assertion I could find, the scientists qualify the finding extensively, including that the study is limited by the measures it used, by a small subsample, and by the cross-sectional (one-time) nature of part of the study. Where’s that information, USA Weekend?
3. Quick and confusing generalizations. The Doctors in USA Weekend make the somewhat astonishing recommendation: “Stop counting the calories (if you’re a woman over 65)” and they go on to suggest that it may be better for you stay at your current weight, because “Older women who lose weight can double their risk of hip fracture.” Now try as I might, I couldn’t find the exact reference, although there is research suggesting that weight loss can affect bone density negatively. But this says nothing about the total picture. Should a morbidly obese, diabetic person not lose weight because of a potential increase in hip fracture? Probably not, because the other obesity-related health problems can trump the increase in hip fracture risk.
Here’s a study idea for you: I wonder how many women read that comforting advice and dropped their diet, even if they are very overweight and at no particular risk of hip fracture. That’s why simple generalizations about studies do more harm than good usually.
What’s the lesson here? These snippets of information won’t necessarily do you any good unless you know where they come from, how the study was done, and how it applies to you. Does it fit with other scientific research? We’re told in this week’s issue that we should “sprinkle on the cumin” because “In a scientific study from India, cumin was found to be just as effective as an anti-diabetes drug in controlling diabetes in lab rats.” Does that apply to you? Who knows?
So go to the source whenever you can, and take your Saturday paper’s science reporting with a grain of salt!
It was clear from context that the student above had read the entire work, and the student clearly understood the conclusion of Saint Thomas’s argument. However, he did not understand the most important part: how Saint Thomas reached that conclusion. He grasped the external features of the treatise, but he did not comprehend its internal anatomy of ideas. Though intelligent and possessing a keen memory, the student had learned to read in a certain way that was only useful for extracting information. He had not learned how to read beyond that level. He had not practiced reading in a way that allowed him to grapple substantively with an idea. Thus he could not provide any useful commentary of his own, only summary.