I am guessing that many families reading the paper at breakfast today had this happen: Somebody said to someone else: “See, I told you drinking diet soda was bad for you!”
And that is because of a study reported widely in the media regarding the relationship between consumption of diet soda and stroke. Strokes are very bad things, often devastating the person to whom they occur, so a finding about anything that might increase our risk for stroke is worthy of notice.
At Evidence-Based Living, one of the most fun things we do is to track back from the media coverage to the actual research findings. In so doing, we hope to help people figure out the nature of the evidence and whether we should immediately change our behavior. This was an unusually big story, and so we ask: Believe it or not?
First, let me say that media coverage was a little more measured than usual. Some news outlets did use headlines like that from Fox News: “Diet Soda Drinkers at Increased Risk for Stroke” which make it sound like a firm finding (and probably led to some of the heated breakfast-table conversations). But many other outlets included the all-important “may” in the headline, and the articles themselves included qualifications about the study.
So let’s take a look at this finding, using some of the key questions EBL recommends you always employ when you are trying to figure out whether a scientific finding should change the way you live.
1. What kind of a study was this? Was it a good one?
This is what scientists call an observational study. It was not a randomized, controlled experiment in which some people were asked to drink diet soda and others were not. It uses a longitudinal study called the Northern Manhattan study (or NOMAS). And yes, it is a very good study of its kind. It looks at stroke risk factors across white, black, and Hispanic populations living in the same community (northern Manhattan). It is a large and representative sample, followed up annually to determine if people suffered a stroke (verified by doctors on the research team). Many publications in top referred scientific journals have been published from the study (some of which are available for free on the website).
2. Where did the information in the media come from?
Here, in EBL’s opinion, is the first problem. The results were presented at a scientific conference this week (the American Stroke Association). This is not the same as being published in a referred scientific journal. In addition, we cannot follow an EBL cardinal rule: Go to the original article. The only information that is available on the study is from a press release issued by the association and subsequent interviews with the study’s lead author and other experts. So we need to wait until the results are published before we even think of changing our behavior in response to them.
3. Are the results definitive?
No, no, and again no. There are some good reasons not to drink diet soda (including possible increased risk of diabetes and osteoporosis), but these findings do not “prove” that diet soda leads to strokes.
Some reasons why this is a very tentative and preliminary finding include the following:
All the data are self-report, so we are dependent on people remembering their diet soda consumption.
It’s the first study to show this association. EBL readers know that we need multiple studies before we even begin to think about recommending behavior change.
It’s not all diet soda drinking: It looks like only people drinking diet soda every day show the association with stroke, suggesting that lower consumption may not increase risk.
The study is not representative of the U. S.population. First of all, you had to be over 39 years old in 1990 to get in the study and the average age of the sample now is in the late 60s, so the results can’t be generalized to younger people. Further, the sample for this study included 63% women, 21 % whites, 24 % blacks and 53 % Hispanics. In the U.S as a whole, 51% of the population are women, 77% are white, 23% black, and 16% Hispanic. So it’s a very different group from what a random sample of Americans would get you.
We don’t know the reason for the association. The lead author, Hannah Gardener, is open about this: “It’s reasonable to have doubts, because we don’t have a clear mechanism. This needs to be viewed as a preliminary study,” By “clear mechanism,” she means that even if this relationship exists between diet soda and stroke, we don’t know why.
There’s more we could say, but our main point is this: It doesn’t take very long for you to “deconstruct” what the actual evidence is behind a news story. With a basic understanding of how studies are done and access to the Web, you can often find out as much as you need to know. In this case, the media have reported the first highly tentative findings of an association between two things. Now other scientists need to test it again and again to see if it holds up, as well as finding out why the association exists.
I go for sparkling water instead of diet soda because of other problems mentioned earlier with diet beverages. But regarding stroke risk, the data just aren’t there yet.
The blog is based on one key principle: Now more than ever, people need help separating the good scientific information from the bad. We are all about assessing the scientific evidence on human problems and looking at how to use it every day. You can click on the topics above to see blog posts on specific subjects.