Evidence-based systematic reviews: As close to certainty as it gets

Sometimes when I give talks, I like to use this catchphrase: For almost all of human history, our major problem was a lack of information, but over the past half century the problem has become an overabundance of information. Not only can you access multiple opinions on any topic, but the scientific evidence can seem to be all over the place. For any social or health problem humans experience, there are typically hundreds of studies. Making things worse, the studies can seem to contradict one another.

So people have come up with a solution. Rather than simply summarizing research findings in narrative form (remember what you used to do for a high school or college term paper?), researchers conduct systematic evidence-based reviews, in which they use sophisticated methods to bring together and evaluate the dozens, hundreds, or even thousands of articles on a topic. (There’s a good summary of the methods for systematic reviews on the Bandolier site.

When thinking about evidence-based reviews, you have to decide whether you agree with one basic proposition. This proposition holds that the findings of sound scientific research studies provide more credible evidence about solving human problems than personal opinion, anecdotes, or “gut feelings.”  Not everyone believes this (or at least not all the time). For those who do agree (as we do at Evidence-Based Living), then what is required is a systematic review of the research evidence, leading to guidelines for program development and their use in actual programs with audiences we serve.

The authors of a systematic review will tell you exactly what methods they used to identify articles for the review, how the articles were critically assessed, and how the results were synthesized across studies. Then the systematic review itself is peer-reviewed at a scientific journal, providing even more scrutiny of its findings. In some cases, the authors will use highly technical mathematical methods to synthesize the findings of studies, producing what is called a meta-analysis.

A systematic review has many benefits over the kind of review that simply summarizes a bunch of studies. You’ve seen this kind of review, which usually runs something like: “Smith and Wesson found this finding, but see Abbot and Costello for a different finding” or “Although most research shows this finding, there are a number of studies that fail to support it.” The systematic review looks at why the studies differ, and can exclude those studies that have inadequate samples or methods. And by looking at many studies, it allows us to make general conclusions, even though participants and study settings might be different.

Let’s take one example of how a systematic review is different from other reviews. If you read statements by groups advocating one perspective, they usually cite just the research articles that support their position. The hallmark of a systematic review, on the other hand, is a search for all articles on a topic. They go to great lengths to find every study done, so all can be evaluated. For this reason, systematic reviews are usually done by teams, since it’s rare that an individual has the time to find all the available research. By looking at all studies, a systematic review can come to conclusions like: “All studies using method X found this result, but studies using method Y did not.”

Systematic reviews can be disappointing, because they often come up with the conclusion that the research isn’t sufficient to come to a conclusion. But that in itself can be useful, especially if there’s one published study that has gotten a lot of attention in the media, but isn’t supported by other research.

The best library of systematic reviews has been described in a previous post: The Cochrane Collaboration.” But there are plenty of systematic reviews published each year from other sources. An example is in our post on antidepressants. If you want the most definitive evidence available as to whether a program or practice works, look to systematic reviews.

Trackbacks

  1. […] == "undefined"){ addthis_share = [];}By now, our regular readers know that here at EBL, we value systematic reviews above other forms of evidence. That’s because these reviews collect all of the available […]

  2. […] = {"data_track_clickback":true};Frequent EBL readers are well aware of the importance we put on systematic reviews, studies that synthesize many articles on a given topic and draw a conclusion about what the body […]

  3. […] Everything you need to know about vitamin supplements October 26, 2011 By Sheri Leave a Comment var addthis_product = 'wpp-256'; var addthis_config = {"data_track_clickback":true};Now and again, a study is published that leads to a sea change in public opinion. If you’re a regular EBL reader, you already know how we feel about this. (While a single study provides some evidence about an issue, we should really consider all of the ava…) […]

  4. […] what’s a parent to do? That’s where information from a recent systematic review breaks new […]

  5. […] But one thing everyone who reads Evidence-Based Living knows: What we look for are scientific, systematic reviews of the literature. Individual studies may show one thing, but what happens when you put together all […]

  6. […] who actually was King George VI’s speech therapist? Of course, at EBL we know where to turn: To systematic reviews of the literature. Fortunately, a great review of stuttering treatment research exists – in fact, I suggest you […]

  7. […] heard us tout the benefits of systematic reviews over and over again here at Evidence-based Living.  The truth is they are the best way to evaluate […]

  8. […] weight reduction than diet changes alone.  (For a refresher on the value of systematic reviews, click here.) The review also found that exercise is associated with improved cardiovascular disease risk […]

  9. […] At Evidence-Based Living, we always maintain a healthy skepticism. So I decided to look at the effects on the age group that is driving the social networking phenomenon: adolescents. What about the supposedly dire consequences for their social relationships? As regular readers know, when we want a more definitive answer about scientific findings, we turn to reviews that incorporate multiple studies. […]

Speak Your Mind

*

Skip to toolbar