Is science reliable?

Here at EBL, we’re written before about the pitfalls of science reporting in popular media.   Even well-researched, comprehensive scientific reports often draw conclusions that we later learn are inaccurate.

I’ve been thinking about this topic recently and wondering, is there a good way to assess scientific evidence that will stand the test of time?  That’s when I came across an article published in the New York Times by Gary Gutting, a philosophy professor at Notre Dame University. His opinion piece explains the steps in the scientific process, and why early studies often don’t offer evidence we can use.

Often, scientists begin testing a theory with a correlational study, which can show a connection between two things. But a positive result doesn’t mean that one thing causes another.  For example, a famous study of middle school students showed that youth with larger foot sizes had better reading comprehension.  But having larger feet does not cause someone to be a better reader. The correlation most likely had more to do with the age of the students.

Yet researchers often begin with a correlational study because they are shorter and less expensive to conduct in contrast to more robust randomized-controlled trials.

“Contrary to what many non-scientists seem to believe, the key feature of empirical testing is not that it’s infallible but that it’s self-correcting,” Gutting explains. “As the physicist John Wheeler said, ‘Our whole problem is to make mistakes as fast possible.'”

The trouble comes when media sources report the results of correlational studies as proven facts, instead of the first step in a years-long research process.

Another problem occurs when science reporters take research findings out of context, Gutting says. Instead, we must take into account the conditions under which research is conducted.   This is especially important in social science research, where it is more difficult to control for all of the variables in a given situation. Gutting provides the following example: “A feather and a lead ball dropped from the same height will reach the ground at the same time — but only if there is no air resistance.  Typically, scientific laws allow us to predict a specific behavior only under certain conditions.  If those conditions don’t hold, the law doesn’t tell us what will happen.”

In his article, Gutting proposes a rating system for research studies to explain how easily we can translate the findings to our daily lives. Here at EBL, we use systematic reviews and meta-analyses to provide the most reliable research findings to use in everyday life. But even these sweeping analyses can become outdated or miss an important aspect of specific topic.

The take home message?  Scientific research provides important information to help guide decisions about our lives. But it’s important to think critically about the research you read, and be open to new developments as scientists learn even more about our world and how it works.