Evidence-based Living is built around the idea that scientific study should guide our lives – in decisions we make for our families, in community initiatives, and of course in choosing medical treatments.
A new review this month is the Journal of Oncology raises important questions about the validity of medical studies. The report reviewed 164 trials of treatments for breast cancer including chemotherapy, radiation and surgery conducted from 1995 to 2011.
It concluded that: most of the studies were clouded by overemphasizing the benefits of the treatment, or minimizing potential side effects.
For example, they reported on 92 trials which had a negative primary endpoint – which essentially means the treatment was not found to be effective for the main goal of the study. In 59 percent of those trials, a secondary end point – another goal – was used to suggest the experimental therapy was actually beneficial.
And only 32 percent of the studies reported severe or life-threatening side effects in the abstract – where medical professionals who are scanning the report might miss them. Studies that reported a positive primary endpoint – meaning the treatment was effective for the problem that researchers were targeting – were less likely to report serious side effects.
What does all of this mean?
Elaine Wethington, a medical sociologist at the College of Human Ecology, says the review reveals some important findings about medical studies.
“I would speculate that the findings are due to at least three processes,” she explained.
“First, trial results should be published even if the primary outcome findings are negative, but it can be difficult to find a journal that will publish negative findings,” she said. “As a result, there is a tendency to focus on other outcomes that are secondary in order to justify the work and effort.
“Second, presentation of findings can be influenced by a variety of conflicts of interest. There is a lot of published evidence – and controversy — that scientific data collection and analysis can be affected by the source of funding, private versus public.
“Third, this could also be explained as a problem in scientific peer review. Reviewers and editors could insist that this type of bias in reporting be controlled,” Wethington said.
In short, she sees the publication of this review as an important step in improving the scientific review process.
Valuable info. Fortunate me I discovered your web site accidentally, and I am stunned why this accident didn’t came about earlier! I bookmarked it.
In most cases, the primary endpoint is all-cause mortality, isn’t it? So even in cases, where mortality is unchanged, there can still be a benefit in other aspects, such as morbidity. I think it is important to consider a variety of endpoints because there is more to medicine than just curing the disease. As medical students are taught these days, “treat the patient, not the disease.”
This is a very good comment. Secondary outcomes such as benefits to quality of life during treatment are very important to patients, and are a concern in patient-centered approaches to medical treatment.
I don’t believe the authors of the journal article mean to discourage the reporting of secondary outcomes. They were concerned about how reporting of secondary outcomes in article abstracts could be used to “spin” the interpretation of the findings away from a negative or non-significant primary outcome result or from findings that suggested the treatment was highly toxic. They used a decision-tree coding system to classify the articles by whether or not “spin” was used. They found that there was prevalent use of spin when primary outcome results were negative. They recommend that now that this pattern has been uncovered, editors should issue guidelines to authors and reviewers that minimize the tendency to “spin” the findings.
In addition, the authors tested whether funding source was associated with the tendency to spin. It was not, thus my second speculation for why they found this pattern is not supported.
I think that’s a very valid point regarding spin and issuing stricter guidelines on how results should be presented. I see this as a side-effect of how science is conducted these days, however. There is such a push to get a publishable result, because of its effects on funding, tenure, reputation, etc. Furthermore, with the risk of being scooped increasing, it would not surprise me if underpowered studies were also getting published because the evidence seemed “close enough”.