You are here: Home / Archives for evidence-based programs

Behave! Using the science of behavior change

June 3, 2010 By Leave a Comment

There are some problems we can’t do much about — hurricaines and earthquakes, for example. But a vast amount of things that make life tough — and sometimes miserable — relate to the choices human beings make and the way we behave. For this reason, a whole science of behavior change has grown up, focusing both on theoretical models and empirical studies of how to change damaging human behaviors, ranging from smoking, to crime, to overeating, to taking excessive risks.

A very helpful new article reviews models to promote positive behavior change that are highly relevant to people designing or implementing interventions. The authors note that getting individuals to make lasting changes in problem behaviors is no easy matter. They synthesize various models of behavior change “to provide a more comprehensive understanding of how educators can promote behavior change among their clientele.”

The authors apply their framework to the issue of financial management. Very interesting reading, available here.

(While you’re at it, take a look at other issues of this free on-line journal, called the The Forum for Family and Consumer Issues, published by North Carolina State University Extension — many interesting articles related to program development and evaluation.)

Filed Under: Evidence-Based Living, Health and Wellness, The Learning Center Tagged With: , , , , ,

Drugs, Medicare, and the older consumer: Economics to the rescue

May 10, 2010 By Leave a Comment

Okay, let’s have a show of hands. First, how many of you have a relative or someone you care about who is age 65 or older? Thanks.

Now, how many of you tried to help one of these beloved relatives or friends understand and choose a plan under Medicare Part D, the prescription drug benefit for older Americans? Thanks again.

My final question: How many of you who tried to help someone understand their options under Medicare Part D sighed, wept, and eventually wanted to pound your head against the wall in an attempt to lose consciousness? I thought so.

I had this experience myself, trying to help my 80-year old mother-in-law decide which program was best for her. I’m a gerontologist, for heaven’s sake, and I tore out what little hair I have left trying to figure out what her best option was.

 To the rescue comes a highly innovative and effective translational research project, led by Cornell Professor Kosali Simon (Department of Policy Analysis and Management). An economist, Prof. Simon’s desire to apply her expertise to this real-world problem has helped people in New York and across the country make this complex and important decision.

 Medicare Part D was passed in 2003 and is the federal program that subsidizes the costs of prescription drugs for people on Medicare (the federal health insurance program for Americans 65 and over). Some people were basically going broke paying for prescription drugs, and the federal government stepped in.

It sounds good, but here’s the problem: It is extraordinarly difficult to understand the coverage. A beneficiary has to choose among dozens of plans, which include dizzying combinations of deductibles and co-payments, and use different terminology for what they cover.

 

That’s the problem Prof. Simon took on. She had spent her career studying things like the economics of state regulation of private health insurance markets for small employers. But then she did an exercise for one of her classes, and students looked at Medicare Part D. Their work led her to become interested in the topic, and she began to do research on it.

Then she got in touch with psychologist Joe Mikels (Cornell Department of Human Development), who looks at how older people make decisions. Together, they used psychological theory and experimental methods to study older persons’ perceived difficulties of choosing a plan when the number of options available under Medicare Part D is increased in a lab setting. She also studied how seniors may actually benefit from increased breadth of choice in plan offerings using econometric methods and data on plan enrollment.

But here’s where it gets really interesting. Prof. Simon saw that there was practical value in learning how to help older people to understand the differences in medication coverage between plans. She used her data to create guides that can form the basis for choosing the right plans based on examining the coverage of medications, rather than simply going by general marketing materials that were mailed to older people.

Working with Project Manager Robert Harris, an experienced pharmacist, she has expanded the reach of the program in many different ways. Based on the research evidence, they have created a variety of materials such as pocket guides to Medicare Part D, posters, counter cards for pharmacies, customized mailings to residents of nursing homes, and an email newsletter and website with thousands of hits per month. 

All of this is very nicely summarized on her project web site CURxED, which I recommend you visit not just for the information, but as a great example of how complex information can be disseminated on the web.

Prof. Simon summed up the translational research approach very well when she told me: “It is very rewarding to be able to use the same data I collect for my research in ways that are practically useful to actual human beings being served by the program I study.”

Filed Under: Cornell Cooperative Extension, Evidence-Based Living, Health and Wellness, policy Tagged With: , , , , ,

Evidence-based programming: What does it actually mean?

May 4, 2010 By 2 Comments

Anyone who loves detective novels (like I do) winds up being fascinated by evidence. I remember discovering the Sherlock Holmes stories as a teenager, reading how the great detective systematically used evidence to solve perplexing crimes. Holmes followed the scientific method, gathering together a large amount of evidence, deducing several possible explanations, and then finding the one that best fits the facts of the case. As everyone knows, often the “evidence-based” solution was very different from what common sense told theo other people involved in the case.

In our efforts to solve human problems, we also search for evidence, but the solutions rarely turn up in such neat packages. Whether it’s a solution to teen pregnancy, drug abuse, family violence, poor school performance, wasteful use of energy, or a host of other problems – we wish we had a Sherlock Holmes around to definitively tell us which solution really works.

Over the past decade, efforts have grown to systematically take the evidence into consideration when developing programs to help people overcome life’s challenges. But what does “evidence-based” really mean?

Take a look at these three options: Which one fits the criteria for an evidence-based program?

1. A person carefully reviews the literature on a social problem. Based on high-quality research, she designs a program that follows the recommendations and ideas of researchers.

2. A person creates a program to address a problem. He conducts an evaluation of the program in which participants rate their experiences in the program and their satisfaction with it, both of which are highly positive.

3. An agency creates a program to help its clients. Agency staff run the program and collect pretest and post-test data on participants and a small control group. The group who did the program had better outcomes than the control group.

If you answered “None of the above,” you are correct. Number 3 is closest, but still doesn’t quite make it. Although many people don’t realize it, the term “evidence-based program” has a very clear and specific meaning.

To be called “evidence-based,” the following things must happen:

1. The program is evaluated using an experimental design. In such a design, people are assigned randomly into the treatment group (these folks get the program) or a control group (these folks don’t). When the program is done, both groups are compared. This design helps us be more certain that the results came from the program, and not some other factor (e.g., certain types of people decided to do the program, thus biasing the results). Sometimes this true experimental design isn’t possible, and a “quasi-experimental” design is used (more on that in a later post). Importantly, the program results should be replicated in more than one study.

2. The evaluation studies are submitted to peer review by other scientists, and often are published in peer-reviewed journals. After multiple evaluations, the program is often submitted to a federal agency or another scientific organization that endorses the program as evidence-based.

3. The program is presented in a manual so that it can be implemented locally, as close as possible to the way the program was designed. This kind of “treatment fidelity” is very important to achieve the demonstrated results of the program.

As you might already be thinking, a lot of issues come up when you consider implementing an evidence-based program. On the one hand, they have one enormous advantage: The odds are that they will work. That is, you can be reasonably confident that if implemented correctly, the program will achieve the results it says it will. A big problem, on the other hand, is that a program must meet local needs, and an evidence-based program may not be available on the topic you are interested in.

We’ll come back to these issues in later posts. In the meantime, I recommend this good summary prepared by extension staff at the University of Wisconsin. In addition, I’d suggest viewing this presentation by Jutta Dutterweich from Cornell’s Family Life Development Center, on “Planning for Evidence-Based Programs. And check out our web links for some sites that register and describe evidence-based programs.

Filed Under: Evidence-Based Living, The Learning Center Tagged With: , , , ,

Evidence-based systematic reviews: As close to certainty as it gets

April 20, 2010 By 9 Comments

Sometimes when I give talks, I like to use this catchphrase: For almost all of human history, our major problem was a lack of information, but over the past half century the problem has become an overabundance of information. Not only can you access multiple opinions on any topic, but the scientific evidence can seem to be all over the place. For any social or health problem humans experience, there are typically hundreds of studies. Making things worse, the studies can seem to contradict one another.

So people have come up with a solution. Rather than simply summarizing research findings in narrative form (remember what you used to do for a high school or college term paper?), researchers conduct systematic evidence-based reviews, in which they use sophisticated methods to bring together and evaluate the dozens, hundreds, or even thousands of articles on a topic. (There’s a good summary of the methods for systematic reviews on the Bandolier site.

When thinking about evidence-based reviews, you have to decide whether you agree with one basic proposition. This proposition holds that the findings of sound scientific research studies provide more credible evidence about solving human problems than personal opinion, anecdotes, or “gut feelings.”  Not everyone believes this (or at least not all the time). For those who do agree (as we do at Evidence-Based Living), then what is required is a systematic review of the research evidence, leading to guidelines for program development and their use in actual programs with audiences we serve.

The authors of a systematic review will tell you exactly what methods they used to identify articles for the review, how the articles were critically assessed, and how the results were synthesized across studies. Then the systematic review itself is peer-reviewed at a scientific journal, providing even more scrutiny of its findings. In some cases, the authors will use highly technical mathematical methods to synthesize the findings of studies, producing what is called a meta-analysis.

A systematic review has many benefits over the kind of review that simply summarizes a bunch of studies. You’ve seen this kind of review, which usually runs something like: “Smith and Wesson found this finding, but see Abbot and Costello for a different finding” or “Although most research shows this finding, there are a number of studies that fail to support it.” The systematic review looks at why the studies differ, and can exclude those studies that have inadequate samples or methods. And by looking at many studies, it allows us to make general conclusions, even though participants and study settings might be different.

Let’s take one example of how a systematic review is different from other reviews. If you read statements by groups advocating one perspective, they usually cite just the research articles that support their position. The hallmark of a systematic review, on the other hand, is a search for all articles on a topic. They go to great lengths to find every study done, so all can be evaluated. For this reason, systematic reviews are usually done by teams, since it’s rare that an individual has the time to find all the available research. By looking at all studies, a systematic review can come to conclusions like: “All studies using method X found this result, but studies using method Y did not.”

Systematic reviews can be disappointing, because they often come up with the conclusion that the research isn’t sufficient to come to a conclusion. But that in itself can be useful, especially if there’s one published study that has gotten a lot of attention in the media, but isn’t supported by other research.

The best library of systematic reviews has been described in a previous post: The Cochrane Collaboration.” But there are plenty of systematic reviews published each year from other sources. An example is in our post on antidepressants. If you want the most definitive evidence available as to whether a program or practice works, look to systematic reviews.

Filed Under: Evidence-Based Living, The Learning Center Tagged With: , , ,

Food Revolution or Evidence-Based Solutions?

April 2, 2010 By 5 Comments

I tuned in to Jamie Oliver’s Food Revolution the other night. I’m not a lover of reality shows, but, in this case, my curiosity got the best of me. For those of you who haven’t watched TV in the past few months, Food Revolution is a show that documents the antics of celebrity chef, Jamie Oliver, as he rides into the “fattest city in the US” and turns the population (especially the school kids) into healthy eaters. All this in slick, sensationalistic, sixty-minute segments!

As we all know, childhood obesity is taking a terrible toll on our kids. There’s no doubt that a crisis of this magnitude requires us to enact policy changes and programs aimed at addressing the problem. But do programs like Oliver’s Food Revolution really work? How do educators, concerned citizens, and policy makers know which programs will give us the best return on our investment?

John Cawley, a professor in the College of Human Ecology’s Policy Analysis and Management department, has recently published a study that addresses this question. Cawley, an economist, examined recent studies of several programs to reduce obesity, and found that CATCH (Coordinated Approach to Child Health), a multistate program that teaches elementary schoolchildren how to eat well and exercise regularly, is the most cost-effective. On the other hand, the study found that many other popular programs are not as effective and were much more costly than CATCH. Cawley’s study can be found here.

Cawley, who has served on the Institute of Medicine’s committee to prevent childhood obesity, says “It’s a bit of a Wild West, anything-goes environment when it comes to creating anti-obesity programs and policies. With limited resources, it would be counterproductive to rush into programs that are not cost-effective and won’t provide the greatest return on investment.

So, what does any of this have to do with Oliver’s Food Revolution? It suggests that policy makers need to look beyond the glitz when they consider which programs to invest in. It’s important to investigate which programs are “evidence-based” and which are merely entertainment. Food Revolution has not been rigorously evaluated. A preliminary study conducted by the West Virginia University Health Research Center to investigate the program suggests that the program had few positive impacts and a negative impact on meal participation and milk consumption.

In the end, as with most persistent societal challenges, the obesity epidemic is a complex problem best addressed by concerned citizens and policy makers who are committed to finding the best evidence-based solutions. And, unfortunately, it’ll probably take us longer than the sixty-minute segments of a reality TV show to fix the problem.

Filed Under: Evidence-Based Living, Health and Wellness, In The Media, policy Tagged With: ,
Newer Posts »
Skip to toolbar