I’ve been thinking a lot recently, about how evaluations of social innovations can more effectively communicate the effects and impact of their efforts. In particular, I’ve been asking myself (and anyone who will wonder with me), How can evaluation findings, insights, and recommendations be translated into actionable knowledge that inform and influence decision making and action?
I have to admit that this question is not all that new to me – as a professional evaluator, I have always searched for ways to make evaluation findings useful and used by a wide range of stakeholders. And, while I have even co-written a book on how to communicate and report evaluation findings, I still struggle with finding ways to motivate and inspire stakeholders to not only consider the findings, but to do something with them. And, while we talk a lot in the field about “how to measure social impact,” and debate the relevant questions, evaluation designs and data collection methods, we pay scant attention to how we might translate what we learn from evaluations into action.
This issue has led me to think about the ways in which people are motivated to act, and how and when they use data to inform their actions. I believe that people tend to do something with information, when: a) they have a personal experience (e.g., fear, frustration, opportunity, a change in circumstances), b) they hear or read a compelling story (often with visual images), and/or c) they engage with others – through relationships. While I think most of us can cite an example of any of these, I often think about the following story when considering how I might translate social impact evaluation findings into action:
A New York Times Magazine article described the following in an article, “Selling Soap: How Do You Get Doctors to Wash Their Hands?” The piece begins by explaining that a 68-year-old urologist was recently on a cruise with his wife when he noticed that passengers who went ashore were not allowed to reboard the ship until they had some Purell (antibacterial gel) squirted on their hands. As he watched this happen repeatedly, he began to wonder if the cruise ship was being more diligent about killing germs than his own hospital.
Shockingly, it is estimated that between 44,000 to 98,000 Americans die each year because of hospital errors—and that one of the leading errors is the spread of bacterial infections, many of which are spread by germs on one’s hands. Numerous medical studies have shown that hospital personnel wash or disinfect their hands much less often than they should . . . and doctors are even more lax than nurses or aides.
When the doctor arrived home, he called a meeting of the hospital’s leadership, where together, they identified a number of reasons for poor hand hygiene (informal evaluation!). Their reasons included: (a) doctors are too busy to wash, (b) a sink is not always handy, (c) Purell dispensers are not convenient, (d) self-deception—doctors think they are washing more often than they are, and (e) arrogance—“not me” attitude. After this meeting, the hospital devised a number of “social interventions” to incent doctors to wash their hands more frequently. They sent e-mails and faxes, and developed posters that were placed around the hospital. They concluded, however, that none of these worked (another informal evaluation). They then started a campaign in the hospital parking lot where nurses handed out Purell (another social intervention). They also roamed the wards and handed out $10 Starbucks cards to doctors who were washing their hands. Although compliance did rise to 80% from about 65%, they were still not meeting the Joint Commission on Accreditation of Healthcare Organization’s requirement of 90% compliance (and a visit from this body was imminent). (Note that the informal evaluation results were being used, but behavior wasn’t changing to the degree necessary.)
When these results were presented at a luncheon attended by the chief of staff advisory committee, the doctors were quite discouraged. However, much to their surprise, after they finished their lunch, each person was handed a sterile Petri dish loaded with a spongy layer of agar and were told they were going to give a culture of their hand. Obediently, they pressed their palms into the plates, which were then sent to be cultured and photographed. According to the doctor who initiated this activity, the resulting images “were disgusting and striking, with gobs of colonies of bacteria”. As a result, the administration decided to photograph the Petri dish and make it into a screen saver, which is now on every doctor’s computer. As one person explained, “When you present them with good data, they change their behavior very rapidly. Some forms of data, of course are more compelling than others, and in this case, an image was worth 1,000 statistical tables”. Hand-hygiene compliance shot up to nearly 100% and has stayed there since this occurred.
The story provides with one answer to my earlier question – How can evaluation findings, insights, and recommendations be translated into actionable knowledge that inform and influence decision making and action?, in that it wasn’t the data that changed the doctor’s behavior; it was how the findings were communicated that motivated the doctors to act differently.
As we continue to develop and refine our approaches to evaluating social impact, let’s not forget to pay attention to the ways in which we communicate and report our findings and learnings. If you have found a way to inspire action based on evaluation findings, please feel free to share!
