In philanthropy, evaluation is often used to test a specific theory of change: to see whether a grant-funded intervention has had the desired effect. But there is another, underlying theory of change that determines the role of evaluation itself. That broader perspective begins with the question “How can a foundation’s modest resources influence any large scale social problem?”
For decades, the prevailing answer has been that foundations should conduct the social equivalent of research and development. Pilot innovative new programs, test them to see if they work and, once a solution has been discovered, let governments or others with larger resources fund the solution at scale. This is a tempting approach for foundations – it encourages them to continually test exciting new ideas, each requiring only a small amount of funding, and it holds out the promise taking credit for an important breakthrough. Under this paradigm, the familiar role of evaluation is to isolate the impact of a specific program, test its effect on a problem, and render a judgment as to whether it works or not.
The only problem is that this paradigm rarely – if ever – works. Social problems are complex. They are created and perpetuated by a wide range of factors beyond the control of any single organization. Even if a promising new approach is discovered, countervailing political factions may block its adoption, more funding may not be available, or the necessary management talent may be scarce.
In recent years, therefore, foundations have experimented with new paradigms of social impact: Funders that practice venture philanthropy take direct responsibility for growing effective organizations to scale. Others choose a catalytic approach, impacting an issue by leveraging non-monetary tools to shift government or corporate policies, or by motivating changes in individual behavior. More recently still, foundations have begun to pursue collective impact, working to align and coordinate the efforts of many different private, civil, and governmental organizations toward a unified plan of action. Each of these new approaches to social impact defines a different role for evaluation. The venture philanthropist needs measures of organizational effectiveness and sustainability. The catalytic philanthropist needs ways to understand the overall problem, rather than any individual grant. And the collective impact instigator needs a shared measurement system that enables numerous organizations to track and compare their progress. As a result, the field of evaluation today is in continual ferment as many new methodologies are being tried. What all of these new approaches have in common is a focus on learning so that the funders themselves can become more effective over time.
Yet old paradigms die hard. Many funders still adhere to an evaluation approach that aims to test pilot programs to prove whether or not they work, discounting other approaches as lacking scientific validity. What matters, of course, is not whether any one approach to evaluation is right or wrong, but whether it fits the underlying theory of change.
The time when a single approach to evaluation served all funders is gone forever. It has been replaced by an outpouring of innovation in evaluation that is showing foundations new ways of achieving impact, such as the use of developmental evaluation to evaluate new, innovative approaches to social change. And that is a very good thing.
