Recently thousands of evaluators came together in Chicago to celebrate the growing field, share exemplary practices, and to push our collective thinking on how evaluation responds to the complexity of social change efforts.
During the conference, my colleagues and I facilitated a discussion among about 40 evaluators about what complexity means for thinking about and using theories of change. Our conversation was designed to build on questions that emerged during our webinar on Evaluating Complexity last year.
One of the questions we tackled as a group was:
“What are the essential elements or “minimum specifications” of a theory of change for complex initiatives?”
Our conversation led to the following insights that highlight what the key elements might be:
- The process of asking questions that challenge mental models about the strategies being used to effect change is more valuable than the culminating visual or product. The purpose of developing a theory of change is not about creating a beautiful visual of the initiative, although there was agreement that a nice graphic goes a long way to creating buy-in and revisiting the theory of change over time. The real value is in the conversation that takes place in developing the theory of change. Ideally this conversation challenges existing mental models about how the world works.
- Assumptions and external factors are (the most) essential elements of a theory of change under conditions of complexity. One evaluator said he had developed a list of external factors for an advocacy initiative that was 2 pages long! Why? Because the success of the advocacy strategy was predicated on a confluence of factors that was outside of advocates’ direct control. Gathering data on assumptions and external factors became more vital to success than monitoring of progress toward specific, predetermined outcomes, since those outcomes were constantly shifting.
- Values are important to include in a theory of change, particularly when they are deeply embedded within the initiative strategy. A theory of change should capture the underlying beliefs (historical, cultural, philosophical, religious, and/or ethical) that affect how an organization seeks to have an impact in the world. Including values in the theory of change communicates to key stakeholders that these fundamental values exist and help to determine which strategies are being used and toward what end.
- Evaluators should be explicit about where the outcomes in the theory of change stop short of the ultimate impact. For example, it is not uncommon for theories of change around advocacy efforts to portray policy change as the ultimate goal. However, policy change is not the end goal. In the big picture of impact, policy change is a leverage point to improve some specific social or environmental condition. Implementation and enforcement of the policy is needed in order to bring about the intended impact.
There is an important note and caveat to this discussion. First, it is important to note that a theory of change reflects a “best hypothesis” for how change is expected to happen. As an evaluation unfolds, data may call into question whether hypothesis should change. Theories of change, especially for complex initiatives, should be refined over time.
Second, theories of change are not always the best starting point for an evaluation. When complex initiatives are in their planning and early implementation phases, there is rarely a clear sense of what’s happening or where to intervene. Rather than guessing how best to effect change at the outset, evaluators and practitioners can work together to collect the data that helps a complex initiative arrive at a theory of change as a product of the evaluation, rather than an input into it. System mapping has replaced the theory of change conversation as a starting point for some developmental evaluations, including those underway at the William and Flora Hewlett Foundation and Kresge Foundation.
Learn more about system mapping and how to use it in our recent system mapping blog series.
