Over the past few years, we’ve supported a growing number of foundations in adopting a systems change approach to creating social change. These organizations are now thinking about how their grantmaking and other value-added activities can, to use one of our favorite definitions from Social Innovation Generation, “shift the conditions that are holding a problem in place.” This evolution often requires foundations to experiment with tools and practices that are new to their organization.
We believe that supporting and guiding this experimentation with ongoing, intentional learning is essential to achieving results. While evaluation is often conducted as a means to learn about the progress or impact of an initiative, evaluative thinking and continuous learning can be particularly important when working on complex issues in a constantly evolving system. And, when evaluation goes hand in hand with strategy, it helps organizations challenge their assumptions, gather information on the progress, effects, and influence of their work, and see new opportunities for adaptation and change. Here are 5 ways that evaluating your systems change efforts can improve your odds of achieving impact.
1. Gain deeper insights into the complex contexts that are holding the problem in place.
Systems are multifaceted and constantly evolving, and we’re often so entrenched that it’s hard to see the structures, relationships, and mental models influencing the systems we are in. Evaluation can provide the opportunity to dive deeply into understanding the system(s) which can illuminate ways in which change efforts can be more effectively supported.
A few years ago, we supported the creation of Operation Youth Success, a collaborative initiative working to transform the juvenile justice system and improve youth outcomes in Douglas County, Nebraska.
The initiative has taken a number of steps toward this goal, but early evaluation findings highlighted the success of two new practices:
- Training School Resource Officers on the teenage brain and the impacts of juvenile justice involvement
- Holding weekly meetings among school resource officers, school administrators, and probation officers to discuss youth on probation
Over the course of one year, these changes led to a 22 percent decrease in felony arrests and a 41 percent decrease in misdemeanor arrests by School Resource Officers. School Resource Officers who had taken the training and participated in the meetings reported they had become more aware both of issues affecting youth and of new strategies for dealing with youth.
This data illuminated the opportunity to create change by working on 2 important but challenging components of systems change—the relationships between students and School Resource Officers and the mental models of school and community leaders who interact with students.
2. Bring stakeholders together to test assumptions, insights, and to engage in collective sensemaking.
Systems change can’t be driven by one person or a single organization. Foundations adopting a systems change approach must create opportunities for the many actors involved in a system to learn from and make decisions with each other, as well as build new and stronger relationships—and we’ve observed that various kinds of evaluation and learning activities help open up that space, especially between foundations and grantees.
When we collaborated with The Chicago Community Trust and Kinship Foundation to develop a learning, research, and evaluation framework for its Food:Land:Opportunity—Localizing the Chicago Foodshed initiative, we convened grantees and foundation staff to discuss the food system in Chicago, using an actor map we had created with the foundations as a starting point for the conversation. As expected, the grantees had a lot to say about our assumptions and understandings of the systems in which they work. To engage grantees, we enlarged the map and put it on the wall, and invited them to redraw, comment on, and question the content of the actor map we had developed. This activity sparked a rich conversation between the foundation staff and grantees about the Chicago context, the grantees’ activities and relationships, and gaps within the Chicago food system.
3. See the bigger picture and contextualize your work within the system.
The process of developing a theory of change during an evaluation necessitates taking a system-wide look at your work.
An example of this is our developmental evaluation work with the Palix Foundation’s Alberta Family Wellness Initiative (AFWI), a program designed to catalyze system changes that will improve health and wellness outcomes for children and families. To help improve policy and practice, the foundation engages with researchers, policymakers, and health practitioners to disseminate, mobilize, and promote the use of knowledge about brain and child development and its connection to addiction and mental health outcomes.
To better understand the multiple contexts in which the work was happening, we used a blend of traditional data collection methods such as interviews and surveys, as well as other evaluation methods specifically designed to look for change in complex systems. These included systems mapping, reflective practice sessions, ripple effects mapping sessions, and interactive learning-oriented, stakeholder meetings. The evaluation’s results were shared “far and wide,” and in particular, helped build a deeper understanding of the role the foundation was playing in the larger system. The results were also used by the foundation to focus more on professional development, as opposed to large-scale convening, and on smaller, ad hoc, community-focused convenings.
4. Identify positive and negative unintended consequences and prepare for or respond to a variety of unexpected ripple effects.
If your organization is experimenting with new, systems-focused practices, evaluation can help you unearth positive and negative unintended consequences of these shifts. For example, when the Lumina Foundation evaluated their Lumina Latino Student Success effort, a $12 million investment in support of potential and current Latinx college students, they realized that promoting the effort as programs exclusively for Latinx populations in communities where the overall population was majority-minority, led to questions from other underserved communities of color that also required support.
To address this unintended consequence, sites made new efforts to establish relationships with other underserved groups. For example, to create a sense of partnership and shared liberation between Latinx and black communities, they hosted events where historically black colleges welcomed Latinx high school students. They also highlighted the reasons it was important to provide tailored interventions for Latinx students—ensuring all information was available in Spanish, for example. And they opened the Latino student success programs to other underserved students of color.
5. Have greater confidence in leveraging resources.
Evaluation can help funders understand the system better and identify leverage points—defined by Donella Meadows as “places within a complex system where a small shift in one thing can produce big changes in everything.”
As Meadows discusses in her seminal article, “Leverage Points: Places to Intervene in a System,” equal investments in different parts of a system won’t lead to equal results. Evaluation can help foundations see those leverage points and have greater confidence in how aligned the use of their financial resources, position and reputation, relationships, and power, are with their goals.
Early evaluation of the California Endowment’s 10-year $1 billion “Building Healthy Communities” Initiative, for example, revealed that youth in the community were very eager to get involved with the initiative. Moreover, the youth in the program were effectively driving policy change within their community and, even more critically, changing the mental models of leaders in the community from politicians to school principals.
With this information, the California Endowment invested in training to equip youth with leadership and public speaking skills, platforms for engagement, and stipends for youth to become more actively involved. These investments created a network of motivated, activated youth leaders in the movement to create healthy and just communities who can support systems change for years to come.
If you are interested in learning more about how learning and evaluation can support systems change efforts, check out the following resources:
- Evaluating Complexity, Hallie Preskill and Srik Gopal
- Systems Grantmaking Guide, Grantmakers for Effective Organizations and Management Assistance Group
- What We Know so Far About Evaluating Systems Change, Mark Cabaj
- Culturally Responsive Evaluation Meets Systems-Oriented Evaluation, Veronica G. Thomas, Beverly A. Parsons
- A Practical Guide to Evaluating Systems Change in a Human Services System Context, Nancy Latham
- Systems Thinking Toolkit: Putting Systems Thinking into Practice in Your Organization, FSG
Learn more about FSG’s Strategic Learning and Evaluation work >
