In 2009, the Norlien Foundation established the Alberta Family Wellness Initiative (AFWI), to invest in improving the health and wellness of children and families in the province of Alberta, Canada. The initiative aims to share and promote knowledge about brain and biological development as it relates to early childhood development, mental health, and addiction. Given the cross-sector and multi-disciplinary nature of what AFWI aims to achieve, the initiative is designed to engage and catalyze relationships across stakeholders from science, policy, and practice domains.
Now in its sixth year, the initiative continues to navigate complex political and relational dynamics as it seeks new and better ways to share knowledge about brain science, change the behaviors and practices of direct service providers, and deepen the momentum for broad-based systems change in the province. When looking into an evaluation of the initiative, its leaders sought an approach compatible with its emergent rather than pre-determined approach to strategy.
However, this soon raised a question: How can we best evaluate such a complex strategy?
Traditionally, the evaluation field has focused on assessing the specific effects and impacts of programs according to a set of pre-determined outcomes and indicators, with an emphasis on connecting the initiative to the outcomes in a tangible way. Many evaluators have drawn from Newtonian notions of cause of effect, and considered “context” to be merely described, or possibly, controlled for. While this approach may still work for evaluating boundary-defined, stand-alone “programs” in fairly stable environments, it falls short when it comes to evaluating complex initiatives, as well as initiatives that operate in complex environments.
In work with foundations, nonprofits, and corporations to foster the use of evaluation and learning as a key ingredient in strategy and decision-making, we are increasingly noticing a realization for crafting evaluations to be cognizant of complexity, rather than “assuming it away.” However, there still remains a gap between conceptual understanding of the implications that a different set of assumptions about how social change happens has for evaluation, to more explicit principles, tools, and processes for how to make that shift effectively.
In other words, how does the practice of evaluation have to be different to accommodate complexity?
To answer this question, we developed a set of “propositions” that one could adopt into practice when faced with evaluating complex initiatives and/or initiatives in complex environments. These propositions are based on known characteristics of complex systems. We preview three of the propositions below from the full list of nine that we have identified in Evaluating Complexity: Propositions for Improving Practice.
|
Characteristics of Complex Systems |
Propositions for Evaluation |
|
A complex system is always changing and evolving; it is never static |
Design and implement evaluations to be adaptive, flexible, and iterative |
|
Relationships between entities are equally if not more important than the entities themselves |
Focus on the nature of relationships and interdependencies within the system |
|
Cause and effect is not a linear, predicable, one-directional process; it is much more iterative |
Explain the non-linear and multi-directional relationships between the initiative and its intended and unintended outcomes |
The three propositions outlined above were used, intentionally and purposefully, as part of the evaluation of AFWI. An adaptive and iterative approach to designing the evaluation ensured that data collected informed the foundation’s most pressing questions. The map of systems players and the theory of action captured the complex relationships and interdependencies, as well as the multi-directional path to progress embedded in the initiative. This drove data collection and analysis in a way that shined light on tangible progress in the proximal outcomes of the initiative’s knowledge translation and mobilization strategy, and provided valuable input into strategic decisions about how to continue supporting organizational and systems change.
To learn more, read Evaluating Complexity: Propositions for Improving Practice, which details additional propositions, along with case examples, methods, tools, and resources, that we hope will serve as a compass for those engaged in the complex and messy, yet important and rewarding work, of creating social change.
