What does it take for lots of different organizations measuring their own performance in different ways to move to using a common set of measures to track progress toward goals?
Since publishing the SSIR article “Collective Impact,” we at FSG have heard from many, many exciting examples of multiple organizations working together to create positive social change around the world. These examples span different geographies (from a single neighborhood to multiple countries) and a wide range of social issues (environment, health, education, public safety, workforce development, homelessness, etc), and many are employing the five conditions of collective impact outlined in the article: a common agenda, a shared measurement system, mutually reinforcing activities, continuous communication among stakeholders, and a backbone organization to support the effort. Of these five conditions, practitioners report that one of the most challenging to achieve is shared measurement—the use of a common set of measures to monitor performance, track progress toward goals, and to learn what is working and not working.
It’s not surprising that shared measurement is so hard. As outlined in FSG’s white paper “Breakthroughs in Shared Measurement and Social Impact,” the traditional paradigm of evaluation focuses on isolating the impact of a single organization, or of a single grant, rather than assessing the impact of multiple organizations working together to solve a common problem. It can be very challenging to come to agreement on common measures, with competing priorities among stakeholders or even concerns about comparative measurement across providers working in the same space. There is often limited capacity for measurement and data analysis within individual organizations, not to mention the significant time and cost of developing and maintaining a shared measurement system among multiple organizations.
But for many collaborations, the benefits of shared measurement outweigh the challenges. These benefits include greater alignment among the goals of different organizations, more collaborative problem solving, and the formation of an ongoing learning community that gradually increases the effectiveness of all participants. In some cases, simply the process of defining shared measures has led to significant changes in the way that organizations do their work.
When stakeholders in Calgary, Canada came together to define common programmatic measures around reducing homelessness, they found that providers had very different definitions of key homeless populations, such as “chronic” or “transitional” homeless, and that their services were not always aligned to the needs of the individuals served. The process of developing common measures led some organizations to fundamentally shift their service offerings and to coordinate with other organizations in order to better serve the homeless population.
One key to success is that developing common measures is itself a collective effort, with broad engagement by many organizations in the field and with clear expectations about confidentiality and transparency. The Road Map for Education Results, an education-focused collective impact effort in Seattle and South King County, established four work groups across the cradle-to-career continuum to develop common measures using clear criteria, and then refined these measures through iterative meetings with hundreds of stakeholders—including mayors, superintendents, CBO leaders, and parents—before finalizing a common goal and set of shared measures.
Other key success factors for shared measurement include:
- Funding: Shared measurement systems are expensive to develop and maintain. Having strong leadership and substantial funding throughout the development period is critical.
- Infrastructure: It is important to have ongoing staffing support to provide training, facilitation, and to review the accuracy of data. (This is sometimes the role of a “backbone” organization, but not always). Web-based technology enables multiple users to access and use a shared measurement system.
- Learning and Improvement: Once shared measures are defined, it is critical that there is a facilitated process for participants to gather periodically to share results, learn from each other, and refine their individual and collective work based on the learning. This learning and improvement process is often supported by the “backbone” organization. The shared measurement system itself must also be tested and continually improved through user feedback.
For those interested in learning more about this topic, I will be leading a session about shared measurement in the context of collective impact at the Stanford Nonprofit Management Institute on September 27th. Participants will learn about innovative examples of shared measurement and details on the steps to developing shared measures, as well as the benefits and challenges involved in implementing this in their own work. Also, stay tuned to hear from collective impact practitioners about how to develop and learn from shared measurement in an upcoming FSG webinar.
