One of the most difficult elements of creating and managing an effective collective impact initiative is shared measurement – the use of a common set of measures to monitor performance, track progress toward goals, and to learn what is working and not working. As discussed in my colleague Fay Hanleybrown’s September blog, while challenging, the actual process of defining shared measures can in itself lead to greater alignment and coordination of participating organizations.
However, once shared measures are defined, it is just as critical that there is a facilitated process for participants to gather periodically to share results, learn from each other, and refine their individual and collective work based on the learning. This learning process is what makes a shared measurement system an adaptive learning system. Without a collective learning process, the ultimate benefit of improving individual and collective practice can be lost.
Creating a structured approach to learning from information generated by the system is important to ensure consistent use of the data occurs. Strive, the collective impact initiative focused on improving education outcomes in Cincinnati, has adapted the Six Sigma quality improvement process to review and learn from data, and then improve practice based on the learnings.
Another example of using data to learn and improve practice is employed by the Magnolia Place Community Initiative. Magnolia Place unites county, city, and the community to build sustainable change for families by improving education, health, economic stability, and safe and nurturing parenting. This initiative has brought together approximately 70 organizations from a five square mile neighborhood of Los Angeles to unite around a shared set of goals and measures. To learn from the data collected and improve initiative effectiveness, Magnolia Place has adapted the “Plan, Do, Study, Act” (PDSA) process from the healthcare sector’s “Model for Improvement.” Using the process, initiative participants meet and review data frequently – typically once per month. These small groups review data dashboards together, plan what to do to address areas where data does not meet targets, and then select groups do and pilot a new approach that may improve outcomes. After this pilot occurs, the groups collect data again and study the improvements, and then act to refine and if relevant, spread this improvement more broadly across the initiative.
The PSDA process has led to system-level, cross-organization improvements – such as how participant agencies make referrals to each other. At the beginning of the initiative, agencies noticed that they were often serving the same clients – but data showed that clients often were not following through on referrals provided by one agency to another. In particular, the data showed that when agencies referred clients to government providers such as the city or county office, clients less frequently followed through with the referral. Upon digging into the data and discussing across organizations, participants learned that this was occurring because the referral was seen to be impersonal, and clients were skeptical of referrals to public agencies due to perceived bureaucracy.
To improve the rate of follow through for referrals, the agencies planned and piloted a new approach – making a “warm handoff” for referral. In a warm handoff, clients were referred to a person rather than just an agency name. When this pilot for warm handoffs was implemented, and the subsequent round of data collected, studying the data showed an increase in the referral follow through rate and this process was scaled across the initiative. The process of PDSA led to the agencies trying new approaches to improving referrals rates and created a deeper understanding across agencies on referral process – ultimately benefiting individuals in the community.
We are interested in hearing more about how other collective impact initiatives learn and improve practices using their measurement systems. Have you adapted a formal process such as Six Sigma or the Model for Improvement? Or have you designed your own structured learning and improvement approach? How do you learn across organizations in your initiative? Please let us know!
To learn more about how organizations are creating and learning from shared measurement systems, join FSG, Magnolia Place, and The Calgary Homeless Foundation for a Shared Measurement webinar on November 9th.
