In June 2019, FSG began work with the Headwaters Foundation—a health conversion foundation serving Western Montana—to build their strategic learning and evaluation system. As a new philanthropic entity, the foundation wanted to do things differently from traditional philanthropy when it came to building their evaluation and learning practice. Headwaters has been an early adopter of trust-based philanthropy, an approach championed by the Whitman Institute and a growing number of philanthropic organizations that, at its core, requires funders to approach grantmaking with empathy and trust in community expertise and community-led solutions.
As we moved into 2020, the COVID-19 pandemic hit, and hundreds of foundations pledged to provide more flexible funding to grantees, reduce asks of grantee partners (e.g., grantee reporting), and commit to listening to partners. In addition, the massive support for the Black Lives Matter movement and the appallingly disparate impact of the pandemic on Black and Brown communities have amplified calls for the philanthropic sector to examine and change many of its policies and practices that sustain structural racism.
If you’re part of one of those foundations who signed the pledge, or have committed to doing things differently, you might now be thinking: How might our evaluation and learning practices need to adapt to be in line with our new commitments to our grantees? And what does this mean for how we’ve traditionally thought about how we hold grantees accountable?
Our work with Headwaters offers insight into what this might look like since they’ve been committed to the values of trust-based philanthropy since beginning grantmaking in 2018.
First, we had to reconcile how evaluation and learning were going to support Headwaters’ work while fitting with the core tenets of trust-based philanthropy. To be frank, when we thought about the way in which evaluation has traditionally been practiced by foundations (and, to be fair, evaluation consultants), it felt like trying to fit a round peg (evaluation practices) into a square hole (trust-based philanthropy). The Headwaters team was clear—we needed to think about the peg differently. We needed a new evaluation paradigm. Together, we questioned:
- Who is this for? From the beginning, we knew that the foundation’s evaluation and learning practices were for the grantees, NOT for the foundation. Headwaters CEO Brenda Solorzano explained this vision for evaluation and learning: “We are learning to help grantees and communities understand what is working and what is not so that community change efforts are effective. Not evaluating and learning to pat ourselves on our backs or to navel-gaze internally.”
- Who defines success? In standard practice, we often defer to “research-based” conceptions of success, or to “experts” who could help us define markers of success. But this approach didn’t make sense for Headwaters because of their belief that the people living and working in communities have the answers to the deepest issues facing Western Montana’s children and families. If you follow that belief, then to know what success looks like, you have to ask the communities to define success.
- How should grantees experience evaluation and learning? In a recent webinar on evaluating collective impact, participants shared images that came to mind when they heard the word “evaluation:” a checklist, a test, bar charts, spreadsheets, anxiety, fear, more spreadsheets, eye-rolling, forms, and more forms. What these participants raised was that—even with good intentions—traditional approaches to evaluation have not been used in ways that actually support grantees. When we asked Headwaters staff in our first meeting what they wanted their evaluation and learning practices to look and feel like for staff and for grantees, they shared the words: feedback, co-creation, agility, shared vision, questions, storytelling, data, insight. Those words—not spreadsheets and bar charts—provided the scaffolding for Headwaters’ evaluation and learning practices.
- What do we mean by accountability? The use of the word “accountability” has risen exponentially in the United States since the 1960s. Evaluation buffs might note that this coincides with a ramp-up of evaluation, spurred by evaluation of the federal government’s war on poverty efforts in the 60s. [Note: correlation does not imply causation, but it is interesting, isn’t it?] And though the use of the word has increased, clarity around what it means has decreased. For many grantees, the way in which “accountability” has manifested itself in their relationships with funders has been painful and fraught with challenging power dynamics (remember those spreadsheets and anxiety?). Headwaters wanted something different for their grantees. They approached “accountability” with a different paradigm. As Brenda explained: “We built a shared definition of success with our grantees through the theory of change, so accountability is about working towards a shared vision. For our board, “accountability” is an understanding that staff will be doing our homework to know when grants/projects/initiatives are working and not working (relative to that shared definition of success) and using this information to make different investments or stop investments.”
Once we were clear on our “new peg” approach to evaluation and learning, we structured activities that helped ensure that, as we developed the recommendations for Headwaters’ evaluation and learning practices, these paradigm shifts took center stage:
- We co-created with grantees. We structured the project in two phases. In the first, we spent a lot of time talking with foundation staff about what they thought their funding was helping to change in the community, and what kinds of information they thought would be helpful to 1) tell the story of change, and 2) position grantees and other partners to do their work better. Though this might not feel like co-creation from the start, it was important for staff to be on the same page about their vision for change and the purpose of their evaluation and learning system. From there, we used the community engagement spectrum shared by Paul Schmitz (adapted from iap2), to confirm with staff how they wanted to engage with grantees and partners. Instead of “informing” grantees of the drafts we developed, we took at collaborative approach by asking grantees open-ended questions about how they thought about outcomes and heard from them about how they wanted to experience evaluation and learning with the foundation. We then reconciled the multiple perspectives into a final theory of change.
- We got rid of traditional grantee reports. One of the principles of trust-based philanthropy is “doing the homework,” and a traditional approach to reporting—where grantees pick a few outcomes and then report on them in interim and final reports—didn’t fit. Instead, we developed templates for Headwaters staff to capture data during conversations with grantees, on-site visits, or by reviewing materials from grantees (if the grantee chooses to share a report or other materials that they have on hand). On the backend, staff will use the information they capture in the templates to create spreadsheets and charts linked to the outcomes on the theory of change, but what the grantees will experience will be conversation, feedback, and learning (remember our scaffolding words?).
- We highlighted trust as a bedrock of the foundation’s work. To truly live into a trust-based approach, we had to find ways to hear from both foundation staff and grantees about the extent to which trust is present. Knowing that trust is, at its core, about human connection, we needed to get a variety of perspectives on what that looked like and what were signals to foundation staff and grantee partners that trust existed. Once that was defined, we not only made it a “bedrock” outcome upon which all other outcomes sit on the theory of change, we also integrated questions about trust into the overall learning framework and tools used to capture data.
In conclusion, we will leave you with advice from the Headwaters team about the importance of approaching evaluation and learning in a trust-based way:
“Truly center community and don’t let it be just lip service to community centeredness. Share power in defining success. Shift paradigms about this work being for shared learning instead of accountability. And work on trust as a cultural norm internally and externally.”
Learn more about FSG’s Strategic Learning and Evaluation practice >
