FSG’s Next Generation Evaluation Learning Brief was aimed at starting a conversation about how evaluation should evolve and grow to respond to social sector innovation in an increasingly complex, interconnected, and rapidly changing world. We posed five questions on the future of Next Generation Evaluation, and here Cyril Pervilhac of Public Health International Consulting (PHIC) shares his thoughts.
In what ways have you seen the trends we described (new philanthropic innovation, different rules of interaction, and proliferation of digital infrastructure) influence your work?
These trends have influenced my work over the last 10 years in global health, and in particular, I’ve observed them when evaluating large HIV/AIDS national programs or projects. We have moved gradually to Developmental Evaluation and Shared Measurement. Indeed, the environment in which we work is changing rapidly and we are continuously learning and adapting our strategies to address changing epidemics and to make evaluation most helpful for understanding whether new interventions make a difference and have an impact.
What do you think the implications are for how we think about evaluation? What explicit or implicit assumptions about evaluation are being challenged?
The way we think about evaluation is changing. We have moved a long way toward harmonizing the data we collect with agreed upon indicators in the monitoring system, but we still have a long way to go to have one evaluation system. Each stakeholder is still entrenched behind his own right to evaluate what his institution pays for. This creates a cacophony of similar questions and burdens the countries, instead of allowing us to investigate and best use the “Big Data” and instead of having partners investing in the analysis and use of the data we collect for policy purposes.
The evaluation of the largest U.S. Government-funded international HIV program (OGAC/PEPFAR report) is an example of opportunities to use Shared Measurement Platforms. We still need to explore how to join our forces between agencies (e.g. the Global Fund to Fight HIV/AIDS, TB and Malaria grants, or bi-lateral and multi-lateral agencies) and overcome political barriers, in order to carry out evaluations related to the performance of global programs – for example, jointly measuring national outcomes and impact.
To what extent are the characteristics of Next Generation Evaluation evident in your work?
The characteristics of Next Generation Evaluation appear in several ways: we are moving to quick learning processes and lighter annual reviews with immediate feedback and consolidation within a few weeks. Shared Measurement Platforms (such as Figure 3 in M. Kramer et. al., 2009) are being constructed with the creation of single platforms of data on reproductive health in the countries where we work.
Which of the game-changing approaches do you find intriguing? Which ones do we need more clarity around as a field?
Of the “game-changing approaches,” Big Data is most intriguing because it is nascent and under-used. It represents huge potential to collect essential data from communities on up to the national or global level with more powerful means to analyze, disaggregate data, compare, look at trends over time, visualize, report, and influence policies.
Do you see other game-changing approaches to evaluation emerging?
Other “game-changing approaches” may be the mix between Shared Measurement and Big Data, with the two overlapping because Big Data is becoming a key cross-cutting approach of all evaluation types. In addition, qualitative analysis may become increasingly important, perhaps indispensable, for improving our understanding of what the data mean. As we move into the future, we may have large organizations participating in joint national evaluations and using the same Shared Measurement Platform to better assess the outcomes and impact of programs, while at the same time complementing that information to still be accountable and answer specific questions that will inform their own constituents and justify tax payers’ investments (“value for money”).
