In October 2012, we wrote a blog post titled, “Big Data and the Social Sector: Let the Revolution Begin!” In it, we posited that the use of big data revolution in the social sector is closer than we think; however, social sector stakeholders have to effectively address concerns around privacy, accuracy, and reliability. In the ensuing months, the chatter around use of big data in the social sector has exploded to a full-throated roar. Markets for Good has run a series of blog posts on the topic, as has the Skoll World Forum and the UN Global Pulse. The tone has ranged from wide-eyed wonderment to skepticism to downright hostility. Everyone, it appears, has an opinion.
So, what does the big data revolution mean for the field of evaluation? A recent discussion on the American Evaluation Association’s LinkedIn page asks if evaluation may soon be facing an existential crisis. In other words, will the availability of large amounts of data and the analytic engines that crunch the data make traditional evaluation designs and methods obsolete? Who needs sampling techniques when you can pretty much capture characteristics of whole populations? Why convene a focus group when you can just analyze twitter feeds? Or, why hire an evaluator at all, when a well-structured algorithm can draw the same conclusions? After all, we live in an age when machines are grading essay papers.
We at FSG have set out to explore the implications of big data in evaluation as part of a project that we are calling “Next Generation Evaluation”. Rather than the doomsday scenario described above, we are taking a “both/and” approach to the intersection of big data and evaluation. We believe that big data is a powerful force that can be harnessed for good, and that utilizing a mix of big and traditional (small) data in complementary ways would be better than using either of them in isolation. In our work, we’ve encountered several exciting examples of how social sector stakeholders are leveraging big data to better understand social problems, and “what if” scenarios of how big data can be better leveraged in the social sector. However, we’ve encountered relatively few examples of how social sector stakeholders are using big data for the specific purpose of evaluating the effectiveness of programs. Here’s where we need your help. If you have an example of how social sector stakeholders are using big data to evaluate their programs, please respond using the comment box below, or email us directly here. We would love to hear from you.
