Over the years, people have approached us with wonderings about what really counts as evaluation. For example, people have asked, “Is evaluation the same as research?” “What’s the difference between performance measurement and evaluation?” “Is assessment the same as evaluation?” “What is the difference between a process evaluation and outcome evaluation?” We acknowledge that the field of evaluation (and related disciplines) has unleashed a torrent of definitions and concepts that have, unfortunately, created significant confusion over just what evaluation is and its value to strategic and programmatic decision making.
In our work, we use a definition of evaluation that is intentionally broad:
Evaluation is a systematic and intentional process of gathering and analyzing data (quantitative and qualitative), to inform learning, decision-making and action.
There are also some practical ways to think about what evaluation is – such as, evaluation is about increasing one’s confidence in making decisions or taking action. The idea here is that if you have credible and useful information, you will have greater confidence in moving forward. Evaluation is also about reality checking; we all walk around thinking we know things about the world. These are typically based on our experiences, assumptions, and beliefs. However, we only know what we know, and sometimes, that’s not enough. The data, findings, and insights generated from an evaluation can act as a check on these assumptions – in other words they can provide a window into the real world as experienced by others.
Here are a few more helpful ways to think about what evaluation is and isn’t:
Evaluation is….
- Systematic: It is intentional, planned, and purposeful; it is not ad hoc
- Grounded in a set of key questions: The questions provide the boundaries, scope, and direction for evaluation
- Intended to inform decision-making: Evaluation’s ultimate purpose is to inform decisions, be it decisions about process, outcomes, improvements, resource allocation, or even whether to continue the program or initiative, or change a strategy
Evaluation is not…
- Research: The purpose of research is to generate new knowledge, while evaluation is about making evaluative claims and judgments that can be used for decision making and action
- Just About Measuring Impact: Evaluation can provide information on a range of things – process, relationships, implementation, changes in systems, and early stage outcomes – not just impact
- Always done by an external third party: As long as an evaluation is systematic, planned, guided by key questions, is designed and implemented to inform decision-making, and follows the principles and ethics of good evaluation practice, evaluation can be done by internal staff
- Always expensive: Evaluations come in many different shapes and sizes. Depending on the purpose of the evaluation, how much data need to be collected and from how many sources, and over what period of time, evaluations can be small scale, taking only a few weeks, or larger scale, taking multiple years
Why understanding what evaluation is, matters
Language and definitions are a means of communicating knowledge and intent. Unfortunately, the image that the word “evaluation” often conjures up for many in the social sector is one of a post-facto, episodic, externally done impact assessment. Yet, evaluation is much broader than this. Reducing evaluation to only one approach or one purpose, as well as confusing it with other forms of inquiry, significantly limits its potential to provide deep learning, insights, and recommendations for making the best decisions possible in an evolving, complex, and unpredictable world.
We often wonder, “What’s the cost of not evaluating?” Can we really afford to design and implement social interventions without knowing how a new program actually works in the real world, with real organizations and beneficiaries? Or, how likely an intervention will achieve its goals and outcomes as the context changes? Or, how an intervention is affecting the lives of people we are hoping to change? Or, what really works with a certain population and/or in a specific community? Or, what kinds of ripple effects a new policy will have on organizations, communities, and residents? Evaluation helps answer these and many more questions.
All of this doesn’t mean that evaluation as we know it doesn’t need an “upgrade”. We recently researched and wrote about the idea of “Next Generation Evaluation”, which suggested that evaluation has to evolve in order to stay timely, relevant, and useful in a complex and changing world. However, the road to getting there begins with a fuller understanding of what evaluation is and isn’t.
