We know that metaphors matter. They shape the way we think about ideas, concepts and entire disciplines. More often than not, we use metaphors to describe or highlight a point of view and in the process perpetuate both conscious and unconscious biases.
The field of evaluation is no stranger to metaphors. Most often, evaluation draws its metaphors from the medical research field. We talk about “trials” using “control” and “treatment” groups that received “dosages” of the intervention and exhibit a certain “response” (or not) to that dosage. In her recent article in the Stanford Social Innovation Review, Lisbeth Schorr traces the roots of Randomized Controlled Trials to the a 1948 paper on Streptomycin treatment of tuberculosis.
At other times, we use metaphors from the natural sciences, particularly physics. Ever since Auguste Comte and his colleagues enshrined the doctrine of positivism in the early 19th century, we have tended to hold the natural sciences to the same exacting standards as the physical sciences, leading some to coin the term “physics envy.” Just as we do in the physical sciences, we test “hypotheses” and seek to create “models,” “theories,” and “empirical” inferences based on evidence.
However, the more experience I gain in evaluation, the more I’m drawn to metaphors from another field… the legal field!
Here’s why. Evaluators are often in the unenviable position of piecing together available evidence and making the best of what they can cobble together from research and experience in order to make meaning. Not unlike the legal field where a combination of evidence combined with probable cause supported by eyewitness testimony and possible situational cues lays the groundwork for building a case. None of these by themselves is 100% credible (exhibit A: the films Roshomon and Twelve Angry Men) but taken together they help build a fairly solid case, either for the prosecution or for the defense.
I shared this insight at a recent panel discussion and was met with almost raucous audience approval. As you might imagine, I was feeling pretty self-satisfied at having come up with a clever new metaphor for evaluation. The feeling was short-lived, however, as a conversation with a colleague revealed that the evaluation field had indeed dabbled with the legal metaphor in the 1970s and early 1980s. An “adversary model” of evaluation (sometimes also called the “judicial model”) was in vogue, where two different evaluators present conflicting positions using data and the whole process is set up like a jury trial. Alas, I thought, could this be one more example of the time-worn phrase “there is nothing new under the sun?”
Upon further investigation, I learned that the model was applied to some educational evaluations, but had several limitations and eventually fell out of favor. In a paper from 1980, Worthen and Rogers describe how terms such as “statement of charges” and “guilty or not guilty” were problematic and how the model reinforced a tendency to view evaluation as something you do when a problem or crisis occurs.
However, it is important to separate the failing of a specific “model” from the usefulness of a broader metaphor. I’m hopeful that the legal metaphor – a kindler, gentler version of it – will still gain popularity in 2013. Given the increasing complexity of the social sector, traditional evaluation approaches grounded in past metaphors are becoming less and less useful, and new metaphors are needed for dealing with ambiguity and uncertainty. The legal metaphor is far from perfect, but the legal field does have a long history of working with less-than-ideal circumstances; after all, there is no such thing as a randomized controlled “murder” trial.
I rest my, ahem, case.
What other metaphors are you drawn to?
