Social Sector Measurement: Down but Not Out

This blog originally appeared on the Stanford Social Innovation Review website.

For those pining to see a revolution in how social sector organizations view and use measurement, two reports were launched towards the end of last year with good insight into where we are and how far we still have to travel. To sum up: Buckle your seatbelts—it’s going to be a long ride!

Innovation Network’s “State of Evaluation 2012” study (October 2012) draws from a representative sample of US nonprofits and provides a bevy of statistics and insight, including comparisons to results from a similar 2010 survey. The good news is that 90 percent of nonprofits report measuring their work, and while funders and boards remain the primary audiences for measurement, more than three quarters also report using their measurement to plan and revise strategies and programs.

That said, the proportion of nonprofits that have at least one full-time employee devoted to measurement remains frighteningly low (18 percent), as does the proportion of them that are spending at least 5 percent of their budget on measurement (27 percent)—figures that have increased only a few, potentially statistically insignificant percentage points since the 2010 survey. Further, the proportion of nonprofits that have revised their theories of change or logic models within the past year has actually declined, with fewer than half of nonprofits using measurement to adjust their programs at least annually. Finally, nonprofits continue to rate research and evaluation as their two lowest organizational priorities.

Not surprisingly, the study paints funders as one of the biggest culprits, both in how they support grantees in using measurement and how they use it themselves. (One particularly telling datapoint: Funders are nearly half as likely as nonprofits to use measurement to plan and revise their programs).

Taken together, these and other findings suggest a lot of measurement is happening, but it’s unlikely the quality required for organizations to truly learn what’s working and continuously improve their programs. With fewer than one-third of nonprofits demonstrating promising capacities and behaviors to meaningfully engage in measurement, the authors conclude that the state of evaluation in the social sector can best be judged only as “fair.”

The disappointing role of funders is echoed in the Center for Effective Philanthropy’s “Room for Improvement” (September 2012). The headlines from this study:  only 32 percent of nonprofits believe foundation funders have been helpful to their ability to measure progress, and only 29 percent have foundation funders that provide financial and/or non-monetary support for measurement efforts. And given that foundations place relatively high importance on nonprofits’ evidence-bases (according to recent Bridgespan research), the figures above likely overstate the support nonprofits are receiving from their typical funders.

Both reports conclude that funders can play a pivotal role by changing their approaches to working with and supporting grantees in measurement. But if 2012 taught us anything, the greater promise may lie in funders supporting initiatives that get nonprofits learning from one other. For instance, Mario Morino’s Leap of Reason, which recently surpassed the threshold of 50,000 books in circulation, is chock-full of practitioner essays and examples that provide rich, often first-person accounts around how measurement has changed the trajectories of nonprofit organizations. PerformWell, which launched earlier this year, is attracting hundreds of nonprofit leaders to its free webinars to hear how their peers are establishing performance cultures and regularly using data to improve. And the “Saving Philanthropy” documentary, with its compelling footage of leaders and staff of Nurse Family Partnership, Roca, and others describing how they use performance measurement, has met with great acclaim from those who have participated in screenings and workshops.  

We believe the biggest returns on investment in educating and inspiring nonprofit leaders around performance measurement will tend to accrue at the earliest stages of a nonprofit’s development, when leaders are still forming their operating philosophies and cultures. To see greater improvement by the time “State of Evaluation 2014” is released, we’ll need many more initiatives like the ones described above—initiatives that help nonprofit leaders (particularly of young organizations) learn directly from their peers about how measurement can first and foremost be a tool to improve outcomes for those they serve and better advance their missions.

What are your views on the future of social sector measurement? What other exciting initiatives have you seen?

Posted: 1/15/2013 2:30:50 PM by Matthew Forti | with 1 comments


Comments
Glenn
You are too considerate of the unwillingness and defensiveness of funders to engage in transforming the way they pay for services.

In our state we joined with another provider and went to our legislature with a proposal to reform the social and mental health services are paid for. Our recommendation was to pick one area, children's mental health, and create a pilot project to develop a payment system that pays for services based on outcomes. This proposal received unanimous support from both parties in committee and is moving through our current legislature.

Even though there was no written or testimonial support from the state agency that administrates children's mental health, at least they have not come out publicly in opposition.

We can lead positive change when we ALWAYS live our mission and values.
1/25/2013 4:11:49 PM