March 1, 2016

Unbroken News: A Federal Program that Is Accountable and Working

From its inception in the early days of the Obama presidency, the Social Innovation Fund (SIF) was positioned as a different kind of program, actively touted by the White House to be a "new way of doing business" for the federal government.

Seven years later, despite its early hype and controversy, a set of independent reports released in December by the Corporation for National and Community Service (CNCS), the SIF's parent agency, clearly establish that the SIF is very different—in its readiness to embrace accountability for results and its demonstrated success to date in generating them.

As the SIF's initial director, I'm very sad to say, however, that these reports appear to have gone unnoticed by the nonprofit press, the public policy community, even the blogosphere. While under some circumstances this neglect might be benign, today it is not. Indeed, given the prevailing skepticism about Washington, the SIF's recent Congressional appropriations struggles, and the fast-approaching end of the Obama administration, it is essential that all who genuinely care about accountability in government and the concept of funding programs that really work take notice, digest this rich vein of information, and let your views on its value be known.

The basis for high expectations of the SIF was compelling. The program was born out of the anti-establishment zeal of the social entrepreneurship movement and was purposefully designed to equip the government to play a more productive, collaborative role in driving real change in low-income communities:

  • Focus on program models devised by innovators throughout the US—not "experts" in Washington DC;
  • A two-stage grant process that awards federal money to experienced nonprofit intermediaries (rather than building additional federal bureaucracy) who then provide sub-grants and capacity-building assistance to selected, promising, community-based nonprofits;
  • High match requirements at both stages to attract private capital likely to provide a more sustainable funding base for selected nonprofits;
  • Belief in openness and transparency in all key processes and competitions;
  • Emphasis on active sharing of knowledge and experience among participating organizations and the broader social sector;
  • Most critical of all, total commitment to the use of valid evidence and the principle of true accountability for performance.

Significantly, the SIF's commitment to evidence and accountability began with itself. The very first initiative funded from the SIF's discretionary pool of funds was a systematic effort to engage external stakeholders in specifying the SIF’s most critical outcomes and to design a rigorous process for third-party evaluation—what became the SIF "national assessment." Through an open, competitive bidding process, CNCS chose the firm ICF International to conduct this multi-year study.

To make this worthwhile, the evaluators focused the national assessment on two fundamental questions of impact: whether the SIF was having the intended effects on the capacities and practices of its direct grantees, the intermediaries; and whether the community-based nonprofits selected and supported by those grantees were actually making a difference for the people their programs aimed to serve.

To make this valid, the evaluators then selected sound, if imperfect, approaches to executing the study. For the former question, ICF chose to employ a "quasi-experimental design" using two comparison groups—SIF applicants who were not selected for grants and a nationally representative sample of grantmaking nonprofits—to assess 14 specific areas of the grantees' capacity and practice, including their adoption of evidence-based grantmaking strategies and their ability and willingness to support, scale and evaluate the nonprofit service models they had selected. For the latter, ICF agreed to do a "meta-synthesis" (i.e. comprehensive roll-up analysis) of all formal evaluations of these nonprofit service models that had been completed by the cut-off date—just 26 of the nearly 100 underway, all of which will be publicly assessed.

The reports posted in December are the first products of the national assessment, with much more to come in 2016, and they contain a diverse and tantalizing body of information, analysis, and insights in seven documents:

  • The meta-synthesis of evaluations of the grantees' nonprofit service models;
  • A technical report and an executive summary analyzing the SIF's impact on the 14 areas of organizational capacity and practice;
  • Four issue briefs delving deeper into key aspects of the SIF grantee's experience: partnering with various parties; build nonprofit capacity for scale, evidence and impact; using data to improve programs; and actually implementing changes in critical organizational capacities.

So what makes these reports special?

First, going back to the animating spirit of the SIF, they represent an emphatic affirmation by SIF program participants—SIF/CNCS, the grantees and their nonprofits—of their commitment to hold themselves accountable for their performance, based on solid data.

Second, and possibly most important, these reports contain a unique, rich trove of information about practices, processes, and functions that are critical to the success of any high-performing nonprofit organization. These include methods and costs of rigorous evaluation, lessons on conducting open, competitive selection processes, and approaches to building operating capacities and organizational scale.

Last but not least, these reports contain data-driven observations and conclusions about how well the SIF actually works, as well as substantive recommendations on how it can do better. Bottom line, the key findings are clear, well-substantiated and, by and large, positive:

  • Regarding the efforts of the SIF to build the capacities of its grantees, between 2009 and 2014 grantees improved on 13 of the 14 measures. Relative to the comparison groups, the SIF’s performance was strong on rigorous evaluation.  On the other measures, results were positive but more mixed, especially vis-a-vis the cohort of rejected SIF applicants;
  • As for the formal evaluations of nonprofit service programs selected by grantees, 85-percent were found to have been implemented with fidelity and to have delivered positive impact on all or some targeted outcomes. Although only four examined program cost-effectiveness, all four found "low-cost" solutions to addressing the targeted outcomes.  

Is this the last word on the SIF's effectiveness? Absolutely not—there is much more content yet to be mined. But it does reinforce the SIF's core value proposition to the Congress and the American public that a well-structured, well-implemented government program can successfully deliver the substantive value it was created to generate. and prove it.

Of course, this fact does not preordain continued support for the SIF. To that matter, one question—very much informed by the national assessment—looms large: having come this far, how can the government best proceed to derive the greatest public benefit from the considerable asset the SIF has built and still embodies?"

How the Congress and next administration resolve that question will reveal the extent to which the federal government has embraced a "new way of doing business."


Creative Commons License logo
This work is licensed under a Creative Commons Attribution 4.0 International License. Permissions beyond the scope of this license are available in our Terms and Conditions.