Big data analytics is the process of obtaining and analyzing large data sets to generate useful insights, connections, correlations, and patterns, with the ultimate goal of improving a nonprofit's decision making and strengthening its impact.

Driven largely by advances in technology, big data analytics provides an opportunity to process data in four novel and improved ways:

Read the Full Report

1. Volume: enables the analysis of massive data sets
2. Velocity: provides real-time insights and analysis
3. Variety: allows the linking of data from multiple data sets and sources
4. Variability: manages differences in data formats

How it's used

Big data analytics can help nonprofits make better decisions about the most effective way to allocate their scarce resources to maximize social impact. For example, big data analytics could be employed to:

• Generate insights about the behavioral patterns of beneficiaries and how they use programs and services. Then use that insight to allocate resources to the beneficiaries or geographies with greatest need, which can be determined using methods such as collecting and analyzing hotline data both for immediate response and large-scale pattern recognition (Global Human Trafficking Hotline)
• Target new programs or services at underserved geographies or unmet needs that can be identified by, for example, integrating emergency response service and labor statistics data to locate neighborhoods with a high need for new services (Buffalo's Operation Clean Sweep)
• Increase response rate to disasters and emergencies by employing techniques such as using GPS data to identify population movements after catastrophes (2010 Haiti earthquake response)

Big data that is useful for nonprofits is most likely to come from local, state, or federal government databases relating to education, public assistance, health, criminal justice, or transportation systems. Increasingly, however, social entrepreneurs are using more newly available sources such as social media interactions, consumer transactions, GPS data, weather, and traffic patterns. In many cases, important insights can be gained by linking previously separate databases (e.g., identifying parents involved in the criminal justice system and targeting their children with education support services, or using student performance data to rate teacher training programs).

Though there are many barriers to overcome in the social sector (e.g., lack of coordination, unreliable or inconsistent data, privacy concerns, underdeveloped data-processing systems, insufficient talent, scarce resources), there may be an increasing number of significant opportunities for the social sector to take advantage of big data analytics.

Methodology

Given the challenge of data availability and access, and the talent and technology required for analysis, nonprofits should carefully consider when big data analytics is appropriate. When it is, the steps are:

1. Identify an appropriate topic: Identify the key questions or critical decisions that could benefit from big data analytics (e.g., participant selection, prioritization of new geographies)
2. Determine the available data sets and sources: This may include the organization’s own data, government data sets, and data from other nonprofits, universities, or corporations. Clarify what it would take to access the desired data (e.g., completing a memorandum of understanding with a government department and adhering to privacy regulations, negotiating a nonprofit rate with a for-profit data warehouse). It is possible that the ideal data is simply not available, in which case proxies will have to be used instead or the desired data will have to be tracked and gathered over time.
3. Decide on the analysis plan: Clarify exactly what analysis will need to be done to generate the desired insights, and what talent and technology is required to complete this assessment. Depending on the scope of the project, it may be possible to conduct the analysis with existing staff, or it may be necessary to hire an expert firm, a consultant, or a new staff member. New software may also be required, especially if the analytics are likely to be repeated regularly. Ensure that the analysis plan complies with relevant privacy regulations and the organization’s ethical standards.
4. Conduct the analysis and generate insights: Implement the analysis plan, and identify the key findings, reality-checking all insights using firsthand knowledge and expertise.
5. Learn, share and refine: Decide how to change the nonprofit’s programs or activities in response to the findings. Share the findings with other organizations that could benefit from them. Test and refine new approaches to create deeper impact.

Related topics

• Performance measurement and improvement
• Beneficiary satisfaction measurement
• Social media
• Beneficiary and donor segmentation

Additional resources

Harnessing big data to address the world's problems
In this report, McKinsey & Company outlines the ways nonprofits and other social sector organizations can utilize big data analytics to gain greater insight into problems and, in turn, design better solutions.

Big Data for Social Innovation
In this article, the authors highlight key steps the social sector needs to take to better leverage the possibilities of big data analytics for social change.

Examples and case studies

How Google Is Fighting Sex Trafficking With Big Data
Funded by a grant from the Google Impact Awards, The Polaris Project's Global Human Trafficking Hotline Network is using big data analytics to track patterns of human trafficking around the world in order to provide better interventions.