On November 16, 2015, Change Capital Fund (CCF) and Philanthropy NY co-hosted a forum about how community organizations, evaluators, and government agencies are using data as an anchor for multi-service strategies in high-poverty neighborhoods. The discussion focused on new efforts to use data to evaluate and track outcomes and the accompanying big challenges of unifying fractured systems given their number, divergent metrics and conflicting regulatory requirements.

The discussion clearly identified that City government and CCF-funded community-based organizations (CBOs) are working toward holistic, coordinated services to meet residents’ needs (e.g., housing, education, and employment services). Working across program silos, they are using data to align efforts, hold program partners accountable, and improve outcomes. An outstanding question is how to marry institutional data and community knowledge to increase collaborative CBO/City problem solving.

Below is a synopsis of the panel conversation:

Basil Reyes, Director of Evaluation and Research, Cypress Hills Local Development Corporation (CHLDC)

CHLDC, which offers educational, housing and workforce services to 10,000 residents of Cypress Hills and East NY Brooklyn each year, is establishing an organization-wide database and performance management system. CHLDC is using the Efforts to Outcomes software to improve and enhance data tracking to better understand clients’ needs and outcomes. This system and the new position Mr. Reyes’ fills as Director of Evaluation, is enabling staff to monitor and improve programs and services.

Mr. Reyes cited their High School Choices Program as an example of how CHLDC is using data to identify and address program gaps. When CHLDC looked at their baseline data, they found that none of the middle school students in the organization’s programs were being accepted into the 10 high schools that were regularly listed among students’ top choices; rather, students were attending local schools with graduation rates of less than 50%. As a result, the program director worked with the choice high schools to better understand admissions criteria.

Data collection and integration remain challenging due to CHLDC’s limited resources, a common problem in the nonprofit sector. Greatly adding to that challenge are the numerous, proprietary databases required by funders. CHLDC is required to report out program outcomes using 13 different databases. For youth programs alone, six separate data systems are required. These challenges are common throughout the nonprofit sector.

View the CHLDC Presentation

Matthew Klein, Executive Director, NYC Center for Economic Opportunity and Senior Advisor for Service Innovation in the Mayor’s Office of Operations

According to Mr. Klein, the Center for Economic Opportunity, located in the Mayor’s Office of Operations is using data as a locus for city government– creating integrated, comprehensive, cross-agency data systems that are being used to reduce inequities among rich and poor New Yorkers.

One example is the Social Indicators Report which analyzes social conditions across eight domains: education; housing; empowered residents and neighborhoods; economic security and mobility; core infrastructure and services; diverse and inclusive government; health and well-being; and, personal and community safety. The indicators are being tracked to determine what works to improve social conditions. The annual Social Indicators Report will inform the Mayor’s short and long term plans for responding to the significant disparities identified.

Starting in Spring 2016, the City will map the location of its services contracts. Additionally, more than ten agencies are creating an integrated data platform that will allow better coordination of services, which will, for example, allow them to identify and support the individuals/families with the highest multi-agency interactions.

Despite significant progress in building cross-agency data systems, the City too faces enormous challenges: it is awash in data and has to ensure the right data is being analyzed; it must ensure that all agencies are using the same defined metrics; and it must coordinate responses across many agencies.

Eric Cadora, Chief Research and Data Strategies Officer, Mayor’s Office of Criminal Justice (MOCJ)

MOCJ is playing a coordinating role in several initiatives to reduce both crime and incarceration. Mr. Cadora stated that the agency has entered “a new age in data use in NYC” with the Mayor’s Action Plan for Neighborhood Safety (MAPS), a nine-agency partnership that will use a multi-strategy approach to improve safety in and around selected, high-crime public housing authority developments. The MAPS initiative will undertake corner-by-corner surveys to glean local knowledge and concerns. Beginning in February 2016, residents, agencies, CBOs and Police will use and share data to problem solve.

MAPS’ Neighborhood Safety & Justice Index goes beyond crime statistics to track risk factors in targeted geographic areas such as housing instability, child trauma, educational disruption, and neighborhood satisfaction, and others. MAPS will coordinate activities to improve safety by also addressing health, employment, community cohesion and the disadvantages of concentrated poverty. For example, MAPS is changing “hot spots to cool places,” by improving the physical environment with lighting, new community centers and social events.

Challenges include translating risk factors to actionable, effective strategies and also moving partner agencies’ leadership to ask, ‘how does my agency fit into a problem solving solution for this place?’ rather than singularly focusing on pre-determined agency metrics.

David Greenberg, Senior Associate, MDRC

Mr. Greenberg posed the overarching question of what it takes to turn data into action. The cholera epidemic in SoHo, London in the 1800s provides a model example. Only by mapping where the sick lived, was it discovered that the affected were concentrated around the Broad Street water pump. What action did the government take to remedy the epidemic? It removed the handle from that pump.

Although most problems are not so easily analyzed and solved, with more open and available data, the work is starting to be done. For example, CCF grantees may soon be able to use a new Unemployment Insurance data law to match their clients to employment and wage data over time. Greenberg sites this effort as a case that demonstrates the requisites to make data meaningful. The Unemployment Insurance data is relevant because CCF is an anti-poverty initiative and all the grantees offer workforce programming; it is accessible; and, the grantees have ‘agency,’ that is, they have the ability to interpret data and to alter program strategy to improve outcomes.
Mr. Greenberg offered these insights about turning data into action for high-poverty areas:

● Build data infrastructure among community organizations;
● Align research to well-resourced community interventions; and,
● Be cautious about data biases, interpret change contextually, e.g, no change may signify progress in slowing a negative change whereas big changes may not be meaningful locally and only indicate national trends.

Cross-program data analysis is necessary for CBOs, government, and funders to know if integrated programming is successful.

Data Big and Small – MDRC presentation 

Responses to Panel Discussion:

Ilene Popkin from the Citizens Housing and Planning Council responded to the panel, emphasizing the importance of linking government spending to the data on location and impact of government services.

Maryanne Schretzman, Executive Director of the NYC Center for Innovation through Data Intelligence (CIDI) underscored the challenges in time: social problems take years to resolve but government wants to document results in months.

Ms. Schretzman offered a cautionary tale: in analyzing the bullet-ridden planes that came back from war, it was concluded that it is not necessary to protect the engines since none of the planes exhibited bullet holes on the engine. Of course, planes shot through the engine were not among those that came back. As with bullet-ridden planes, we need to protect the people and neighborhoods who may not readily show up in our data.