Impact Measurement – Part One of Three

 A post by James V. Toscano

The Wall Street Journal Report on “Innovation in Health Care” of April 16 has a number of articles on directions  in the way health care is moving, with constant referral to the mountains of data routinely collected on the multitude of variables considered important in measuring outcome.

Having spent over thirty years of my career in nonprofit health care organizations managing research and education, I am most familiar with data, its collection, analyses, and uses. Modern medicine is built on empirical studies which determine significant directions in the diagnosis, treatment and care of patients.

Measurement of effectiveness and efficacy determine which drugs, which procedures, which devices will be used. Determination of scores on risk factors influence the course and outcomes of treatment.

The Centers for Disease Control and Prevention have pioneered in the use of empirical measures to study the spread of disease over time. Their series of maps of the United States demonstrating the spread of obesity, for example, are classics.

The use of epidemiological methodologies has made a  significant difference in many aspects of our health care model in the US. Is there further applicability to these techniques?

Prevention as the New Normal

Most observers in health care know that the only real long-term solution to the rising health care cost problem is through prevention. Rather than continuously pulling people out of the disease well, finding out and preventing them from falling in, is now the preferred solution.

The Affordable Health Care act rewards providers for limiting and preventing illness. Whether it stands or falls in the next months, the prevention reward structure will continue in health care and we will see more and more preventive measures taken in our own health care.

Given the movement, and demand in some quarters, for outcome measurement in the rest of the nonprofit sector, is there anything we can learn from the experience of health care?

The answer is an emphatic “yes,” so let us proceed to take small steps in this and succeeding posts.

First Things First

Nonprofits struggle with measuring outcomes for a number of reasons. First, they really don’t know what is really wanted. At one time they reported on numbers of people served, only now to be told that that’s like a body count, an input, and what is wanted is an output.

Often, however, no time period is suggested. Is it graduation from high school, entrance into college, graduation from college, getting a good job, earning a certain income, becoming the CEO, retiring well, what?

The above example demonstrates a number of questions for which we need to develop standardized measures. We are dealing with individuals, groups, whole populations in a discrete neighborhood, ward, city, region, state, section, nation.

Let’s start simply and just use two measures: population and geography. If these are the defining characteristics, what additional empirical measures do  we then use?

A Concrete Example

Let’s take a concrete example: the Harlem Children’s Zone. The Zone, at any one point in time, is a distinct number of square blocks in  upper Manhattan. Through Census tracts, school records and other registries, an approximate number of children of a certain age group can be identified. Certainly the number is dynamic, but systems can be developed to track the identified cohorts.

Now, what do we want to know about these kids? First we establish a baseline of the variables in which we are interested: number of cohort kids living in single parent homes, numbers of unemployed parents of the cohort children, number of cohort children achieving  median scores on standards tests by grade, number, percent of those starting school who graduate from high school, etc,etc.

Clearly, there are a multitude  of variables we could measure, and should measure. The point here is that all of these variables can be and some are already measured, so we can begin building a data base that can be used not only  in Harlem, but in other comparable locations.

Similar to Framingham in medicine, Harlem, or our own Northside Achievement Zone in Minneapolis, can begin to delineate in bolder fashion education risk factors that lead to dysfunction and those which lead to graduation. Much is already known although more needs to be known if we are to maximize whatever effective prevention inputs are available to us.

We are at this impact measurement crossroads where hard data, satisfaction studies of those impacted directly, and long-term, longitudinal assessments of existing and new data need to be used for new and creative insights. The work  has already begun. Now we need to standardize it.

Copyright 2012 The Good Counsel, division of Toscano Advisors, LLC. May be duplicated with citation.

---