MENU TITLE: Performance Measures: What Works?. Series: OJJDP Fact Sheet #71 Published: December 1997 Author: Eileen M. Garry Subject: Program evaluations 4 pages 7,517 bytes ------------------------------ Figures, charts, forms, and tables are not included in this ASCII plain-text file. To view this document in its entirety, download the Adobe Acrobat graphic file available from this Web site or order a print copy from NCJRS at 800-851-3420. ------------------------------ Performance Measures: What Works? by Eileen M. Garry The single most important question in any endeavor that seeks to solve a problem or effect change is: What works? Particularly in times when resources are limited, taxpayers are understandably reluctant to support programs and activities without evidence that they are effective. Consequently, it is imperative for public agencies and organizations funded by tax dollars to be able to measure and evaluate their performance. Many different approaches can be used to measure results. Much of the tradition of performance measurement comes from the industrial part of the private sector, where work measurement looks at how to improve production. This model does not translate well into public or private-sector enterprises that provide services. A better approach for service-oriented entities is the change-agent model. This model recognizes that the agency or program provides services (inputs) that act upon the environment to produce demonstrable changes in the well-being of clients, families, or communities (outputs). The change-agent model avoids a common problem that arises when the industrial model of performance measurement is used with service agencies -- mislabeling inputs as outputs. For example, "number of clients served" is not an output. It is an input, an action that should lead to a change in client or social conditions, which is the real output being sought. An approach to performance measurement based on the change-agent model is set forth in A Guide to Developing and Using Performance Measures in Results-Based Budgeting, a May 1997 working paper of The Finance Project. The paper was developed to assist in the important work of achieving and measuring outcomes for children, their families, and the communities where they live. As such, it should be of particular interest to juvenile justice agencies and youth-serving organizations. The paper is a user-friendly guide for taking steps to hold initiatives accountable for their performance by meeting reasonable targets while maintaining a focus on overall desired results. In addition, the Guide does the following: identifies characteristics of an effective performance measurement system, gives examples of what works for States and local governments, notes a number of potential obstacles to the implementation of an effective system, and utilizes a "four-quadrant approach" for developing measures that are useful and quantifiable and that fit into a broader results accountability measurement framework. The paper also contains a selected bibliography of results and performance accountability documents. As the title suggests, the Guide makes a distinction between two types of accountability: results and performance. Results accountability deals with conditions of well-being for children, families, and communities that cut across agencies and programs. Examples include stable and self-sufficient families, young people avoiding trouble, and safe and supportive communities. Performance accountability is concerned with how well agencies and programs perform. Some examples of performance accountability are percentage of child abuse investigations initiated within 24 hours of a report, amount of child support collected for each dollar spent on child support enforcement, and police or fire response time. Taken together, the two levels of accountability cover the whole range of questions from the broadest level view of community accountability for child and family well-being to the smallest increment of performance by a particular program or individual. The four-quadrant approach offered in the Guide is based on two sets of interlocking questions that all performance measures try to answer. The first set addresses quantity and quality: How much was done? How well was it done? These questions must be crossed with another set that relates to effort and effect: How hard did agency or program staff try? What did they produce? All performance measures to answer these questions can be sorted into four categories, represented by the four-quadrant matrix. One of the immediate consequences of this sorting scheme is the recognition that not all questions are equally important. Quality is more important than quantity, and measurement of effort is of less concern than measurement of effect. Thus, the numbers in the matrix quadrants rank the questions from most important (1) to least important (4). Many performance measurement documents provide a great deal of information on quantity of input (upper left quadrant) but very little on quality and output (the other three quadrants). The approach proposed in the Guide focuses on the quality column measures and, in particular, on measures of the quality of outcomes. The Guide provides helpful examples of program performance measures using the four-quadrant approach. Sample quadrants are given for education, health, child welfare, welfare reform, mental health, juvenile justice, and child care licensing. Performance measurement, as part of a results-based accountability system, can help build public confidence in government and community institutions and lead to improved results for children, families, and communities. The Guide can serve as a blueprint for a new, clear, and useful way to obtain an answer to that most basic and crucial question: What works? ------------------------------ For Further Information For a copy of A Guide to Developing and Using Performance Measures in Results-Based Budgeting by Mark Friedman, send a check for $7.50 to The Finance Project, 1000 Vermont Avenue NW., Suite 600, Washington, DC 20005. This Guide is one in a series of papers and toolkits for creating results-based planning and budgeting systems under development by The Finance Project. To learn more about these tools, call The Finance Project at 202- 628-4200 or visit their World Wide Web site at www.financeproject.org. For additional information on related topics, including evaluation measures and techniques, program evaluation, and a variety of juvenile justice issues, call the Office of Juvenile Justice and Delinquency Prevention's Juvenile Justice Clearinghouse at 800-638-8736. ------------------------------ Eileen M. Garry is Program Analysis Officer in the Office of Juvenile Justice and Delinquency Prevention. The Finance Project, a nonprofit organization established by a consortium of national foundations, conducts an agenda of policy research and development to improve the effectiveness, efficiency, and equity of public financing for education, other children's services, and community building and development. Excerpts from A Guide to Developing and Using Performance Measures in Results-Based Budgeting appear in this Fact Sheet with the permission of the author and The Finance Project. FS-9771