NCJ Number
252901
Journal
Journal of Experimental Criminology Volume: 14 Issue: 2 Dated: June 2018 Pages: 155-185
Date Published
June 2018
Length
31 pages
Annotation
This article highlights the importance of documenting the step-by-step processes used for the selection of comparison areas when evaluating a community-level intervention that targets a large-scale community.
Abstract
The proposed method is demonstrated using a propensity score matching framework for an impact analysis of the Cure Violence Public Health Model in Philadelphia. In selecting comparison communities, propensity score models were run using different levels of aggregation to define the intervention site. The trade-offs made are discussed. The authors found wide variation in documentation and explanation in the extant literature regarding the methods used to select comparison communities. The size of the unit of analysis at which a community is measured complicates the decision processes, and in turn, can affect the validity of the counterfactual. The authors note the importance of carefully considering the unit of analysis for measurement of comparison communities. Assessing the geographic clustering of matched communities to mirror that of the treated community holds conceptual appeal and represents a strategy to consider when evaluating community-level interventions occurring at a large scale. Regardless of the final decisions made in the selection of the counterfactual, the field could benefit from more systematic diagnostic tools that document and guide the steps and decisions along the way, and ask "Could there have been another way of doing each step, and what difference would this have made?" Overall, across community-level evaluations that use quasi-experimental designs, documentation of the counterfactual selection process will provide a more fine-grained understanding of causal inference. (publisher abstract modified)