Evidence-Based Policing Matrix

Home › Evidence-Based Policing › The Matrix › Inclusion Criteria & Methods Key

Inclusion Criteria

For a study to be included in the Matrix, it must:

  • Be methodologically rigorous and able to be categorized as one of the three types of methods listed below under “Methods Key”). Specifically, a study must either be a randomized controlled experiments or quasi-experiments using matched comparison groups or multivariate controls;
  • Focus on interventions that were primarily police initiated or dominated (even though other agencies might be involved); and
  • Include crime or disorder as a measured outcome. Thus, studies examining fear of crime or police legitimacy were not included in this Matrix, although we encourage further matrices to be developed in these areas.

Methods Key

We only include studies which satisfied at least a “moderate” threshold of scientific rigor as defined below. To decide which studies to include, we were guided by the Scientific Methods Scale (SMS) designed by Lawrence Sherman and colleagues for the University of Maryland’s 1997 report to Congress entitled Preventing Crime: What Works, What Doesn’t, What’s Promising (available at: http://www.ncjrs.gov/works/):

Moderately Rigorous “M”: Separate comparison group present but non-randomly constituted; extensive information provided on pre-treatment equivalence of groups; obvious group differences on important variables. This is somewhat similar to the SMS score of “3”.  However, more specifically, we included studies only if the comparison group was the same type of unit as the intervention group (e.g. a police beat if the target area is a police beat). A study that used a target neighborhood and then the rest of the city as the comparison, for example, would be excluded.  Additionally the study had to meet at least one of the following criteria: 1) comparison group was well-matched, 2) use of multivariate controls, or 3) use of rigorous time series analysis.

Rigorous “R”: Separate comparison group present; extensive information provided on pre-treatment equivalence of groups; only minor group differences evident. This group was similar to an SMS score of “4”.

Very Rigorous “VR”: Random assignment to comparison and treatment groups; differences between groups are not greater than expected by chance; units for random assignment match units for analysis. This group was similar to an SMS score of “5”.