Evidence-Based Policing Matrix

Home › Evidence-Based Policing › The Matrix › Inclusion Criteria & Methods Key

Inclusion Criteria

For a study to be included in the Matrix, it must:

  • Be methodologically rigorous and able to be categorized as one of the three types of methods listed below under “Methods Key.” Specifically, a study must either be a randomized controlled experiment or quasi-experiment using matched comparison groups, multivariate controls, or rigorous time series analyses.
  • Focus on interventions that were primarily police-initiated or dominated (even though other agencies might be involved).
  • Be written in or translated to English and describe intervention activities in sufficient detail to provide tangible implications for police practitioners and policymakers.
  • Include crime or disorder as a measured outcome analyzed using statistical significance testing. Thus, studies focused on fear of crime, police legitimacy, police use of force, or investigative outcomes were not included in this Matrix.

Methods Key

We only include studies which satisfied at least a “moderate” threshold of scientific rigor as defined below. To decide which studies to include, we were guided by the Scientific Methods Scale (SMS) designed by Lawrence Sherman and colleagues for the University of Maryland’s 1997 report to Congress entitled Preventing Crime: What Works, What Doesn’t, What’s Promising (available at: http://www.ncjrs.gov/works/):

Moderately Rigorous “M”: Separate comparison group present but non-randomly constituted; extensive information provided on pre-treatment equivalence of groups; obvious group differences on important variables. This is somewhat similar to the SMS score of “3”.  However, more specifically, we included studies only if the comparison group was the same type of unit as the intervention group (e.g. a police beat if the target area is a police beat). A study that used a target neighborhood and then the rest of the city as the comparison, for example, would be excluded.  Additionally the study had to meet at least one of the following criteria: 1) comparison group was well-matched, 2) use of multivariate controls, or 3) use of rigorous time series analysis.

Rigorous “R”: Separate comparison group present; extensive information provided on pre-treatment equivalence of groups; only minor group differences evident. This group was similar to an SMS score of “4”.

Very Rigorous “VR”: Random assignment to comparison and treatment groups; differences between groups are not greater than expected by chance; units for random assignment match units for analysis. This group was similar to an SMS score of “5”.