At the request of the Seattle Office of City Auditor, researchers from the School of Criminology and Criminal Justice at Arizona State University and the Center for Evidence-Based Crime Policy in the Department of Criminology, Law and Society at George Mason University undertook a review of the policing evaluation literature to categorize strategies on our Review of the Research Evidence page based on “what works?”, “what’s promising?”, “what doesn’t work?”, and “what do we need to know more about?”

Our focus was largely on evaluations examining the effectiveness of policing interventions for reducing crime, but we also examine “what works?” for increasing fairness and improving citizen perceptions of police legitimacy.  We also encourage you to visit the Resource Library page to learn more about the sources we used for the review. Much of our assessment of what does and does not work drew on the work by Cynthia Lum and colleagues in the Evidence-Based Policing Matrix. The Matrix is a research-to-practice translation tool that organizes these stronger studies visually, allowing agencies to view the field of research, from its generalizations to its particulars. We also drew upon systematic reviews of policing topics, many from the Campbell Collaboration Crime and Justice Group. Our review initially covered studies published through the end of 2013.  The review was updated by Cody Telep in Spring 2016 to cover studies published through the end of 2015 and in Spring 2018 to cover studies published through the end of 2017.

---

Seattle Police Department Case Study

As a case study, the Seattle Police Department (SPD) provided short descriptions of how their practices aligned with the literature and ideas in effective policing.  This case study is current as of March 2014 (update for 2018 coming soon).

For each item in the table on the Review of the Research Evidence page, SPD has provided a baseline measure of the degree to which it currently incorporates this research in its practices (as of March 2014-updates pending for 2018).  This is intended to provide a frame of reference for SPD as it builds its capacity to apply scientific research on what is effective in policing.

 

Symbol	Definition	Criteria
+	SPD is a leader in using this research	Existing research is systematically incorporated into practice in SPD; SPD has performed rigorous scientific evaluations in this area.
X
√	SPD is emerging in its use of this research	Existing research is incorporated into practice in a limited but consistent way in SPD; rigorous scientific evaluations may be underway in SPD.
X
–	SPD could take steps to incorporate this research	Existing research not incorporated only used in isolated instances within SPD; no rigorous scientific evaluations have been performed or are underway in SPD.

---

Coding Interventions

We used the following four categories for police interventions designed to address crime and/or increase the fairness and legitimacy of policing, drawing upon a number of resources.

---

What works?

We coded strategies under “what works?” when the bulk of the research evidence suggests the approach is effective in reducing crime and/or increasing legitimacy and fairness. This assessment is based on a review of rigorous research studies (using the Evidence-Based Policing Matrix for studies on crime control effectiveness), the conclusions of past narrative reviews, and the conclusions of systematic reviews (when available).

Example: We coded hot spots policing under “what works?” because there is a large rigorous evidence base that suggests hot spots policing is an effective approach for reducing crime and disorder. This conclusion was echoed by the National Research Council in 2004 and by a 2012 update of a Campbell Collaboration systematic review by Braga and colleagues.

---

What's promising?

We coded strategies under “what’s promising?” when the available research evidence suggests the approach is effective in reducing crime and/or increasing legitimacy and fairness, but this evidence base remains limited in size or scope. This assessment is based on a review of rigorous research studies (using the Evidence-Based Policing Matrix for studies on crime control effectiveness), the conclusions of past narrative reviews, and the conclusions of systematic reviews (when available).

Example: We coded the information-gathering interrogation approach under “What’s promising?” because a systematic review by Meissner and colleagues concluded the strategy can help elicit confessions from suspects while also minimizing the number of false confessions when compared to the accusatorial interrogation approach. All of the studies comparing the two approaches in an experimental setting used laboratory studies, typically with college student volunteers. While the evidence to date shows the promise of information-gathering approaches, we did not code this as “what works?” because of a lack of “real world” evaluation studies on interrogation methods.

---

What doesn't work?

We coded strategies under “what doesn’t work?” when the bulk of the research evidence suggests the approach is ineffective in reducing crime and/or increasing legitimacy and fairness. This assessment is based on a review of rigorous research studies (using the Evidence-Based Policing Matrix for studies on crime control effectiveness), the conclusions of past narrative reviews, and the conclusions of systematic reviews (when available).

Example: We coded Drug Abuse Resistance Education (D.A.R.E.) under “what doesn’t work?” because there is a large, rigorous evidence base of studies suggesting D.A.R.E. has little or no impact on adolescent, drug, alcohol, or tobacco use. These were also the conclusions of two systematic reviews examining the effectiveness of D.A.R.E. It is important to point out that nonstatistically significant findings do not always mean that a program is ineffective- it could indicate issues with the implementation of the program or with the ability of the evaluation method to detect a program effect. With D.A.R.E., however, the overwhelming evidence that D.A.R.E. has little or no effect on adolescent drug use led us to place this program in the “what doesn’t work?” category.

---

What do we need to know more about?

We coded strategies under “what do we need to know more about?” when the research evidence is too limited to reach strong conclusions about whether an approach is effective in reducing crime and/or increasing legitimacy and fairness or when the current evidence is too mixed to reach an assessment of whether the strategy works or does not work. This assessment is based on a review of rigorous research studies (using the Evidence-Based Policing Matrix for studies on crime control effectiveness), the conclusions of past narrative reviews, and the conclusions of systematic reviews (when available).

Example: We coded broken windows policing under “what do we need to know more about” because of conflicting results from the literature on the crime control effectiveness of this approach and the difficulties of evaluating most broken windows strategies. Some scholars conclude that broken windows policing was a major contributor to the New York City crime decline, while others argue the approach had a smaller impact, and still others conclude there was no impact of broken windows policing on crime. These divergent findings made it difficult to reach strong conclusions about the effectiveness of this strategy.