NRA CEO and Executive Vice Prisdent Wayne LaPierre said that the assault weapons ban didn’t work because the 4/20/1999 shooting at Columbine occurred. In the same interview he said that the only way to keep kids safe at school is to have trained armed guards protect the schools.
John Lott has said that gun moratoriums don’t work because England saw crime increase.
Things that have been proven to work don’t work all the time. A 100 percent success rate is not how something is defined as ‘what works‘.
The Definition of ‘What Works’
‘What works’ is defined as a statistically significant reduction in the occurrence the crime. Statistically significant means that the intervention or program is responsible for the reduction in the occurrence, and the reduction is not due to chance or part of a trend.
For example, in medicine, what works is defined as a treatment working for a significant proportion of people who get the treatment compared to people who don’t get the treatment. It is not something that works for everyone all the time.
In crime policy, if we have a prison program (as an example) that is designed to reduce violence after an inmate is released from prison, we need to compare people who get the program to people who don’t. So, we take two groups of inmates who have the same risk to re-offend and the same needs. There is virtually no difference between the two groups except for the one group gets the program or intervention, whereas the other group does not. Both groups have post-release challenges, both groups have the same risk to re-offend, the same needs, etc. If we see a statistically significant reduction in violence after release we can say with a reasonable certainty that the program is ‘what works’ in reducing violence after prison. This is an empirical approach to finding out what works.
Anecdote Example Isn’t an Example of “What Works” or “What Doesn’t”
What partisans do is find one example of how something worked or didn’t work and offer that as evidence. That is not evidence, it is anecdote. It is a cherry-picked story and not representative of the bigger picture. They are doing a disservice to ‘what works.’
We can’t say something did or didn’t work unless we have something to compare it to. Without that equal comparison group, we can’t say crime rates would have been any different because the increase or decrease could be:
- pre-test / post- test confound, which doesn’t rule out third variables
- part of a trend
When John Lott says that homicide rates increased after Britain started a gun moratorium in 1997, that is true, but crime was already increasing so the gun moratorium had nothing to do with increasing homicide rates.
On the flip side, when anti-gun advocates say that the crimes rates would have been higher with the guns, they need to prove such a statement. However, doing so would be very difficult because one can’t prove a negative (maybe unless there is are two groups of virtually equal groups of places where one had a moratorium and the others did not and we saw a statistically significant reduction in shootings, and third variables and pre-trends were ruled out.)
Wayne LaPierre said that schools with armed guards is ‘The one thing that would keep people safe.’ By his own standard, this would not work. Virginia Tech had its own police force. Fort Hood was a military base. LaPierre said that the police procedure has been changed since Columbine, and that may be true, but the armed guard still had a shootout with the shooters that was ineffective. Furthermore, armed guards at schools would do nothing to prevent the mass shootings in movie theaters or malls, like the shootings on 7/20/2012 in Colorado or 12/11/2012 in Oregon.
Until the advocates on both sides of an issue start using ‘what works’ properly and not for political gain, we won’t have good advocacy for public policy. We are likely to hear more junk science and bad research be positioned as fact. This is very dangerous.
Higher standards in rigorous measurement methodology are important to really find out what works and what doesn’t.
As awful as high profile examples of mass shootings are, it is important to remember that high profile events are high profile precisely because they are unusual and unlikely. Making policy based on high profile events is a surefire way to overreact and make inefficient and, worse, ineffective policy. A high profile event is good time find out where a shortcoming of a policy or a failure of a policy might reside, but a high profile event is not necessarily what policy should target. Doing so would result in the majority of cases being marginalized and a strategy designed around an unlikely event.
Paul Heroux is a State Representative-elect from the Second Bristol District (Attleboro). He previously worked as Director of Research for the Massachusetts Department of Correction, Assistant to the Commissioner of the Philadelphia Jail System, and he has a Master’s in Criminology from the University of Pennsylvania and a Master’s in Public Administration from Harvard. Paul can be reached at firstname.lastname@example.org.