Posted by Eugene Volokh:
John Lott Responds
to some posts (linked to at the end of this one) that criticize his
work in light of the National Academy of Science report on gun control
laws:
Last month, the [1]National Academy of Sciences issued a 328-page
report on gun control laws. The big news that has been ignored on
all the blog sites is that the academy's panel couldn't identify
any benefits of the decades-long effort to reduce crime and injury
by restricting gun ownership. The only conclusion it could draw
was: Let's study the question some more.
The panel has left us with two choices: Either academia and the
government have wasted tens of millions of dollars and countless
man-hours on useless research (and the panel would like us to spend
more in the same worthless pursuit), or the National Academy is so
completely unable to separate politics from its analyses that it
simply can't accept the results for what they are.
Based on 253 journal articles, 99 books, 43 government
publications, and some of its own empirical work, the panel
couldn't identify a single gun control regulation that reduced
violent crime, suicide or accidents.
From the assault weapons ban to the Brady Act to one-gun-a-month
restrictions to gun locks, nothing worked. (Something that I have
been the first person to investigate empirically for many of these
laws, and I also had been unable to find evidence that they reduced
violent crime.)
The study was not the work of gun-control opponents. The panel was
set up during the Clinton administration, and of its members whose
views on guns were publicly known before their appointments all but
one had favored gun control. Something that I [2]wrote up about the
panel three years ago is still relevant.
While the panel dealt with a broad range of gun control issues,
only one issue has received attention on different blogs:
right-to-carry laws. In fact, the panel apparently originated with
the desire from some to respond to the debate on that issue and to
respond specifically to my research that concludes that allowing
law abiding citizens to carry concealed weapons reduces crime. I
originally overheard Phil Cook and Dan Nagin discussing the need
for a panel to "deal with" me in the same way that an earlier panel
had "dealt with Isaac" Ehrlich's work showing that the death
penalty deterred murder. They agreed and Nagin said that he would
talk to Al Blumstein about setting up such a panel. Needless to
say, that is what ended up happening.
1) James Q. Wilson's very unusual dissent is very interesting (only
two out of the last 236 reports over the last 10 years have carried
a dissent). Wilson states that all the research provided
"confirmation of the findings that shall-issue laws drive down the
murder rate . . . " Wilson has been on four of these panels and
never previously thought that it was necessary to write a dissent,
including the previous panel that attacked Isaac Ehrlich's work
showing that the death penalty represented a deterrent.
Wilson said that that panel's conclusion raises concerns given that
"virtually every reanalysis done by the committee" confirmed
right-to-carry laws reduced crime. He found the committee's only
results that didn't confirm the drop in crime "quite puzzling."
They accounted for "no control variables" - nothing on any of the
social, demographic, and public policies that might affect crime.
Furthermore, he didn't understand how evidence that was not
publishabled in a peer-reviewed journal would be given such weight.
The non-results are basically due to dropping all the control
variables (particularly the arrest rate which is not defined when
the crime rate is zero). When that happens a lot of observations
with zero crime rates are introduced. The problem with using OLS
when you have all these zero crime rates is that if a crime rate is
already zero, no matter how good the law is, it can't lower the
crime rate any further. There is thus a positive bias in these
results. Plassmann's two papers (his piece in the Journal of Law
and Economics with Nic Tideman and his paper with Whitley in the
Stanford Law Review) show how you can address this as a count data
problem. Although his research consistently shows statistically
significant results that shall issue laws reduce crime, the
National Academy report ignores the research.
The panel's discussion of Duggan's results focuses on the
regressions without any control variables and that use the OLS
estimates when they have a large number of zero values for the
crime rates.
2) As an interesting aside, there are a number of factual mistakes
in the NAS report and those mistakes work against my findings. For
example, Figure 6.1 makes a mistake where it shows the increase in
violent crime of 7 percent in year one, when the amount is 5
percent (7-2, where 2 is from the trend). (Of course, the overall
problem with the hybrid approach is discussed below.) There are
significant drops in crime in Table 6-3 that are statistically
significant, but they are not properly marked to indicate that is
so. Even something trivial as the number of states currently with
right-to-carry laws is wrong, 36 (not 34) (and if Minnesota is
included the number is 37).
3) Last year there was a debate over the use of clustering between
Ayres and Donohue and me, but [3]the statements of the NAS panel
corresponds extremely closely to what was written in my original
paper with David Mustard.
4) p. 127: "We focus on the conflicting results . . ." No attempt
is made to give readers an idea of the frequency or importance of
unusual results. Take the results in Table 6-3. For Plassmann and
Whitley, the panel doesn't mention that Plassmann and Whitley say
that there are "major problems" with the particular regressions
that the panel decides to report and more importantly that the
effects in those regressions are biased towards zero (see point 2
above). For Moody's results, they show only two specifications of
all the results that he reports and don't mention that the one
weird result that he got was from a specification that he flagged
as problematic and not controlling for other factors.
Even with the very selective sample of regressions that they pick,
there is not one statistically significant bad effect of
right-to-carry laws on murder. Only one case for robbery and that
is one problematic specification from Ayres and Donohue.
5) Hybrid model. The so-called hybrid model used by Ayres and
Donohue finds that the law dummy variable is positive while the
trend variable indicates that crime rates decline over time. While
Plassmann and Whitley do a good job explaining why the "hybrid"
model produces misleading results and the panel never discusses
their critique (looking at the crime rates on a year by year basis
show no initial increase in crime), it still would have been useful
for the panel to at least say whether the "hybrid" results produced
a statistically significant temporary bad effect. The problem with
determining statistical significance is that when both the dummy
and trend variables are on at the same time, we are concerned about
the net effect not just the dummy variable by itself as Ayres and
Donohue argue. The answer for all those results in the panel's
Table 6-4 is "no."
6) Reset tests. Professor Horowitz's discussion of the reset tests
seem too strong since I provided the panel with the reset tests
done for a wide range of estimates. Even accepting that the Reset
test is appropriate (and no one else on the panel also uses this
test in their work), there are many estimates where the results
pass this test and he should thus conclude that those indicate a
drop in violent crime.
7) Using too many control variables. Bartley and Cohen and I report
all possible combinations of the control variables and show a great
deal of consistency in the results. The only difference between
these and those discussed in the NAS report is that these
regressions included the arrest rate because of the zero crime rate
problem.
8) Process. While the NAS is in name an academic organization, the
process was hardly an academic one. Members of the panel were
forbidden to talk to me about the issues being examined by the
panel. Despite promises to get my input on the panels' review as it
went forward, that never occurred. In particular, Charles Wellford
promised me that I would be able to look at the tables and figures
in the report. If I had been involved, I could have helped catch
some of their mistakes. When the report was finally released to the
public, I was promised that I would get a copy at the beginning of
the presentation and that I would be allowed to ask questions. I
was told that they preferred that I not attend the presentation,
but there would be no problem with me asking questions. Instead
even though the presentation ended a half hour earlier than
scheduled because there were supposedly no more questions, my
questions were never asked. (I had one main question: Professor
Wellford mentions all the research that has been done on
right-to-carry laws, but if he is correct that right-to-carry laws
are just as likely to increase as decrease crime, can he point to
one refereed journal article that claims to find a bad effect from
the law?) Despite promises to the contrary, I did not receive a
copy of the study until well into the afternoon and then only after
a reporter from USA Today sent me a copy.
Minor notes: Despite claims to the contrary, I responded to the
Ayres and Donohue study in [4]January of 2004. (Simultaneously, it
goes unnoticed that Ayres and Donohue themselves ignored virtually
all of Plassmann and Whitley's points.)
In commenting on the report, others have raised additional issues
that the NAS study did not find relevant. As to the claims raised
again in these posts reguarding Jim Lindgren's investigation of the
"phantom survey," many are apparently unaware that David Gross,
David Mustard, and I have said that Lindgren has grossly
mischaracterized what we said to him. For comments by Gross and
Mustard, please see statements 3 and 4 in this [5]link.
For a general response to the charges on the survey and other
issues you raise see this [6]link. False claims have been made with
regard to these issues and the pseudonym.
Claims have also been made by Jim Lindgren regarding the
demographic control variables, but he fails to note that it is only
for the state level regressions and not the county level
regressions where some of the significant results are affected.
Given all the combinations of control variables that have been
examined, even in that case, one wants some theory for why you
selectively include what appears to be a weird combination of
demographic controls. I think that Lindgren is a biased observer.
He was upset after a critical piece that I published on his work in
2003 and his attacks started shortly after that. Further his
attacks are untrue.
Final comments.
It is hard to look through the NAS panel's tables on right-to-carry
laws and not find overwhelming evidence that right-to-carry laws
reduce violent crime. The results that don't are based upon the
inclusion of zero values noted in point 1 above. Overall, the
panel's own evidence from the latest data up through 2000 shows
significant benefits and no costs from these laws.
My impression is that Gary Kleck also has a very similar reaction
to the panels' findings regarding surveys on self defense.
I express no opinion on this debate, but I thought I'd pass along John
Lott's reactions.
References
1. http://johnrlott.tripod.com/gun_violence_.pdf
2. http://johnrlott.tripod.com/op-eds/LATimesNASFirearms.html
3. http://johnrlott.tripod.com/other/ClusterDebateReconsidered.html
4. http://ssrn.com/abstract=523002
5. http://johnrlott.tripod.com/surveysupport.html
6. http://johnrlott.tripod.com/malkinsoped.html
_______________________________________________
Volokh mailing list
[email protected]
http://highsorcery.com/cgi-bin/mailman/listinfo/volokh
