The Reality-Based Community

  The John Lott Affair

The toughest thing to deal with in any sort of argument is apparent misconduct by people on your side. It's no surprise that the gun-control folks have been giving John Lott a hard time about his apparent fabrication, not just of survey results, but of an entire survey, in an attempt to cover up a mis-citation of someone else's work in his book Less Guns, More Crime. But those on the pro-gun side of the question must have been tempted to keep quiet, to deny the evidence, or to make excuses for Lott.

It is very much to the credit of Glenn Reynolds that he addresses the issue head-on, and doesn't cut Lott much slack. The Lindgren post he links to is really devastating. Tim Lambert has a summary of the blogospheric action, which also leaves Lott looking quite bad. Lambert asserts that Lindgren was an early critic of Bellisles, the historian who faked data in an attempt to show that privately-owned guns were rare in Colonial America.

It's not quite inconceivable that Lott actually conducted the survey in question, but his explanation really requires a lot of believing. (Did Lott's dog eat his homework a lot when he was in school?) In particular, one has to believe not only that Lott can't remember the names of any of the students who did the survey for him in 1997 but also that not a single one of those students, hearing about the controversy, would have come forward to identify himself and his (or her) co-workers. (Recall that the University of Chicago isn't the University of Ohio; it has only hundreds of students per undergraduate class, so if Lott really wanted to he could just get a class list from the alumni office and run his finger down it until he spotted some of the names.)

A second survey, also done by Lott, purportedly replicates the results of the first. Unlike the first survey, the second has a still-extant questionnaire and data tabulation available for inspection. One scholar on the pro-gun side of things has argued that the correspondence of results is evidence, not merely for the truth of the underlying claim about defensive gun uses, but also for the proposition that the first survey had actually been conducted (else how could Lott have guessed the result so accurately)? But that reasoning doesn't seem sound to me. As Lindgren points out, the reported results are inconsistent with many prior studies. And survey results aren't at all hard to fake, especially considering that the people making the phone calls for the second survey were very likely aware of the desired results and of the high stakes involved.

How much does any of this matter? That really breaks into two questions:

--How should Lott's findings now be treated?
--How should Lott himself now be treated?

The answer to the first of those questions turns out, I think, to depend on the answer to the second.

Glenn makes the point that the claim in question -- that only 2% of self-reported "defensive gun uses" actually involve firing the weapon, as opposed to merely brandishing it -- is quite peripheral to the claim for which Lott is most famous: that laws permitting anyone allowed to own a gun to get a permit to carry a concealed weapon reduce the incidence of crime.

Fair enough. But what basis is there for believing that more important claim? Why, Lott's own reports of the results of his econometric analysis, based on a data set he assembled. Both assembling a data set and doing econometrics involve making a very large number of choices, leaving plenty of room for legitimate differences of opinion -- and plenty of room for outright cheating.

Other scholars have re-analyzed the "More guns, less crime" claim and found it wanting. That's not surprising, and the result would, in the ordinary case, be simply a research dispute, which might either be resolved by further investigation or left an open question. But that assumes that we have researchers of professional standing on both sides of the question.

With respect to Lott's standing in the research community, if he faked results it doesn't matter whether he faked them on something serious or something minor. A single case of unmitigated scientific fraud should, and usually does, end an academic career. No one will hire that person, give him research grants, or publish his findings. And it has to be that way. Scientists don't have auditors. They have to trust one another. Faking is hard to detect. Once someone has faked results, there's simply no reason to believe that any other results he comes up with aren't faked.

If Lott found that liberalizing concealed-carry laws caused reductions in crime, and other researchers using similar methods and data found otherwise, and if Lott can be shown to have practiced scientific fraud, then unless and until someone else does the same work from scratch and gets Lott-like results, the default option is to believe the other researchers.

Nothing we discover about Lott can take away the fact that years of experience with liberalized concealed carry have provided little or no evidence of increased firearms-using crime as a result. The Violence Policy Center's report License to Kill, which details every single recorded crime committed by anyone who obtained a concealed-carry permit in Texas from 1996 to 2001 in an attempt to show that the policy had bad results, in fact demonstrates the contrary. (Someone ought to tell the VPC that it's not necessary to have a concealed weapon, or even a permit for a concealed weapon, in order to drive drunk, so that the finding that some permit holders were arrested for DUI tells you precisely nothing about the merits of the policy.)

That finding seems to me to be a very strong argument for "shall-issue" laws: they give some people a right they value, at very little apparent cost to anyone else. Not everyone will be convinced by that argument; for example, it's at least conceivable that concealed-carry laws increase the level of fear without increasing the actual rate of armed assault. If I were writing a law about concealed carry, I'd like to tighten up a little on who gets to have a gun, and I'd like a law against possessing a loaded firearm (concealed or not) while under the influence of alcohol or other intoxicants. Still, the case seems like a strong one even without the Lott claim to back it up.

But the claim itself -- that liberalizing concealed-carry laws can significantly reduce crime -- should now, I think, be regarded as probably false, unless and until Lott is vindicated in the survey affair or someone not tainted by a history of scientific misconduct comes up with new evidence in its support.

And even if Lott's thesis is revived, Lott himself should -- again, unless he can demonstrate that he did in fact carry out the 1997 survey as he has described it -- be excluded from the research community. Fool me once, shame on you ...