Here’s another edition of “there is in fact good and nontrivial scholarship in modern historical journals” (we need a catchier name for this series; previously: 1, 2). Today’s installment addresses the question implicit in this post title: how wild was the West?
THE ARTICLE
Randolph A. Roth, “Guns, Murder, and Probability: How Can We Decide Which Figures to Trust?” Reviews in American History 35, no. 2 (2007): 165-175. Accessed 8/20/09, here.
SOME NONTRIVIAL QUESTIONS RAISED
Was homicide really more common in the American West than elsewhere? How can we know?
DISCUSSION
This is a long-vexed issue in western historiography. It’s a normal point of New Western history that the West was a brutal, violent place—the arena of conquest and so forth. It’s probably not actually necessary for the historiography to function for the West actually to have been brutal; it just has to be assumed to be brutal and believed to be brutal—image is quite as important as reality in the history of the American West—but naturally a number of historians have fixed on the question of fact. Clare Vernon McKanna and David Peterson Del Mar say the West was violent; Robert R. Dykstra says it wasn’t.1
Here Roth picks up the thread. McKanna and others calculate western homicide rates by the modern statistical convention of quoting murders per 100,000 persons. But, Dykstra objects, a lot of these western towns had very few persons in them. One murder sends the annual homicide rate sky-high. Suppose that a drunk gunfighter had missed his mark and only wounded his prey: then you’d have a peaceable murder rate of zero. This is silly, right?
Well, if that were the actual character of these studies, maybe. But it’s not, Roth notes. McKanna and Del Mar have taken a number of counties over time, thus significantly increasing the sample size.
Peterson Del Mar found evidence in newspapers of at least 114 homicides in Oregon from 1850 to 1865, when the average adult population was 23,373, so the homicide rate in Oregon was at least 30 per 100,000 adults per year.16 If we assume that the inhabitants of Oregon were “typical” of residents of the Pacific Northwest in the 1850s and early 1860s—and hence selected “randomly,” for all intents and purposes, from the region’s inhabitants—we can arrive at a rough estimate of the region’s homicide rate. That is a reasonable assumption, because a majority of the region’s population lived in Oregon and because Oregon’s diverse communities were typical of the region as a whole. The formula for a 99% confidence interval for a random sample with over 100 cases is
π = p ± √2.58(p*(1-p))/n
Here, “π” stands for the real, but unknown homicide rate; “p” stands for the ratio of the number of homicides to the number of persons at risk in Oregon, 1850–65—114 divided by 373,964 (the average adult population times 16 years); “n” stands for the number of persons at risk—again, for Oregon, 373,964. If we assume that the population of Oregon was representative of the population of the Pacific Northwest, we would estimate that there is a 99% chance that the homicide rate in the Pacific Northwest was between 23 per 100,000 adults per year and 38 per 100,000 adults per year: a relatively narrow and high range.
That estimate, Roth points out, accords with the rate calculated by the mortality census of 1860, which put the murder rate at 24 per 100,000. Nineteenth-century Oregon was a violent place. And what held for Oregon held also for Washington, for British Columbia, and in McKanna’s studies, for California. Roth: “… the interval for all of southern and central California was between 60 and 70 per 100,000 adults per year—seven times the homicide rate in the United States today … An adult exposed to that rate for sixteen years stood a 1 in 96 chance of being murdered, and an adult exposed to that rate for 45 years would have stood a 1 in 34 chance of being murdered.”
But that is not the entirety of Roth’s article; he goes on to address the question raised by the Bellesiles episode and argues that even without knowing Bellesiles’ methods, just comparing his study with others, “The chance that Bellesiles is right is still nil—less than 1 in a zillion. Thanks to these statistics, we can reject Bellesiles’s findings with confidence. We do not have to examine his methods or his selection of counties. His estimates of gun ownership are impossibly low.”
This is the rare article where one really would have liked more. Roth contends,
It is important, however, that scholars be able to recognize works such as theirs as statistically reliable. Skepticism is fine, as is enthusiasm for studies that tell us unexpected things about the past. But if scholars are unable to distinguish good quantitative work from bad, they can make serious mistakes.
It would have been useful to know a bit more about why, in Roth’s view, historians so often fail to make adequate assessments of probabilities. And it would have been invaluable to have a systematic discussion of what kinds of diagnostic tools all historians—and reviewers of historical works—should have to hand.
1Richard Maxwell Brown puts an argument about why people lined up on different sides in the violence that there may or may not have been in the West.
46 comments
August 20, 2009 at 9:09 am
kid bitzer
well of course there was a lot of crime in the old west. anybody could of predicted.
it’s cause there weren’t enough guns out there to keep people from shooting each other.
August 20, 2009 at 9:19 am
James B.
I wonder how the inclusion of Native Americans might alter the rates? I’m assuming (perhaps incorrectly) Roth only tallied Anglo on Anglo murders, but violence is violence, no?
August 20, 2009 at 9:34 am
Sam-I-am
“It’s probably not actually necessary for the historiography to function for the West actually to have been brutal; it just has to be assumed to be brutal and believed to be brutal—image is quite as important as reality in the history of the American West”
I’m not entirely sure what that means, but it has lowered my opinion of historians.
August 20, 2009 at 9:49 am
Michael Elliott
Is Roth saying (or are you saying) that a decently-trained historian should have been able to see plainly how false the claims of Bellesiles were? Since I never did get around to reading the book, it’s never been clear to me how obvious the errors should have been to historians.
August 20, 2009 at 9:55 am
Vance
I initially read the first half of that as “The West was brutal even if the historiography doesn’t function”, but figure and ground reversed themselves soon enough.
Sam-I-am, eric is pretty clear that the reality does matter to him. What I’m not sure about is the extent to which he’s saying the “image” matters. Is it that the West had a reputation for violence at the time? Or that it has been a theater of violence in our collective memory since?
For comparison, we know that the idea of California as paradise has been “out there”. True, the state has never been Paradise, but there’s a sense in which the legend matters regardless.
August 20, 2009 at 10:18 am
SEK
I’m not entirely sure what that means, but it has lowered my opinion of historians.
Actually, that should lower your opinion of literary historians. We’re the ones who are all about image. History historians are more concerned with what actually happened.
August 20, 2009 at 10:38 am
Vance
Is WordPress being wonky? Our comments aren’t showing up on the sidebar.
August 20, 2009 at 10:39 am
Vance
Huh, all you have to do is challenge it, and it shapes right up.
August 20, 2009 at 11:01 am
Sifu Tweety
It would have been useful to know a bit more about why, in Roth’s view, historians so often fail to make adequate assessments of probabilities. And it would have been invaluable to have a systematic discussion of what kinds of diagnostic tools all historians—and reviewers of historical works—should have to hand.
While it should have occured to me before now, I’m very curious if any more elaborate statistical techniques — along the lines of the Lancet’s excess deaths study in Iraq — have been applied to this problem.
Actually I’d love to see a general purpose introduction to the use of statistical methods in historical scholarship. Could you go ahead and write that up, Eric?
August 20, 2009 at 11:06 am
eric
The image of the West as a wild place, as I meant to indicate by linking to Smith, has determined policy toward the West. I’m puzzled by the suggestion that it’s some kind of wild notion to say that people act on the reality they perceive, not on the one that dispassionate observers can ex post discern. Were it otherwise the current healthcare debate would sound very different.
And yes, this measure of violence excludes the Indian wars.
August 20, 2009 at 11:09 am
Jonathan Jarrett
It would be much appreciated, but as with too many of these things, the real trick would be getting the people who need to to read it. I am occasionally frustrated by the attentive audience of someone who’s talking unfounded crap but including enough numbers to dazzle them, not least because there’s almost no question one can ask which isn’t similarly beyond their grasp. I don’t even have any stats training. I wish there was an answer short of outright abusing the speaker, which so far I haven’t done. So far.
August 20, 2009 at 11:50 am
JPool
A couple of thoughts.
Image/myth/collective memory is one thing, but there’s also a cultural history question of subjective experience. How did people living in the West at time experience violence and the fear of violence (I’m reminded of E P Thompson’s point that people can be consuming more calories on average with industrialization and yet still feel more desparate and miserable)? As a result, I’m puzzled by the comparisons, to contemporary murder rates rather than to murder rates for the non-Western US of the same period. This would of course require additional studies to have been done, but if the point is about the West being objectively more violent place, then you’d want to know relative to other places rather than just other times.
To Michael Elliott’s question, I think that he’s saying yes a good historian should be able to figure this stuff out, or rather that that’s what McKanna and Del Mar did. I tend to think that when he said it’s just a matter of having taken a statistics course at some point and keeping the text books handy he’s almost laughably wrong. I know from math and could puzzle out the formulas that he uses if I had to, but I am in no position to apply them or to reanalyze previously published findings. Eric is, I’m sure, but he has special powers.
August 20, 2009 at 11:56 am
teofilo
It would have been useful to know a bit more about why, in Roth’s view, historians so often fail to make adequate assessments of probabilities. And it would have been invaluable to have a systematic discussion of what kinds of diagnostic tools all historians—and reviewers of historical works—should have to hand.
I suspect this would make a useful article all on its own.
August 20, 2009 at 11:57 am
eric
My only special power is the ability to htmlize a formula that RAH cheatingly published as a gif.
August 20, 2009 at 12:11 pm
eric
Sifu, you might try Roderick Floud, a classic, though my own go-to text is Feinstein and Thomas.
August 20, 2009 at 1:00 pm
pain perdu
there’s a sense in which the legend matters regardless.
Maxwell Scott had a view on this.
August 20, 2009 at 1:08 pm
Vance
Just so, French toast.
August 20, 2009 at 1:39 pm
eric
Will no one provide me a snappier title for this series? Think of the children!
August 20, 2009 at 1:50 pm
kevin
Will no one provide me a snappier title for this series? Think of the children!
“Duly Noted”?
“Journalistic Integrity”?
“Articles of Celebration”?
“I Can’t Believe It’s Not Bullshit”?
August 20, 2009 at 2:13 pm
JPool
“EotAW Reading Group and Awesome Club”
“Review Article Fun Time”
“Hey, Look What I Just Read”
“Droppin’ Mad Historiographic Science”
“I Never Meta-Narrative I Didn’t Like”
August 20, 2009 at 2:56 pm
silbey
“The Articles of Contemplation”
August 20, 2009 at 3:03 pm
Charlieford
“Chestnuts Among the Road Apples”
“Rare Reptiles from the Sewer”
“Bougainvilleas Among the Brambles”
“Which of These Is Not Like the Others?”
“How the Heck Did This Ever Get Published?”
August 20, 2009 at 3:15 pm
Ahistoricality
Why not make it part of the Blogging Peer-reviewed Research project? They have icons, an aggregator, etc.
August 20, 2009 at 4:03 pm
dana
“History as She is Spoke”
August 20, 2009 at 4:39 pm
bitchphd
I can’t think of a snappier title off the top of my head, but I do want to say, again, how much I freaking LOVE this series.
And if I think of a snappy title later, I’ll let you know.
August 20, 2009 at 4:42 pm
bitchphd
How about “Nontrivial Questions Raised”? Seriously. B/c I think what I like best about the series is the “let’s explain what academic articles do, and see, it really *is* interesting” aspect.
August 20, 2009 at 4:48 pm
eric
Thanks for the suggestions, all. I think I like “Nontrivial questions raised”, actually, which is brief and to the point. Though I’m tempted by “one of these things is not like the others” because of the Muppet connection.
Ahistoricality: I tried just now to register as qualified for the peer-reviewed blogging program and if I meet their standards I will probably use that too.
Thanks everyone, again.
August 20, 2009 at 4:48 pm
Michael Elliott
How about, “Hey all you right-wing critics of the Academy, maybe you should actually read some scholarship instead of just bitching about how inane it is?”
August 20, 2009 at 4:50 pm
eric
Sorry, Michael. Nominations have been closed.
August 20, 2009 at 5:33 pm
dana
I know from the Internet that no one studies the West in the academy because we’re all studying gay leftist tree-hugging spotted owlish miscegenation.
Is quantitative work standard in history?
August 20, 2009 at 5:44 pm
andrew
As a result, I’m puzzled by the comparisons, to contemporary murder rates rather than to murder rates for the non-Western US of the same period.
Roger McGrath did this in a historiographical essay at the end of Gunfighters, Highwaymen and Vigilantes. This is another one of those books I read a long time ago and don’t have on hand, but as I recall it, he compared crime rates in 19th century Bodie, CA and Aurora, NV with both urban crime rates elsewhere in the country around the same time and with contemporary murder rates (as the book came out in the late 80s, I think he used statistics from the 1980 census). I don’t remember all the results exactly, but I think the gist of it was that homicide rates were higher in Bodie and Aurora than elsewhere, but that property crime wasn’t especially bad.
August 20, 2009 at 5:47 pm
Bitchphd
I win!
August 20, 2009 at 5:50 pm
andrew
Looks like you can get some of McGrath’s analysis under the chapter “Scholarly Assessments” here.
August 20, 2009 at 5:52 pm
Charlieford
Muppets, arise! Defend your hearths, your heathers, your bogs and your blogs!
August 20, 2009 at 5:53 pm
andrew
Actually, upon closer review, the McGrath chapter with the statistical comparisons is the one just before “Scholarly Assessments.” Should have checked more closely before posting.
August 20, 2009 at 7:50 pm
JPool
andrew, thanks. I was puzzled by the way that Roth framed the arguement in the article, but didn’t know whether contemporaneously comparative figures had already been compiled.
To expand on my thought from before, while there have been any number of criticisms raised as to Bellesiles’s statistical analysis and research methods (tick marks on a legal pad, while it’s a method I’m fond of, is not a very clean way to generate quantitative data), Roth focuses on the sort of category errors (“arms” counted as armour) that anyone can understand. All of his analysis about how Lindgren and Heather’s anaysis is convincing is based upon the presumption that Jones’s data was accurate and properly generated, and therefor Bellesiles’s findings become the independent variable to be judged against it. As I understand it, Roth’s probability finding basically establish that it’s not possible for both Bellesiles and Jones/Lindgren and Heather to be right. I’m not trying to be pomo and say that it’s all in how you look at it, just that the work of distinguishing between Bellesiles’s figures and those derived from Jones is done at the level of “Jones’s sample … has been verified by subsequent scholars” and the critique of the categorical methods by which Bellesiles arrived at his figures.
Roth does a really good job of explaining why McKanna’s and Del Mar’s numbers are convincing, and his analysis of their figures point to a more generalizable truth. His critique of Dykstra, could have stopped when he pointed out that McKanna’s and Del Mar’s methods are not what Dykstra claims they are. (I’m even more confused about how Dykstra thinks he’s talking about postmoderism, but that’s a whole nother bramble.)
So, to rephrase eric’s comment in the OP, I’m sure conviced that Roth does fine statistical analysis, but he hasn’t done much to convince me that I need to be able to.
August 20, 2009 at 8:27 pm
Cosma Shalizi
I am really not happy with the first quoted paragraph from Roth (for starters, I cannot figure out what sort of data-generating process he has in mind), but unfortunately CMU doesn’t seem to have access to the journal. Can anyone point me to an ungated copy?
August 20, 2009 at 10:51 pm
Walt
This reminds me of a question I had the other day. I was reading about the history of medieval Germany, and comparing maps of the political boundaries over time, when I thought, “I don’t need maps. I need a database.” Then I idly wondered if you could make a database for the History of the World, and what you could put in it. What sort of systematic knowledge do we have about world history? What parts of that systematic knowledge lend themselves to being flattened to a few numbers?
August 21, 2009 at 9:07 am
erubin
So historians have only just discovered standard deviation? I had no idea there was so little science in social science…
Also, can’t we be fairly certain that the number of murders is systematically low? I would need to read up on their methodology, but I assume that the 114 tallied homicides were surely distinct from each other– i.e., there’s no accidental “double counting” of a victim named John Smith who also went by Jack Smith. Tack on to those confirmed murders many unsolved missing persons cases and unreported murders and the murder rate could conceivably be much higher. Of course, the population could also be underestimated, but that seems less likely to me.
August 21, 2009 at 10:37 am
serial catowner
Well, the Old Medieval Village study groups in England have pretty much remodeled history as their databases of actual medieval villages have grown.
You can get a taste for that here by looking at a railroad map from 1900. You will see dozens of towns and villages listed that no longer exist. Then, get another map from a different railroad serving the same area, and you will find dozens more towns and villages that have disappeared.
History, always best served with a tempting platter of archaeology.
August 21, 2009 at 10:51 am
serofriend
found evidence in newspapers
erubin: Your criticism, I believe, stems from the above four words. New social and economic historians–in addition to sociologists–have faced these and other obstacles for decades. Your critique, in other words, is a bit redundant.
So historians have only just discovered standard deviation?
Not sure you have access to History Cooperative, but you should at least glance at internet and wiki reviews of texts such as <A HREF="http://www.helium.com/items/95016-book-reviews-a-shopkeepers-millennium-by-paul-johnson"/A> and Time on the Cross for introductory purposes only. Reviews of these and other books would introduce you to applied criticism in U.S. socioeconomic history.
Historian Alan Taylor provides an excellent overview in the historiographical trends culminating in the new social history in his foreword to Minutemen and Their World.
August 21, 2009 at 10:54 am
serofriend
Sorry, Shopkeepers Millinneum, didn’t go through the first time.
August 21, 2009 at 1:12 pm
erubin
Sorry, Sero– I got carried away with being a jerk again. My criticism wasn’t leveled at all historians. From what I can tell, Dykstra had a major oversight and Roth provided the necessary correction.
As for the murder rate being systematically underreported, I wasn’t so much arguing against Roth’s analysis but against Dykstra’s. Roth’s study does more than enough to sink Dykstra’s claim, I just think it can go even further. Of course, this opens up questions of whether murders were underreported back East as well, so perhaps it’s best left untouched. I’ll leave issues like this to the people who know what they’re doing.
August 21, 2009 at 2:21 pm
serofriend
I got carried away with being a jerk again
You are no jerk. There are commenters on here (usually random) that approach such a sign, in my not-so-esteemed opinion. From what I can tell, you are not one of them. In any case, things get overheated sometimes. Good to take stock of things.
I’ll leave issues like this to the people who know what they’re doing
I didn’t say that you were wrong, per se.
August 21, 2009 at 3:15 pm
JRoth
Can I claim any credit for the excellent work of my co-Roth?
You’re welcome, historians. We Roths are always happy to help.
August 24, 2009 at 10:11 am
herbert browne
The map/graph of gun manufacture, gun sellers, and handgun homicides in the July, 2000 Harper’s in a map called “The Gun Gap” was… what- memorable, I guess… & maybe salient, as well. It made the impression that the Civil War basically “headed for the hills” (ie Rockies) and was a continuing underlying part of “Western lifestyle”… which has persisted, statistically, in those places where urbanization hasn’t altered the demographics too much, yet.
http://harpers.org/archive/2000/07/page/0088
(may require a subscription)
^..^