[NOTE: so as not to disrupt the ongoing debate in the comment thread I am updating this posting to reflect my rebuttal. Comments are now open again on Sound Politics.]
Last week I posted an analysis based on several studies from the CalTech/MIT Voter Technology Project: Heads or tails… why we’ll never know who really won the governor’s race.”
In a critique of my analysis, posted today on neoconservative blog Sound Politics (“Horsing around with statistics“), Stefan Sharkansky seems to pursue two distinct theses: a) that Dino Rossi will be the “statistical winner” regardless of any likely outcome of the third count, and b) that I am stupid… or at the very least, an unreliable (if “occasionally entertaining”) source of information and analysis.
While I appreciate the compliment part of his closing, backhanded compliment, I hope Stefan understands if I stray from the dispassioned tone of my original analysis to engage him on a more equal rhetorical footing. For if you deconstruct his critique you will find that it is built on three of the main pillars of pop-neocon discourse: Dismissiveness, Misrepresentation, and Confusion.
And now thanks to David, we have even more uninformed debate.
He discusses a couple of research papers, which he apparently read, but didn’t understand very well;
First, I’d just like to point out that the very existence of Stefan’s critique disproves his first statement. My research was in response to contradictory numbers irresponsibly being bandied about as to the relative accuracy of vote counting technology in general, and hand recounts in particular. And due to my initiative, we now have two competing analyses referencing the highly respected CalTech/MIT studies. (And we don’t need a spreadsheet to count ’em.)
I made a point of encouraging people to read the studies for themselves, and apparently Stefan followed my lead. So I’d think Stefan would congratulate me for contributing to an informed debate… unless by “informed” he prefers “informed by rumor and innuendo.”
David then announces the following conclusions, none of which are correct in the context of the Washington gubernatorial vote:
Stefan then goes on to list my four “erroneous claims” (I’ll discuss them separately in a moment), before definitively stating:
Again, all of David’s statements 1 – 4 are incorrect.
Stefan lists his interpretation of my “conclusions”, out of context, and in his words. But notice how clever he is in reinforcing my wrongness. At the top of the list he states that my so-called conclusions are incorrect “in the context of”, but at the bottom, he drops the caveat: “Again, all of David’s statements 1 – 4 are incorrect.”
(I’d say this also falls under the category of “Dismissiveness” but why pick nits?)
I may not have a background in statistics, but I sure as hell know language, and while Stefan might object to my quibbling over his sentence structure, he is well aware of how his readers will interpret that line: these are “David’s statements” and they are “incorrect.”
So let’s be clear about what I wrote. I attempted to answer several questions regarding the general accuracy of voting technologies, using the best research available, and in doing so, I presented the conclusions of the CalTech/MIT studies. I also attempted to define the terminology as best I could.
In an 1172 word essay, I didn’t mention the current WA election until word 976, and the only conclusion that I presented as my own was that a 42 vote margin out of 2.8 million cast was statistically meaningless given the margin of error.
So, let’s take a look at all my statements of error.
1. The “Residual voting” rate (includes both blank and improperly marked ballots), which he calls “the primary statistical measure of the performance and accuracy of voting technologies” is 1 – 2%.
“He calls”… sheesh! In the words of CalTech/MIT:
A number of important studies of the performance and accuracy of voting technologies have sought to measure the error rate of vote tabulations. The main metric that emerges from these evaluations uses “residual votes” — the discrepancy between total ballots cast and votes cast for a particular office, such as president or governor. The incidence of residual votes should be unrelated to the type of technology used, and the difference in residual votes across technologies measures the extent to which errors in the casting or tabulation of votes are attributable to specific technology. Similar jurisdictions using different technologies ought to have the same residual vote rate, on average. By this metric, hand-counted paper ballots and optically scanned ballots have shown the better overall performance than punch cards, lever machines, and electronic voting machines.
Stefan further denigrates my analysis by dismissing residual votes as a meaningful statistic at all, “Furthermore, all indications are that the vast majority of blank ballots were really intended to be left blank.” Um… that’s not what the studies say:
Roughly one third of the residual vote, then, is pure tabulation error. The remainder is either unrecoverable ballots (i.e., people who accidentally voted twice) or blank ballots.
So yes, Stefan, residual voting rates are indeed “the primary statistical measure of the performance and accuracy of voting technologies.” That’s not my conclusion, that is CalTech/MIT’s. Furthermore, roughly one third of these residual votes represent tabulation error. And as to their relative performance:
Punch cards and electronic machines register residual voting rates for president of approximately 3 percent of all ballots cast. Paper ballots, lever machines, and optically scanned ballots produce residual voting rates of approximately 2 percent of all ballots cast, a statistically significant difference of fully one percent.
So tell me… exactly what is it that I didn’t understand about the studies on this particular point?
Let’s see… where else did I go wrong? Oh yeah…
2. The error rate of machine counting (“tabulation error rate”) is 0.56% for optical scanning machines.
I wrote “the study found the tabulation invalidation rate was .83 percent for paper and .56 percent for optical scanning”… and that is exactly what the study found. Vindication!
3. He infers from (2) that
A .5 percent invalidation rate in a gubernatorial election with over 2.8 million votes cast amounts to 14,000 erroneous votes!
I just double-checked my math, and 2.8 million times .5 percent still equals 14,000. Maybe the discrepancy is that you are using Excel, and Microsoft is known for buggy software?
Okay, maybe “erroneous” was the wrong word. But here’s the statement from which I inferred the 14,000 figure:
In a US House election with 250,000 votes, the invalidation rate of .005 for scanners amounts to 1250 votes. The tabulation errors may swing toward any of the contestants in a recount. Assuming a uniform distribution of tabulation errors, any race decided by less than .5 percent of the vote will have a non-trivial probability of being reversed in a recount.
And unlike Stefan, I didn’t place this inference out of context. It appeared right below the block-quote above. And by the way, I can’t find a reliable citation, but Paul Berendt keeps talking about the 15,000 votes that changed between the count and recount… a remarkably close match to what the historical data would predict, thank you very much.
I just want to take a moment here to clarify something about tabulation error rates, because there seemed to be some confusion in earlier comments. This metric has nothing to do with the accuracy of recounts. It is merely a measure of the accuracy of the original, “preliminary” count.
Oh, and my last so-called “conclusion”…
4. Finally, he claims that “Republicans scoff at Gregoire calling this election a tie, but statistically speaking, it is.”
Well that one is my conclusion, and I stand by it.
And this really gets to the gist of Stefan’s entire rebuttal… the fact that he stubbornly insists that a 42 vote margin out of 2.8 million is not only a statistically meaningful victory, but a mandate for sweeping change in Olympia.
Which brings us to…
My primary objective was to try to make a complex issue less confusing, and Stefan’s objective seems to be the opposite.
I explained that hand-counted paper ballots are as accurate as optically scanned ballots, and significantly more accurate than punch card ballots and electronic voting machines. I tried to explain how a machine that certifies as accurate to one in one million could possibly lose one vote in one hundred (one third the 3% residual rate for punch card ballots) due to “pure tabulation error.” What I failed to do was find data on the relative accuracy of hand counting optical and punch card ballots.
In the process I came to the conclusion that the margin of victory in this gubernatorial election was too far within the margin of error of any of the voting systems used, to discern the will of the people to any statistically meaningful degree of certainty.
And this is the only assertion that Stefan seems interested in refuting, though in doing so he saw fit to trash my entire analysis… that I don’t understand math, that I’m contributing to creating an “uninformed debate”, that my “conclusions” are wrong, and that I didn’t understand the studies that I “apparently” read.
First he dismisses me as unqualified to engage in the debate. Then he misrepresents my statements by taking them out of context. And finally, he relies on the the ultimate weapon of mass confusion: statistics.
Now I know that seems like a funny charge to be leveled by somebody whose entire argument is based on statistics. But there’s a big difference: I didn’t try to calculate any of this crap myself… after all, I read phrases like “regression analysis” and I think it has something to do with how my psychiatrist father could always make me feel like I was still thirteen.
Instead, I relied on the conclusions of carefully weighted statistical analyses of detailed historical data from CalTech/MIT, two of the most prestigious technical universities in the world.
Whereas Stefan wants us to rely on… Stefan Sharkansky and his cranky old copy of Microsoft Excel.
I could really give a shit about all his t-distributions and null hypotheses and probability whojamacallits, because even if I could do the math (and I can’t), and even if I trusted him to present honest calculations (and I most certainly don’t)… I’m an experienced enough computer programmer to understand one basic axiom: garbage in, garbage out.
The CalTech/MIT studies are based on years of historical data, whereas Stefan uses a single data point: incomplete results from WA’s 2004 gubernatorial election… results that we have no idea are even close to accurate… after all, that’s the whole damned point of the recount, isn’t it?
No, instead Stefan wants you to believe in the miraculous proposition that King County’s vote tabulations were somehow five to ten times more accurate than national historical averages… a particularly amusing notion coming from a man who has spent much of the past few weeks angrily promoting unsupported conspiracy theories about how KC Dems are stealing the election!
But Stefan’s most incredibly preposterous “calculation” is that even if Gregoire were to win the third count by 250 votes, “statistics would still favor Rossi,” a conclusion he comes to by plotting the results of all three counts.
Don’t you get it Stefan? THE FIRST COUNT WAS WRONG! That’s why we do recounts in close races. According to CalTech/MIT:
Tabulations may change from the initial count to the recount for a variety of reasons: ballots may be mishandled; machines may have difficulty reading markings; people and machines may make tabulation errors. Because recounts are used to certify the vote, greater effort is taken to arrive at the most accurate accounting of the ballots cast. The initial count of ballots, then is treated as a preliminary count, and the recount as the official.
Consistently read their blog and you’d think Stefan and his neocon cohorts apparently believe the sole purpose of our state’s recount provision is to specifically disadvantage Republicans.
But according to CalTech/MIT, the results of a recount are more accurate than the original, and thus it seems likely that the results of the second recount will be more accurate than the results of the first.
I am tempted to continue trashing Stefan’s reasoning in the manner in which he trashed mine, but instead I choose to end this on a note of conciliation. In between calling me stupid, and attempting to snow readers with bullshit numbers and technical jargon, Stefan made one, tempered statement I can agree with entirely:
I do agree with David that our current voting system is prone to inaccuracies, and that we’re not going to emerge from the hand recount with confidence that we measured the will of the voters with ball-bearing precision. I hope after this whole mess we can actually work together for meaningful election reform.
Personally, I intend to try to do something about this mess, and I hope we can all put are partisan differences aside in an attempt to restore faith in our electoral system.