Yesterday I took a break from my all-too-frequent analyses of the R-71 signature counts. I didn’t even look at the numbers until this morning. When I did look, a Spock-esque twitch afflicted my left eyebrow. “Curious”, I though. “But maybe it’s just a one-time fluke….”

The analysis of yesterday’s data showed the probability of NOT making the ballet increased from a nearly impossible 0.04% to an almost-interesting 0.91%. In fact, this slow increase in the probability of not qualifying has continued a trend begun after 13 August.

Well, if you like that result, hold onto your sou’wester, because today’s result will blow you away. I’ll present the results in three parts. First, the basic results for today, then we’ll explore the trends in the daily data dumps. Finally (and below the fold) we’ll look at the micro-level volume data to divine what this trend suggests.

Today’s R-71 data release has the signature count up to 79,195, (about 57.5% of the total). There have been 9,208 invalid signatures found, for a cumulative crude (non-duplicate-corrected) rejection rate of 11.63%.

The invalid signatures include 7,805 that were not found in the voting rolls, 703 duplicate signatures, and 700 signatures that mismatched the signature on file. There are also 38 signatures “pending”; I’ve ignored them in the analyses. The 703 duplicate signatures suggest a final duplication rate of about 1.90% for the petition. This continues the trend we’ve seen this week of the projected duplicate rate growing faster than the mathematical predictions under the assumption of random sampling.

Using the V₂ estimator, the number of valid signatures is now expected to be 120,777 leaving a thin surplus of only 200 signatures over the 120,577 needed to qualify for the ballot. From the cumulative data to date, the overall rejection rate is projected to be 12.28%.

A Monte Carlo analyses consisting of 100,000 simulated petition samples suggests that the measure has an 80.48% probability of qualifying for the ballot, assuming the only “error” is statistical sampling error.

Here is the distribution of valid signatures relative to the number required to qualify.

The red bars on the left show the times R-71 failed to qualify among the 100,000 simulations; green bars show the counts of signatures in which the measure qualified. Compare this to the results from just two days ago. Quite a difference!

Let’s examine the history since the SoS office started releasing accurate data a week and a half ago:

The red line shows the number of signatures needed to qualify, and the blue symbols show the daily projections of valid signatures, surrounded by 95% confidence intervals.

Clearly, since the 13th of August, the projected number of signatures has declined–and, as of today, declined more than we could expect by chance alone. Something is going on.

Tomorrow will be interesting…if the trend continues, success of the measure may dip below a probability of 50%.

The analyses I’ve done here are based on two assumptions: (1) that the signatures evaluated so far are just like signatures that remain to be evaluated, and (2) that the signature validation process is “stable” (the people validating signatures are not changing their standards over time). Today we see some pretty good evidence that one (or both) of these assumptions is (are) violated.

The supporters of R-71 will, no doubt, focus on the second assumption. If the measure fails, Secretary of State Sam Reed will likely take much abuse from fringe homophobes for “personally pushing a homosexual agenda.” To me, the simplest explanation is that the volumes being examined in serial order are chronologically correlated with the signature collection order. ( I don’t know if this is true; but, I cannot rule it out either.)

My thinking is that later-collected signatures (and therefore, later volumes) should have a higher duplication rate, just because there is an increasing chance with time early signers forgot whether or not they signed earlier. Additionally, with the last push of getting as many signatures as possible with an approaching deadline, it seems plausible that errors would increase. I’m thinking errors like collecting more out-of-state signatures, underage signatures, and signatures from people not active on the voter rolls.

Below the fold, I examine the fine-level data to see just what types of errors are increasing as the process proceeds. If you are still interested, click through…

[Read more…]

Today is election day, so make those final selections and turn in your ballot. And the join us tonight for some politics under the influence at the Seattle chapter of Drinking Liberally. The festivities take place at the Montlake Ale House, 2307 24th Avenue E. beginning at 8:00 pm.

Remember, folks, guns and alcohol don’t mix. So check your sidearm with the kids before you come on down.

[youtube]http://www.youtube.com/watch?v=EWg9CqgwZN4[/youtube]

Not in Seattle? The Drinking Liberally web site has dates and times for 335 other chapters of Drinking Liberally for you to shoot for.

Today’s release of R-71 data has the process nearly half-complete. Now 65,531, (47.6%) signatures have been examined in completed volumes. There have been 7,201 invalid signatures found, for an uncorrected rejection rate of 10.99%.

The invalid signatures include 6,165 that were not found in the voting rolls, 470 duplicates, and 566 that did not match the signature on file. There are also 24 pending signatures that I’ve not counted either way.

The 470 duplicates suggest that the overall rate of duplicate signatures for the petition will be about 1.84%. This is slightly higher than what we saw last week, from Tuesday through Friday, 1.62%, 1.74%, 1.73%, and 1.69% respectively.

Using the V₂ estimator, the number of valid signatures is expected to be 121,475 exceeding by 898 the 120,577 needed to qualify for the ballot. The overall (i.e. duplicate-corrected) rejection rate is about 11.78%. (Last week we saw 11.61%, 11.53%, 11.54%, 11,65%.)

A Monte Carlo analysis of 100,000 simulated petitions gives this distribution of projected valid signatures for the petition:

In only 34 of the 100,000 simulations did the measure fail to qualify for the ballot (those red bars on the left). The median number of signatures was 121,478 with a 95% confidence interval between 120,951 and 122,003.

As we saw last week…the measure has almost no chance of being rejected.

After the fold, I provide a rather dry analysis by small batches of signature pages for the R-71 obsessives and stats wonks….

[Read more…]

CBS News examines the similarities and differences among the five different health insurance reform drafts in Congress:

[youtube]http://www.youtube.com/watch?v=irwmdDMZVP0[/youtube]

Now examine this mini-debate between Sen. Orrin Hatch (R-UT) and Sen. Arlen Specter (D-PA):

[youtube]http://www.youtube.com/watch?v=31-1SFncpIw&feature=player_profilepage[/youtube]

This is remarkable. First, Hatch refuses to take sides in the “Obama Death Panel” canard! Hatch then lets loose with a series of blatant lies (“government take-over”, “cut medicare”, “eight in ten of Americans really want their health insurance coverage and don’t want to lose it”, “nameless, faceless bureaucrats setting healthcare”, “single payer”, “government plan”).

Much of what Hatch is saying isn’t simply “alternative interpretations” of facts. They are fabrications that are so discordant with the actual provisions in the drafts that this can only be intentional deceit.

This is what Republicans are reduced to? Putting their senior statesmen on mainstream TV to tell barefaced lies to Americans? I mean, there is a deep tradition of politicians “spinning” facts to their cause. But discarding facts altogether? We tolerate unashamed lying from pundits and political operatives…but a U.S. Senator?

Or perhaps Republicans have become so enmeshed in their lies that they no longer recognize objective truth. This goes beyond deplorable; now it’s just pitiable.

[youtube]http://www.youtube.com/watch?v=LDp3sPN-5fE[/youtube]

(And there are some sixty other clips from the past week in politics posted at Hominid Views.)

A new batch of signature data for Referendum 71 has been released. The number of signatures examined is 58,493 which is 42.5% of the total signatures submitted. To date, 6,348 invalid signatures have been found, giving a raw rejection rate (uncorrected for duplicates) of 10.85%.

The invalid signatures include 5,502 that were not found in the voting rolls, 345 duplicates, and 501 that did not match the signature on file. There are also 30 “pending” signatures at various states of processing for a missing or illegible signature cards. I don’t count these among the invalid signatures.

With 345 duplicate signatures found so far, we can anticipate a final duplication rate of about 1.69%.

The V₂ estimator projects the number of valid signatures to be 121,648 giving an excess of 1,071 signatures over the 120,577 needed for the referendum to qualify for the ballot. The projected (duplicate-corrected) rejection rate is 11.65%.

A Monte Carlo analysis consisting of 10,000 simulated samples give a 95% confidence interval for valid signatures of from 121,175 to 122,415, well above the magic number. Here is the distribution of valid signatures:

There are a few “losses” in red on the left, but the overwhelming majority of outcomes in green have the referendum qualifying. In fact, the referendum failed to make the ballot in only 11 of the 10,000 simulations.

With the results to date, it is pretty clear that, come fall, we will be voting to accept or reject the “Everything but Marriage” law.

Since the Secretary of State’s office started releasing final numbers this week, it has become clear that R-71 is headed for the ballot. Short of some scandalous revelation—you know, like finding out that the numbers being released are not the final numbers—the measure should make the ballot using standard statistical inference.

(I kid the SoS with that “scandalous revelation” quip. In fact, they have done a remarkable job turning last week’s data disaster around. The data are now provided in excruciating detail and they have carefully described the meanings behind the numbers, both on the official release page and on their blog. David Ammons has been kept busy answering questions in both blog posts and the comment threads. And now Elections Director Nick Handy has a nifty R-71 FAQ.)

Back to the projections. One point that has repeatedly come up in the comment threads is that the signatures sampled so far may not reflect a random sample of all signatures. Thus, the statistical inference may be wrong.

The point is valid because the statistical methods do assume that the sampled signatures approximate a random sample. One can imagine scenarios where the error rate uncovered would change systematically with time. For example, if petition sheets were checked in chronological order of collection, the duplication rate might increase if early signers forgot they already signed, or if the pace and sloppiness of collection increased toward the end.

For R-71, we don’t know that the petition sheets are being examined in anything approaching a chronological order. The SoS FAQ states:

Signature petitions are randomly bound in volumes of 15 petition sheets per volume.

Rather than speculate on the systematic error, let’s examine some real data. The SoS office releases data that give the numbers of signatures checked and errors for each bound volume in the approximate chronological order of signature verification. As of yesterday, there were 209 completed volumes covering 35% of the total petition.

After the fold, I give a brief section on analytical details, and then show graphs of the trends over time in error rates and projected numbers of valid signatures. But first, I give an update on today’s data release.
[Read more…]

Data for August 12th are now posted. The total signatures examined in completed binders is 48,299, or 35.1% of the total signatures turned in. There have been 5,121 invalid signatures found, for an cumulative rejection rate of 10.60%.

The invalid signatures include 4,491 that were not found in the voting rolls, 242 duplicates, and 388 that did not match the signature on file. There are also 21 signatures at various states of processing for a missing signature card. (For some reason the SoS office still counts these as invalid signatures; I ignoring them.) The 242 duplicate signatures suggest an overall rate of duplication of about 1.74%.

Using this V₂ estimator, the number of valid signatures is expected to be 121,817 providing a buffer of 1,240 signatures over the 120,577 needed to qualify for the ballot. The overall rejection rate (which includes the projected total number of duplicate signatures) is about 11.53%.

To assess sampling uncertainty, I simulated petition samples, drawing numbers for each invalid signature type from a distribution that properly reflects the underlying statistical uncertainty. After 100,000 such simulations, 121,820 signatures were valid on average, and 95% of the simulations yielded 121,184 to 122,449 valid signatures. For 99.989% of the simulations the measure had enough signatures to qualify for the ballot. This number is an unbiased estimate of the probability that the measure will make it to the ballot.

The low uncertainty in qualifying for the ballot is easily seen as a picture. Here is the resulting distribution of signatures relative to the number required to qualify. Red bars (to the left of the vertical dashed line) means the measure is stopped. Green (on the right) means it qualified.

The data released by the SoS office, so far, suggests that the measure is likely to qualify with an excess of about 1,240 signatures. This projection does not account for potential errors other than sampling error.

This afternoon, the Secretary of State’s office released R-71 data in a brand new format. Apparently, the data now reflect the actual numbers of duplicates, rejected signatures, and accepted signatures.

There are some noticable differences over the previous data releases. As David Ammons explains it:

The error rate is lower than the daily and cumulative numbers that had been previously reported, because the earlier numbers included many signatures that still were being reviewed by master checkers. A prime example is that hundreds of signatures were not initially found on voter rolls by the checker, but a later check by the veteran master checkers did make a match.

He also points out:

State Elections Director Nick Handy said it remains “too close to call” whether R-71 will make the ballot, and cautioned against making assumptions based on the current error rate.

Handy is incorrect in one respect. Given a proper statistical estimate of the duplicate error rate in the total sample, and a proper projection of the other invalid signatures, we can estimate a total number of valid signatures and offer some statistical certainty about the number. (Of course, this assumes we are given the correct numbers in the first place….)

The statistical certainty only accounts for the fact that we have only a sample of the total petition evaluated so far. It cannot account for non-sampling error, biases, correlations among batches of pages, etc. Of course such error may be ignorable. I’ll get back to that issue in a later post.

The total number of signatures that have been completed is 33,214, which is just under a quarter of the total petition. There have been 3,450 invalid signatures found, for an uncorrected rejection rate of 10.39%. This rate doesn’t mean much because it doesn’t include the rate of duplicate signatures in the total petition.

The invalid signatures include 3,117 that were not found in the voting rolls, 130 duplicates, and 203 that did not match the signature on file. There are also 12 pending signatures in which a better signature card is needed. (Oddly enough, the data table includes the 12 pending signatures in the rejected totals; I suspect this is an error, albiet a minor one).

The 130 duplicated signatures from a sample of 33,314 suggests a duplication rate on the entire petition of about 1.62%.

Using the V₂ V estimator, the number of valid signatures is expected to be 121,103, thus squeaking by with 526 signature over the 120,577 needed to qualify for the ballot. (The sampling error is many times smaller than the 526 margin.) The expected total rejection rate is 12.05%.

The bottom line: Unless new errors are found in the processing or counting, or some large, systematic increase in the error rate is seen for the remaining 76% of the signatures, we should expect to see R-71 on the ballot this fall.

Update: I just noticed I used the V estimator, not the V₂ estimator. The V estimator is slightly biased toward too few valid signatures, so the qualitative results are the same.

Join us tonight for a refreshing drink over some stimulating political conversation at the Seattle chapter of Drinking Liberally. The festivities take place at the Montlake Ale House, 2307 24th Avenue E. beginning at 8:00 pm.

Hey…bring your resume along. I hear a Guvmint recruiter will show up, and take applications to serve on a special new Obama panel….

[youtube]http://www.youtube.com/watch?v=SSv7Va8enjc[/youtube]

Not in Seattle? The Drinking Liberally web site has dates and times for 335 other chapters of Drinking Liberally for you to get lost at.

Part I
[youtube]http://www.youtube.com/watch?v=m5HgfwPtxLw[/youtube]

Part II
[youtube]http://www.youtube.com/watch?v=R0gen_HoxrM[/youtube]

The real implication of this story is that we’ve most likely uncovered another illegal felon vote for Dino Rossi.

Sheesh…will it ever end?

[youtube]http://www.youtube.com/watch?v=m9td6dk_lzw[/youtube]

(There are some eighty other clips from the past week in politics posted at Hominid Views.)

Today’s batch of R-71 data has been released. The total signatures examined is now 35,262 (about 25.6% of the total).

Today, there were 4,721 invalid signatures found that were rejected on the first pass. Some of these decisions may be reversed by the “Master Checker.” The preliminary invalid signatures include 4,110 that are not found in the voting rolls, 146 duplicates, and 465 that did not match the signature on file. There are also 101 signatures at various states of processing for a missing signature card.

If all 146 duplicates hold, there is a duplicate rate of about 1.56% for the petition. But some of the duplicates may be deemed valid signatures. What we know now is that 31,199 signatures have been accepted and 4063 rejected. We don’t know exactly what category each rejected signature is classified under, but the daily totals give us some idea. (See Dave Ammons post and the comment thread for more details about today’s surprising revelations from the SoS office.)

Using the best numbers available, I’ve run three different scenarios, making different assumptions about invalid and duplicated signatures. The raw numbers suggest 658 previously rejected signatures have been “saved” by the “Master Checker”. In all three scenarios, I first deduct the 465 from the total number of mismatched signatures (Ammons points out this is the most likely way a rejected signature is “saved”). The remaining 193 are deducted from duplicates and signers not found on the voter rolls (“not founds”) as follows:

1. All 146 duplicates are assumed valid (yeah…this is pretty unrealistic), and deduct the remaining 47 from the “not founds”.
2. All 193 come from the “not founds”.
3. I split them, removing 97 from the “not founds” and 96 from the duplicates.

I use the V₂* estimator to find the expected number of valid signatures. Here are the results:

1. There are 121,824 valid signatures giving an excess of 1,247 signatures over the 120,577 needed to qualify for the ballot. The overall rejection rate is about 11.52%. This is a 0% duplicate rate.
2. Now we expected 119,598 valid signatures, leaving a shortfall of 979 signatures to qualify. The overall rejection rate is about 13.14%. The duplicate rate for the petition of 1.62%.
3. We expect 121,062 valid signatures, giving an excess of 485 signatures. The overall rejection rate is about 12.08%, and the duplicate rate for the petition is 0.55%.

The truth lies somewhere between scenario 2 and 3. As I found earlier today, the fate of R-71 depends almost entirely on the actual number of duplicates found. Until we get the real numbers, both proponents and opponents will be on pins and needles.

Updated twice

The 6th batch of R-71 data was delivered this morning. The SoS office is pooling numbers from the morning and evening counts, but I’ll treat them separately just so that we can look for big swings in the “semi-batches”. Also, the SoS totals don’t seem to match the daily totals today, but what’s a signature-validation watcher to do? I’ll use the daily counts.

The total signatures examined is now 29,937 (about 21.7% of the total). There have been 3,968 invalid signatures found, for an cumulative rejection rate of 13.25%. The invalid signatures include 3,506 that are not found in the voting rolls, 113 duplicates, and 349 that did not match the signature on file. There are also 89 signatures at various states of processing for a missing signature card (by now, some may be good and some rejected–I’ll treat them all as good until I learn otherwise).

The 113 duplicates suggest a duplication rate of about 1.74% in the total petition. Using the V₂* estimator, the number of valid signatures is expected to be 117,049 leaving a shortfall of 3,528 signatures from the 120,577 needed to qualify for the ballot. The overall rejection rate should be about 14.99%.

Here is a snapshot of the trends:

Still no unexpected deviations as the counting continues. R-71 continues on track for failure.

Updated

Dave Ammons has posted some new information about the signature validation process and the numbers that have been posted by the SoS.

Elections Division has decided that a more accurate cumulative error or rejection rate (currently 11.63 percent) should reflect the sizable number of signatures that have been going from the rejected pile to the accepted stack after a master checker reviews the checker’s decision to reject. That is 409 so far. As previously discussed, also nearly 100 signatures that have been set aside because there was no voter signature on file will be shifted over to the “accepted” stack once the counties send the person’s electronic signature.

In other words, the SoS office wasn’t giving us the number of invalid signatures. What they were giving as “invalid signatures”, were only invalid in a first-stage of checking. A second-stage check has resulted in some signatures being considered valid again. All I can say is unbefuckinglievable!

I’ve already been treating the “missing signature” counts as valid signatures. But without knowing the ACTUAL number of invalid signatures by category, it is difficult to project whether I-71 will make the ballot.

Update 2:

I’ve run three scenarios based on incomplete numbers posted at the SoS site.

We know there were 409 signatures that failed at the first checking phase that were subsequently accepted in phase 2. We don’t know which of the bad batches they came from (signatures not found on voter rolls, duplicate signatures, or signature mismatches) .

If I had to guess, I think the majority came from the signature mismatch pile. But there were only 349 signature mismatches, so we have 60 that were either considered duplicates or not found on the voter rolls. Subsequently, they were considered not duplicates or found on the voter rolls. Both seem equally likely (or unlikely) to me, but I don’t really know what the “Master Checker” does.

So here is what I did. First, I assumed 349 of the 409 came from the signature mismatches. The remaining 60 I dealt with in three ways:

1. Assume there are 60 fewer duplicates.
2. Assume 60 signatures were subsequently found in the voter rolls
3. Assume a 30/30 split between duplicates and “found”

Remember, 120,577 signatures are needed to qualify. Here is what happens using the V₂* estimator:

1. The measure makes the ballot with 120,519 120,591 signatures. The estimated duplicate rate for the entire petition is 0.78%, and the overall rejection rate is 12.42%.
2. The measure fails to make the ballot with 119,375 signatures. The estimated total duplicate rate is 1.66% and the total rejection rate is 13.30%.
3. The measure fails to make the ballot with 119,983 signatures. The estimated total duplicate rate is 1.22% and the total rejection rate is 12.86%.

Obviously, the biggest determinant of R-71’s fate is the number of duplicate signatures in the phase one check that are subsequently accepted in phase two. With any luck the actual number of signatures rejected in each category will be released.