[youtube]http://www.youtube.com/watch?v=LDp3sPN-5fE[/youtube]

(And there are some sixty other clips from the past week in politics posted at Hominid Views.)

A new batch of signature data for Referendum 71 has been released. The number of signatures examined is 58,493 which is 42.5% of the total signatures submitted. To date, 6,348 invalid signatures have been found, giving a raw rejection rate (uncorrected for duplicates) of 10.85%.

The invalid signatures include 5,502 that were not found in the voting rolls, 345 duplicates, and 501 that did not match the signature on file. There are also 30 “pending” signatures at various states of processing for a missing or illegible signature cards. I don’t count these among the invalid signatures.

With 345 duplicate signatures found so far, we can anticipate a final duplication rate of about 1.69%.

The V₂ estimator projects the number of valid signatures to be 121,648 giving an excess of 1,071 signatures over the 120,577 needed for the referendum to qualify for the ballot. The projected (duplicate-corrected) rejection rate is 11.65%.

A Monte Carlo analysis consisting of 10,000 simulated samples give a 95% confidence interval for valid signatures of from 121,175 to 122,415, well above the magic number. Here is the distribution of valid signatures:

There are a few “losses” in red on the left, but the overwhelming majority of outcomes in green have the referendum qualifying. In fact, the referendum failed to make the ballot in only 11 of the 10,000 simulations.

With the results to date, it is pretty clear that, come fall, we will be voting to accept or reject the “Everything but Marriage” law.

Since the Secretary of State’s office started releasing final numbers this week, it has become clear that R-71 is headed for the ballot. Short of some scandalous revelation—you know, like finding out that the numbers being released are not the final numbers—the measure should make the ballot using standard statistical inference.

(I kid the SoS with that “scandalous revelation” quip. In fact, they have done a remarkable job turning last week’s data disaster around. The data are now provided in excruciating detail and they have carefully described the meanings behind the numbers, both on the official release page and on their blog. David Ammons has been kept busy answering questions in both blog posts and the comment threads. And now Elections Director Nick Handy has a nifty R-71 FAQ.)

Back to the projections. One point that has repeatedly come up in the comment threads is that the signatures sampled so far may not reflect a random sample of all signatures. Thus, the statistical inference may be wrong.

The point is valid because the statistical methods do assume that the sampled signatures approximate a random sample. One can imagine scenarios where the error rate uncovered would change systematically with time. For example, if petition sheets were checked in chronological order of collection, the duplication rate might increase if early signers forgot they already signed, or if the pace and sloppiness of collection increased toward the end.

For R-71, we don’t know that the petition sheets are being examined in anything approaching a chronological order. The SoS FAQ states:

Signature petitions are randomly bound in volumes of 15 petition sheets per volume.

Rather than speculate on the systematic error, let’s examine some real data. The SoS office releases data that give the numbers of signatures checked and errors for each bound volume in the approximate chronological order of signature verification. As of yesterday, there were 209 completed volumes covering 35% of the total petition.

After the fold, I give a brief section on analytical details, and then show graphs of the trends over time in error rates and projected numbers of valid signatures. But first, I give an update on today’s data release.
[Read more…]

Data for August 12th are now posted. The total signatures examined in completed binders is 48,299, or 35.1% of the total signatures turned in. There have been 5,121 invalid signatures found, for an cumulative rejection rate of 10.60%.

The invalid signatures include 4,491 that were not found in the voting rolls, 242 duplicates, and 388 that did not match the signature on file. There are also 21 signatures at various states of processing for a missing signature card. (For some reason the SoS office still counts these as invalid signatures; I ignoring them.) The 242 duplicate signatures suggest an overall rate of duplication of about 1.74%.

Using this V₂ estimator, the number of valid signatures is expected to be 121,817 providing a buffer of 1,240 signatures over the 120,577 needed to qualify for the ballot. The overall rejection rate (which includes the projected total number of duplicate signatures) is about 11.53%.

To assess sampling uncertainty, I simulated petition samples, drawing numbers for each invalid signature type from a distribution that properly reflects the underlying statistical uncertainty. After 100,000 such simulations, 121,820 signatures were valid on average, and 95% of the simulations yielded 121,184 to 122,449 valid signatures. For 99.989% of the simulations the measure had enough signatures to qualify for the ballot. This number is an unbiased estimate of the probability that the measure will make it to the ballot.

The low uncertainty in qualifying for the ballot is easily seen as a picture. Here is the resulting distribution of signatures relative to the number required to qualify. Red bars (to the left of the vertical dashed line) means the measure is stopped. Green (on the right) means it qualified.

The data released by the SoS office, so far, suggests that the measure is likely to qualify with an excess of about 1,240 signatures. This projection does not account for potential errors other than sampling error.

This afternoon, the Secretary of State’s office released R-71 data in a brand new format. Apparently, the data now reflect the actual numbers of duplicates, rejected signatures, and accepted signatures.

There are some noticable differences over the previous data releases. As David Ammons explains it:

The error rate is lower than the daily and cumulative numbers that had been previously reported, because the earlier numbers included many signatures that still were being reviewed by master checkers. A prime example is that hundreds of signatures were not initially found on voter rolls by the checker, but a later check by the veteran master checkers did make a match.

He also points out:

State Elections Director Nick Handy said it remains “too close to call” whether R-71 will make the ballot, and cautioned against making assumptions based on the current error rate.

Handy is incorrect in one respect. Given a proper statistical estimate of the duplicate error rate in the total sample, and a proper projection of the other invalid signatures, we can estimate a total number of valid signatures and offer some statistical certainty about the number. (Of course, this assumes we are given the correct numbers in the first place….)

The statistical certainty only accounts for the fact that we have only a sample of the total petition evaluated so far. It cannot account for non-sampling error, biases, correlations among batches of pages, etc. Of course such error may be ignorable. I’ll get back to that issue in a later post.

The total number of signatures that have been completed is 33,214, which is just under a quarter of the total petition. There have been 3,450 invalid signatures found, for an uncorrected rejection rate of 10.39%. This rate doesn’t mean much because it doesn’t include the rate of duplicate signatures in the total petition.

The invalid signatures include 3,117 that were not found in the voting rolls, 130 duplicates, and 203 that did not match the signature on file. There are also 12 pending signatures in which a better signature card is needed. (Oddly enough, the data table includes the 12 pending signatures in the rejected totals; I suspect this is an error, albiet a minor one).

The 130 duplicated signatures from a sample of 33,314 suggests a duplication rate on the entire petition of about 1.62%.

Using the V₂ V estimator, the number of valid signatures is expected to be 121,103, thus squeaking by with 526 signature over the 120,577 needed to qualify for the ballot. (The sampling error is many times smaller than the 526 margin.) The expected total rejection rate is 12.05%.

The bottom line: Unless new errors are found in the processing or counting, or some large, systematic increase in the error rate is seen for the remaining 76% of the signatures, we should expect to see R-71 on the ballot this fall.

Update: I just noticed I used the V estimator, not the V₂ estimator. The V estimator is slightly biased toward too few valid signatures, so the qualitative results are the same.

Join us tonight for a refreshing drink over some stimulating political conversation at the Seattle chapter of Drinking Liberally. The festivities take place at the Montlake Ale House, 2307 24th Avenue E. beginning at 8:00 pm.

Hey…bring your resume along. I hear a Guvmint recruiter will show up, and take applications to serve on a special new Obama panel….

[youtube]http://www.youtube.com/watch?v=SSv7Va8enjc[/youtube]

Not in Seattle? The Drinking Liberally web site has dates and times for 335 other chapters of Drinking Liberally for you to get lost at.

Part I
[youtube]http://www.youtube.com/watch?v=m5HgfwPtxLw[/youtube]

Part II
[youtube]http://www.youtube.com/watch?v=R0gen_HoxrM[/youtube]

The real implication of this story is that we’ve most likely uncovered another illegal felon vote for Dino Rossi.

Sheesh…will it ever end?

[youtube]http://www.youtube.com/watch?v=m9td6dk_lzw[/youtube]

(There are some eighty other clips from the past week in politics posted at Hominid Views.)

Today’s batch of R-71 data has been released. The total signatures examined is now 35,262 (about 25.6% of the total).

Today, there were 4,721 invalid signatures found that were rejected on the first pass. Some of these decisions may be reversed by the “Master Checker.” The preliminary invalid signatures include 4,110 that are not found in the voting rolls, 146 duplicates, and 465 that did not match the signature on file. There are also 101 signatures at various states of processing for a missing signature card.

If all 146 duplicates hold, there is a duplicate rate of about 1.56% for the petition. But some of the duplicates may be deemed valid signatures. What we know now is that 31,199 signatures have been accepted and 4063 rejected. We don’t know exactly what category each rejected signature is classified under, but the daily totals give us some idea. (See Dave Ammons post and the comment thread for more details about today’s surprising revelations from the SoS office.)

Using the best numbers available, I’ve run three different scenarios, making different assumptions about invalid and duplicated signatures. The raw numbers suggest 658 previously rejected signatures have been “saved” by the “Master Checker”. In all three scenarios, I first deduct the 465 from the total number of mismatched signatures (Ammons points out this is the most likely way a rejected signature is “saved”). The remaining 193 are deducted from duplicates and signers not found on the voter rolls (“not founds”) as follows:

1. All 146 duplicates are assumed valid (yeah…this is pretty unrealistic), and deduct the remaining 47 from the “not founds”.
2. All 193 come from the “not founds”.
3. I split them, removing 97 from the “not founds” and 96 from the duplicates.

I use the V₂* estimator to find the expected number of valid signatures. Here are the results:

1. There are 121,824 valid signatures giving an excess of 1,247 signatures over the 120,577 needed to qualify for the ballot. The overall rejection rate is about 11.52%. This is a 0% duplicate rate.
2. Now we expected 119,598 valid signatures, leaving a shortfall of 979 signatures to qualify. The overall rejection rate is about 13.14%. The duplicate rate for the petition of 1.62%.
3. We expect 121,062 valid signatures, giving an excess of 485 signatures. The overall rejection rate is about 12.08%, and the duplicate rate for the petition is 0.55%.

The truth lies somewhere between scenario 2 and 3. As I found earlier today, the fate of R-71 depends almost entirely on the actual number of duplicates found. Until we get the real numbers, both proponents and opponents will be on pins and needles.

Updated twice

The 6th batch of R-71 data was delivered this morning. The SoS office is pooling numbers from the morning and evening counts, but I’ll treat them separately just so that we can look for big swings in the “semi-batches”. Also, the SoS totals don’t seem to match the daily totals today, but what’s a signature-validation watcher to do? I’ll use the daily counts.

The total signatures examined is now 29,937 (about 21.7% of the total). There have been 3,968 invalid signatures found, for an cumulative rejection rate of 13.25%. The invalid signatures include 3,506 that are not found in the voting rolls, 113 duplicates, and 349 that did not match the signature on file. There are also 89 signatures at various states of processing for a missing signature card (by now, some may be good and some rejected–I’ll treat them all as good until I learn otherwise).

The 113 duplicates suggest a duplication rate of about 1.74% in the total petition. Using the V₂* estimator, the number of valid signatures is expected to be 117,049 leaving a shortfall of 3,528 signatures from the 120,577 needed to qualify for the ballot. The overall rejection rate should be about 14.99%.

Here is a snapshot of the trends:

Still no unexpected deviations as the counting continues. R-71 continues on track for failure.

Updated

Dave Ammons has posted some new information about the signature validation process and the numbers that have been posted by the SoS.

Elections Division has decided that a more accurate cumulative error or rejection rate (currently 11.63 percent) should reflect the sizable number of signatures that have been going from the rejected pile to the accepted stack after a master checker reviews the checker’s decision to reject. That is 409 so far. As previously discussed, also nearly 100 signatures that have been set aside because there was no voter signature on file will be shifted over to the “accepted” stack once the counties send the person’s electronic signature.

In other words, the SoS office wasn’t giving us the number of invalid signatures. What they were giving as “invalid signatures”, were only invalid in a first-stage of checking. A second-stage check has resulted in some signatures being considered valid again. All I can say is unbefuckinglievable!

I’ve already been treating the “missing signature” counts as valid signatures. But without knowing the ACTUAL number of invalid signatures by category, it is difficult to project whether I-71 will make the ballot.

Update 2:

I’ve run three scenarios based on incomplete numbers posted at the SoS site.

We know there were 409 signatures that failed at the first checking phase that were subsequently accepted in phase 2. We don’t know which of the bad batches they came from (signatures not found on voter rolls, duplicate signatures, or signature mismatches) .

If I had to guess, I think the majority came from the signature mismatch pile. But there were only 349 signature mismatches, so we have 60 that were either considered duplicates or not found on the voter rolls. Subsequently, they were considered not duplicates or found on the voter rolls. Both seem equally likely (or unlikely) to me, but I don’t really know what the “Master Checker” does.

So here is what I did. First, I assumed 349 of the 409 came from the signature mismatches. The remaining 60 I dealt with in three ways:

1. Assume there are 60 fewer duplicates.
2. Assume 60 signatures were subsequently found in the voter rolls
3. Assume a 30/30 split between duplicates and “found”

Remember, 120,577 signatures are needed to qualify. Here is what happens using the V₂* estimator:

1. The measure makes the ballot with 120,519 120,591 signatures. The estimated duplicate rate for the entire petition is 0.78%, and the overall rejection rate is 12.42%.
2. The measure fails to make the ballot with 119,375 signatures. The estimated total duplicate rate is 1.66% and the total rejection rate is 13.30%.
3. The measure fails to make the ballot with 119,983 signatures. The estimated total duplicate rate is 1.22% and the total rejection rate is 12.86%.

Obviously, the biggest determinant of R-71’s fate is the number of duplicate signatures in the phase one check that are subsequently accepted in phase two. With any luck the actual number of signatures rejected in each category will be released.

Today was not a good day for R-71 supporters. With the 5th batch of data, the total of signatures examined is now 27,288 (almost 20% complete), with 3,695 invalid signatures found.

The number of duplicates has risen to 90. (Consider that there were only 7 found by the end of the first day, and you can see how the duplicate count snowballs as the sample grows.) Ninety duplicates in a sample of 27,288 reflects a duplicate rate of 1.67% in the total sample.

Using the methods described here, a good estimate of the number of valid signatures is 118,184, a shortfall of 2,393 signatures of the 120,577 needed to qualify the referendum for the ballot. The total rejection rate should be about 14.2%.

Here is the trend since the start of signature verification:

This graph shows the daily estimates of valid signatures (red) and the number needed to qualify (blue). What is most salient here is that the estimate of valid signatures is extremely stable from day-to-day. If the trend continues, R-71 fails.

Starting today, signatures will be checked in two shifts, with an update in the morning and the afternoon.

The full numbers from Wednesday’s counts were released this morning by the Secretary of State’s (SoS) office. They’ve driven another nail in the coffin of R-71.

A total of 23,457 signatures have been checks, which is 17% of the total. Overall 3,054 invalid signatures have been found and eliminated, including 68 duplicates, 2,764 individuals not on the voting rolls, 221 signature mismatches, and 69 signatures for which the corresponding signature is missing.

The cumulative error rate is 13.3%, if the signatures with missing signature cards (hereafter “missing”) are thrown out, or 13.0% if the missing signatures are fully counted. As Goldy has explained, the cumulative error rate for the sample is misleadingly low. This is because duplicates are exponentially underrepresented as the sample size goes down.

Given the number of duplicates found in this sample, the best estimate is that is about 1.7% of signatures are duplicates on the petition. That gives an estimated total rejection rate of 14.7% (treating all “missing” signatures as valid). A rejection rate over 12.4% keeps R-71 off the ballot.

Rather than focus on percentages, we can use the number of good and bad signatures to estimate the expected number of valid signatures. This figure shows the daily estimated number of valid signatures on the petition (red line) and the number of signatures required (blue line) for the measure to make the ballot:

These estimates are conservative because I am assuming all “missing” signatures will be treated as valid. (I’ve changed my methods a bit since yesterday—a journey through the methodological details begins below the fold).

The important things to notice here are:

• The estimates are stable rather than bouncing around from day to day. This means that there is little evidence for non-sampling error. Such errors can arise if batches of petitions showed widely different error rates (more generally, from non-independence among signatures on petitions and in batches of petitions).
• The 95% confidence intervals are now so small that sampling error is no longer relevant. If God plays dice, she clearly doesn’t want R-71 on the ballot.

The trends, so far, indicate that, short of a miracle, this measure will not qualify for the ballot.

At this point, I am going to totally geek-out and discuss methodological stuff. If you’re interested, venture below the fold.

[Read more…]

Imagine ten years ago as Bill Clinton was approaching the end of his second term as President. Whodathunk that ten years hence, Hollywood superhero Arnold Schwarzenegger would become an emasculated politician, and Bill Clinton would become a real-life superhero:

[youtube]http://www.youtube.com/watch?v=Pv7pIBMYGfw[/youtube]

Join us tonight at the Seattle chapter of Drinking Liberally for an evening of politics under the influence. The festivities take place at the Montlake Ale House, 2307 24th Avenue E. beginning at 8:00 pm.

We may have a little entertainment with discussions about the history of Kenya this evening. Or, perhaps, we should be looking at Australia?

[youtube]http://www.youtube.com/watch?v=FsqodzZ4Ssg[/youtube]

Not in Seattle? The Drinking Liberally web site has dates and times for 335 other chapters of Drinking Liberally for you to get lost at.