An End to "Faith-Based" Voting: Simple, Powerful, UBS Method (Universal Ballot Sampling) Compared to HR 811 Audit Standards
Report Updated June 17, 2007
Original PRESS RELEASE
September 8, 2006
For Immediate Release
Attn: Political assignments
Download a PDF of this original press release here
Download a PDF of the original full report based on HR 550 audits here.
NEW! Updated version, "Universal Ballot Sampling" (UBS).
Calculations and report revised to test audit standards proposed in H.R. 811.
Contacts:
Jonathan Simon, Election Defense Alliance
jonathan[at]electiondefensealliance[dot]org
617.538.6012 Bruce O'Dell, Election Defense Alliance
bodell[at]electiondefensealliance[dot]org
612.309.1330
AN END TO "FAITH-BASED" VOTING:
Computer Security and Statistical Analysts Describe a Simple and Powerful alternative
Summary
Today the Election Defense Alliance released a report describing the practical implementation details of a simple, unimpeachable method for ensuring the accuracy of electronic voting systems by a public handcount of paper ballot records. This “Universal Precinct-Based Handcount Sample” (UPS) is a simple, feasible method of hand-counting a sample of paper ballot records in-precinct, on election night, by citizens themselves. It not only returns oversight of elections to the American people, where it rightfully belongs, the UPS is also far more accurate than alternative election audit proposals—where only a few percent of precincts are hand-counted, often in secret, and always after the fact.
The simple, practical UPS validation approach detects fraud or error from any source altering the electronic tally by as little as one percent (1%) with a minimum ninety-nine percent (99%) level of confidence.
In our current political climate, any challenge to a corrupt election must be timely and have very strong justification, or candidates risk being labeled “sore losers” and accompanying ridicule. The UPS validation, by virtue of being accurate to such a high degree of confidence, enables any candidate of any party to contest any Outcome-altering problems with the electronic tally. And since the UPS hand count is done in-precinct on election night, its findings would be available on election night, enabling candidates in federal or statewide elections to challenge a corrupted tally before the election’s outcome becomes a foregone conclusion in the mind of the public, and before the results are officially certified.
The report describes the specific means of effectively conducting a public hand count of 10% of the paper ballot records in 100% of the precincts in federal and statewide races. The UPS is to be conducted “in-precinct” on election night, by citizens representing all concerned political parties, and open to general public observation. Because it is conducted in-precinct, the UPS avoids the difficult task of protecting the chain of custody of paper ballot records in 180,000 U.S. precincts. In fact, all the alternative after-the-fact “spot-audit” schemes (such as HR 550, often referred to as the Holt bill) impose this monumental burden – since in all those protocols, all precincts must safeguard ballot records until just a few percent are “randomly chosen” some time after the election. Integrity of the chain of custody will be especially suspect, of course, in just those suspect elections which such audits are proposed to safeguard. Since a 10% hand-count sample would be drawn in 100% of precincts on election night, the UPS also eases the transition to decentralized, citizen-monitored hand-count verifications of elections, placing responsibility for the integrity of the vote count in the hands of the American people, where it rightfully belongs.
Most importantly, the UPS is inherently resistant to manipulation. The report describes how any attempt to systematically manipulate the UPS audit would be extraordinarily difficult to conduct and to conceal. Not only would it require a very large number of participants, any effort to skew the 10% paper hand count in favor of a candidate would be very likely to increase the overall discrepancy, not decrease it. The UPS provides a simple, effective, and vastly more powerful alternative for election validation than does the proposed HR 550 audit, and all such “spot-audit” proposals. The UPS provides a decentralized hand count, reduces chain of custody concerns and provides citizens and candidates a clear and timely warning of fraud or error. Therefore Election Defense Alliance recommends UPS as an alternative to the HR 550 audit.
In order to restore and maintain citizen trust in the integrity of American democracy, it is critical that wherever electronic vote tallying is performed, paper ballot records must always be produced and must always be checked by the best possible “security mechanism” – the American people, working together in public.
Background
Despite credible reports of widespread error-prone programming and severe, inherent security vulnerabilities, millions of votes in America are now tallied by machines that lack any independent means of verifying that they tallied the vote accurately. (For example see the recent Brennan Justice Center Report.) Even where such means exist, they are most often not employed, or not employed properly. (A well-known but by no means isolated example is the Ohio 2004 “recount,” where precincts were cherry-picked rather than being chosen at random, as required by law, and where vendors introduced “cheat sheets” to avoid triggering full hand recounts, the result being that of Ohio’s 88 counties, only one proceeded to a full recount.)
In response to this unacceptable risk, Rep. Rush Holt (D-NJ) recently re-introduced HR 550, “The Voter Confidence and Increased Accessibility Act of 2005”—a pending bill to require creation and auditing of a fraction of the paper record of all electronic votes cast in federal elections. According to Representative Holt, HR 550 has received “bipartisan endorsement from one-third of the members of the House of Representatives, and has been endorsed by good-government groups as the ‘gold standard’ in [election] verifiability legislation.” (See
A study released August 16, 2006, sponsored by the Election Defense Alliance, revealed that, despite its good intentions, the proposed election audit mechanism in HR 550 – far from protecting America’s elections – would in practice actually leave the US House of Representatives elections wide open to undetected programming error or deliberate fraud. The problems with HR 550 are so fundamental they cannot be remedied simply by auditing more precincts.
About the authors
Bruce O’Dell, Coordinator of Data Analysis, Election Defense Alliance. O’Dell is an information technology consultant with 25 years experience who applies his expertise to analysis of the technical security and integrity of voting systems. His current consulting practice centers on e-Commerce security and the performance and design of very large-scale computer systems for Fortune 100 clients - recently as the chief technical architect in a company-wide security project at one of the top twenty public companies in America.
Jonathan Simon, JD, co-founder of the Election Defense Alliance. Simon is a graduate of Harvard College and New York University School of Law and is a member of the Bar of Massachusetts. He applies his prior experience as a political survey research analyst for Peter D. Hart Research Associates to studies of the accuracy of exit polls and other election integrity mechanisms. He has collaborated on several studies assessing the accuracy of the 2004 presidential exit polls.
About Election Defense Alliance
Election Defense Alliance, founded July 4, 2006, is a coalition of election integrity activists working at the state and local levels across the nation to detect and counter covert, antidemocratic manipulation of voter registration databases and all electronic voting systems; to regain public control of the voting process in the United States; and to insure that the process is honest, transparent, secure, subject to unambiguous verification, and worthy of the public trust.
EDA Contacts
Jonathan Simon (617-538-6012) jonathan[at]electiondefensealliance[dot]org
Sally Castleman sallyc[at]electiondefensealliance[dot]org
Dan Ashby (510-233-2144) dan[at]electiondefensealliance[dot]org
www.ElectionDefenseAlliance.org
UBS
Universal Ballot Sampling To Validate Computerized Vote Counts In Federal and Statewide Elections
Jonathan D. Simon, JD, and Bruce O’Dell
Election Defense Alliance1
Introduction
There is a deep and spreading distrust of the computer-based systems of vote tabulation in the United States, and a rapidly growing concern that outcome-determinative electronic vote fraud can easily go unchecked and undetected. While there are proposals being put forth to eliminate or drastically limit the role of proprietary computerized counting systems in elections, these have little prospect of immediate adoption in the vast majority of venues, and it is clear that the upcoming 2006 election (“E2006”) will be highly computerized and that “faith-based voting” on computerized systems will be even more widespread in E2006, and again in E2008, than in any previous election.
The need for non-computerized check mechanisms on the vote counts in E2006 and E2008, in the absence of full hand counting, is common ground for virtually all who are working for honest elections, but the advantages and drawbacks of the various potential check mechanisms remain a source of considerable confusion. Having previously established the fundamental shortcomings of legislation currently pending before Congress,2 our purpose in this paper is to introduce and explain the workings of a check mechanism that is both manifestly effective and eminently feasible: Universal Ballot Sampling (UBS).
By drawing and hand-counting a uniform and homogeneous random sample from all of the actual ballots cast in a given federal race, whether for the United States Senate or House of Representatives, we can evaluate the validity of the computerized vote count with great accuracy and confidence. And we can do so with a vastly smaller investment of labor and resources than would be required for the hand counting of every ballot.
Putting The Power Of Statistics To Work
The benefit of statistical sampling lies in the surprisingly strong power of a small part to predict the behavior of a large whole. Although we tend to accept the results of polls and other research based on sampling, it is nevertheless something of a head-scratcher that one can predict with considerable precision the preferences of a nation of 300 million individuals—whether in candidates, policies, or favorite kind of cheese—by questioning a mere 3000, or .001%, of them. This is, however, the case, providing that certain conditions apply and certain procedures are followed.
The laws of statistics tell us that by sampling a small percentage of the ballots cast in a relatively large target venue such as a state or congressional district, we will be able to determine with great confidence whether the otherwise “faith-based” computerized vote counts are accurate or whether significant, potentially outcome-altering “mistabulation” has indeed occurred. As will be detailed below, if the actual ballots are sampled and counted, and if this is done randomly and proportionally throughout the venue, then the statistical laws governing the accuracy of the sample count will be simple and mathematically straightforward, what we might call “crisp.” If bias or certain bows to convenience enter into the sampling process, our statistical process loses its "crispness," the cut-and-dried rule of simple equations, and all bets may well be off. Thus, when only certain precincts are selected for auditing, or when exit polls are used as a check mechanism, factors can intrude which complicate the statistical picture, and the laws governing accuracy become “soggy.”
Factors such as the “cluster effect,” which comes from sampling in convenient bunches (e.g., selecting only some precincts), or “differential response,” which comes from depending on voluntary voter interviews rather than actual ballots, are difficult if not impossible to quantify with the precision required for a consistently effective and reliable check mechanism. To use the power of statistics most effectively as a check mechanism, it is imperative that we keep the statistics “crisp.”
Assuming access to the actual ballots cast (including absentee and early-voting ballots, as well as those cast at the precinct), there are only two significant factors that will determine how accurate a handcount sample will be relative to the count of the whole: 1) number of ballots sampled, and 2) the randomness of the sample.3 The Universal Ballot Sampling protocol is free of such bugaboos as cluster effect or differential response, and satisfies the “randomness” criteria extremely well. The number of ballots sampled will thus be straightforwardly determined by how accurate we will require our sampling to be. Accuracy, in turn, boils down to two familiar statistical measurements: margin of error and level of confidence.
Margin of error (MOE) of a sample refers to the range in which we expect to find the difference between the count of the sample and the count of the whole from which the sample is drawn. And confidence level (CL) tells us how strong our expectation of finding the difference within that range should be. In most research, which by convention uses a CL of 95%, a +/- x%
MOE means that 95% of the time, or 19 out of 20 times, we expect the count of the sample to fall within x% of the count of the whole. This means that in such a sample, at a CL of 95%, 5% of the time a random sample with an x% MOE will count up more than x% off from the whole. That's just the way it is. You don't get 100% certainty. But what is certain is that if you ran that sample a million times, the number of times it missed the whole by more than x% would approach 50,000 (5%) very closely. Computer simulations, which we have employed in developing our protocol, are most helpful because they enable one to actually do this and check the results.
The Sampling Protocol
For votecount checking, we propose a MOE of +/- 1% at a CL of 99%. That is we recommend sampling a number of ballots which will yield results within 1% of an accurately tabulated official count 99 times out of 100. Thus, if the results of the sample are “off” by more than 1%, we would conclude with 99% certainty that the official count was inaccurate. Or, to look at it the other way, there is only a one in 100 chance that the official count is accurate and we’re looking at a “false alarm.”
Once we specify the MOE and CL, the statistical law governing random samples tells us with great precision how many ballots we need to sample for a given race.4 For a +/- 1% MOE at a CL of 99%, in a Congressional District race with 200,000 votes cast, the magic number is exactly 15,317 ballots.5 Because this number varies so little with change in the total number of votes cast, we find that for competitive U.S. House races, where the total number of votes cast varies between about 150,000 and 250,000, we need to sample between 14,936 and 15,556 ballots to achieve a +/- 1% MOE at 99% confidence. Thus it may be seen that if we sample 10% of the total ballots cast in each House race (that is, between about 15,000 and 25,000 ballots), we will achieve 99% (or greater) confidence that our sample results are within 1% of an accurate vote count in all competitive races.
It should be noted that while this analysis of our sample ballots does not appear to tell us how inaccurate a “mistabulated” vote count might be, we can easily use our UBS sample results to determine, again with excellent precision, the likelihood that an inaccuracy of any specified size
