Paul Resnick (web page; email)
Associate Professor
University of Michigan School of Information
This is version v2 last modified 11/10/2000 10:10 AM EST
Click here to check for newest version of this document
Click here for archive of all previous versions
Differences from last version
Several new error models (4-8)
General re-edit
Some thoughts in the conclusion on the concept of statistical dead-heat for actual elections
This page is intended to document some of the controversy surrounding the Palm Beach County, FL ballots for the 2000 U.S. Presidential Elections, from the perspective of the field of HCI, or human-computer interaction, which studies usability of all kinds of systems. I encourage people to send me additional information or analysis that you think might be helpful to include here, or links to other related analyses.
Table of Contents
Models of Ballot-Punching Behavior
Evidence: Which Model(s) Did Voters Use
Remedies for This Election
Remedies for Future Elections
Here's an AP Photograph of the presidential portion of the ballot in Palm Beach County, FL.
Voters are supposed to punch the hole in the middle of the ballot corresponding to one President/Vice-President ticket.
From an HCI perspective, the first thing to point out is that the voter must establish a correspondence between candidate tickets and holes in the middle. Contrast this with a potential selection method where voters would directly manipulate the names. The need to establish a correspondence in the mind of the voter creates the potential for voters to establish an incorrect correspondence.
There are a number of ways a voter might establish a correspondence between candidate tickets and holes in the middle, detailed below. The first of these would cause all votes to be counted as intended; others would lead to particular patterns of invalid ballots and/or incorrectly counted ballots. Each of these models of how voters established a correspondence leads to predictions about patterns of double-punched ballots and/or transfers of votes to other tickets than those intended. These are evaluated in the following section.
In the box for each ticket, near the center, there is an arrow. For example, for the Libertarian ticket there is an arrow at the right that seems to point pretty clearly to a particular punch hole. When commentators argue that "it's easy; just follow the arrows", this is the method that they have in mind. This method would establish a correct correspondence, so that voter intended for any candidate would actually be counted for that candidate.
In the box for each ticket, right next to the arrow, there is a number. For example, for the Republican ticket the number is 3. It's not obvious where to start counting the holes, but if the top hole is counted as 1, this would cause a voter who intended to vote for the Republican ticket to actually vote for the Democratic ticket, a voter who intended to vote Democrat to have their vote counted for the Libertarian party ticket, and so on. [I am grateful to Charles Henkel for pointing out that if this Method were popular, the net effect would probably be to transfer votes from the Republican to Democrat. The way to detect it, however, is to see whether the Libertarian vote count was surprisingly high, as discussed in the section on evidence below.]
A voter might count the candidate tickets and infer the ordinal position of their ticket, then count off the holes down the center. One possibility would be for voters to count the position only in the left column, so that Republican would be 1, Democrat 2, Libertarian 3, and so on. This correspondence would cause a voter intending to vote Republican to do so correctly, but a voter intending to vote Democrat to have the vote actually be counted for Reform.
One of the interesting things about Method 3 is that while it seems fairly natural for the first few tickets in the left column, it seems unlikely that a voter would construct a cognitive map based on ordinal position that included all the candidate tickets. Consideration of the right-hand column would probably cause a voter to reevaluate the tentative correspondence he or she had established for the left-hand side.
Kevin Fox suggests that voters may have grouped the candidate's names with the line above it, then followed that line over to the center and punched the nearest hole. (See the Linear Visual Search hypothesis on his page for a picture). Using this mapping, a Republican vote would be counted correctly, a Reform vote would be counted for the Republicans, a Democrat vote would be counted for the Reform ticket, and a Socialist vote would be counted for the Democrats.
Kevin Fox argues for why grouping names with the lines above rather than below the tickets is more plausible. Still, an alternative hypothesis would be grouping with the line below. Under that mapping, a Republican vote would be counted for the Reform ticket, Reform for Democrats, and Democrats for Socialists. This method is of special interest, because if it were used by a large number of voters, there should be a surprisingly large Reform vote. Thus, it offers a different explanation than Method 3 above, one which suggests that some of the Reform vote could actually have been from Republican voters. If this method were popular, we should also see a surprisingly large Socialist vote.
Kevin Fox suggests another method, visual Gestalt grouping, whereby voters would group together visually the number, the arrow, the hole to punch, and the long line from the opposite side of the page. The long line would then be associated with with ticket below it. This would cause a Democratic vote to be counted for Reform and Reform for Democrat. Since there's no number and arrow opposite the Republican ticket, Kevin Fox argues that Republican voters would not make this incorrect Gestalt grouping at all, and would find some other way to make the association with their candidate.
Again, it seems possible that with visual gestalt, the line would be associated with the ticket above rather than below the line. With this method, a Republican voter would mark Reform; a Reform voter would mark Democrat; a Democrat voter would mark Socialist.
At least one voter claims that he looked only at the left column of candidates, saw that there were two holes in the center for each ticket, and assumed that he was supposed to punch one for the President and one for the Vice-President. A Republican voter who did this would punch Republican and Reform. A Democrat who did this would punch Democrat and either Reform or Socialist, depending on whether they picked the extra hole on top or on the bottom.
In addition to cognitive errors in establishing a correspondence between the voter's preferred ticket and the correct holes to punch, some voters may have had trouble punching the holes they intended to punch. [I am grateful to Andrew Hobgood for pointing out this possibility, and the "fat-pen" method in particular.]
A voter with an unsteady hand might punch an adjacent hole by accident. This would cause some Republican voters to punch Reform, and some Democratic voters to punch both Reform and Socialist (probably in equal numbers). If many voters did this, we should see a distribution of "unexpected ballots" based on these proportions. If a voter didn't punch very far, he or she might not realize it was punched, and then try again to punch the correct hole. This should be reflected in the distribution of double-punches.
My memory of using punch ballots in previous elections (my polling place doesn't use them now) is that it's not possible to simultaneously punch more than one, because there's a solid barrier in between. If it is possible, however, some voters might have done that, and it should be reflected in the distribution of double punched ballots.
How could one tell if voters were in fact confused and cast ballots that did not reflect their intentions? There's no way to know for sure, but there are several possible indicators:
I discuss each of these indicators below, together with the evidence I've been able to assemble so far on whether they happened or not.
"I'll never use facing pages like that (again)," LePore said Wednesday morning, when the impact became clear. "I was trying to make the ballot more readable for our elderly voters in Palm Beach County. I was trying to do a good thing." (Sun-Sentinel report)
The two-column ballot was designed by the Palm Beach Supervisor of Elections, Theresa LePore, a Democrat, and was approved by the other two people on the canvassing board. The two-column ballot was unusual. In prior Florida elections, there have not been so many candidates, and hence two columns were not necessary. In other Florida counties, they either used smaller type and one column, or two separate pages.
No mention has been made of whether there was any user testing of the ballots prior to the election. Sample ballots were, however, mailed out to voters in advance of the election [Thanks to Allison Groff for mentioning this; I do not yet have a citation verifying this claim.] This would have familiarized voters with the locations of the candidates, though not necessarily with the hole punch correspondence.
How to resolve the current Presidential election is beyond the scope of this analysis.
It would be nice, of course, to be able to go back to each of the voters who voted for Buchanan or who punched two holes and ask them who they really intended to vote for. Ballots are secret, however, and there is no way to tell at this point which ballot belonged to which voter. It would probably be possible to tell, approximately but not exactly, who the double-punched ballots were intended to vote for. If there were similar percentages of disqualified ballots in other counties, however, it would seem unfair to do this only in one county. If the percentage in Palm Beach County is high, then it might be reasonable.
Republican analysts appear to be arguing that the election was "ex-ante" fair (while some irregularities are unavoidable, before election day there was no way to know where they would pop up or what they would be, and hence no reason to expect that they would favor one candidate over another) and that's the best we can hope for from an election. Democratic analysts are pushing for an "ex-post" fair election, where every voter who went to the polls should have their vote counted toward the candidates of their choice. I suspect that there's a whole body of literature in political philosophy about ex-ante versus ex-post fairness, so I'll just leave this one for others to wrangle over for now.
One obvious suggestion for future improvements is user testing of all new ballot designs. Subjects could go through a mock voting process, then be debriefed orally about who they intended to vote for. Any pattern of mismatches would then be detected. Note, however, that somewhat large subject samples might be necessary to detect problems. It appears that only about 1 in 20 of the Palm Beach voters were disqualified or cast their ballots unintentionally, though others may have been initially confused but figured it out. This suggests that bringing in half a dozen voters as subjects to test a ballot may not be sufficient to detect the nature and severity of problems.
A number of other HCI evaluation techniques could also be used that involve expert evaluators instead of or in addition to a sample of voters. These techniques include scenario walk-throughs and development of cognitive models of the voter's voting process. For example, the alternative methods described in the first section of how a user might establish a correspondence between tickets and holes to punch reflect (very) informal cognitive models.
Perhaps the most important principle that could be applied is that of direct manipulation. It really would make more sense for voters to directly punch or mark the candidate's name rather than a hole some distance away that corresponds to that name. The hole punches are an artifact of our vote counting machines. New techniques with scanners and OCR (or touch-screens) should enable a more direct-manipulation interface for voting.
Finally, there should be no need for disqualified ballots. Each voter should get immediate feedback from the voting machine if they have voted improperly, and should be given a chance to correct their ballot, or fill out a new one. In computer interfaces, there has been a general trend towards more and more immediate feedback to users about errors in data entry (even web forms now have embedded JavaScript code to validate data entry). In many cases, computer interfaces have gone one step further to simply not allow invalid data entry in the first place, by having all data entry through menus and buttons. For example, in a computer interface, after one candidate was selected, the other candidates would be "grayed out" so that it would not be possible to vote again.
In any case, it seems pretty clear that HCI professionals should be involved in these decisions in the future, rather than leaving it to the intuitions, however well-intentioned, of election officials. Some standardization would probably help, as otherwise there would be too many local ballots to evaluate each election. Perhaps universities that provide HCI training to students could band together to offer a free evaluation service to election ballot designers in the future.
It's probably also important to realize that there will never be perfect user interfaces. Some people will always make mistakes and their ballots will not be counted for the candidate they would have liked. It's important that a democracy minimize these irregularities, and it's almost important that the irregularities be randomly distributed, rather than leading to a biased outcome in favor of one candidate or another. It may be helpful to think of an actual election as having some margin of error or confidence interval, just as we think of opinion polls as having these margins. Here the actual election is a single sample drawn from a hypothetical distribution of things that could have happened on election day. The confidence intervals are much tighter for actual elections than for random sample opinion polls.
But occasionally elections will end in a statistical dead heat (almost certainly this Florida ballot would qualify, and probably the national popular vote count as well, so that eliminating the electoral college would not alter the situation we're in). We need to be prepared as a democracy for how to handle cases where an election simply does not name a clear winner. I have no idea how we should handle it, but we need some way other than both sides jockeying to claim a victory based on the balloting when that balloting was inconclusive. Another election doesn't seem especially useful in such a case, since unless opinions change, it too is quite likely to come out inconclusive.