An HCI Analysis of the Palm Beach Ballot Controversy

Associate Professor

University of Michigan School of Information

Click here to check for newest version of this document

Document last modified 11/10/2000

This page is intended to document some of the controversy surrounding the Palm Beach County, FL ballots for the 2000 U.S. Presidential Elections, from the perspective of the field of HCI, or human-computer interaction. Thus far, the data is entirely assembled from secondary sources (news media coverage). Much of the analysis and interpretation is my own. I encourage people to send me additional information or analysis that you think might be helpful to include here, or links to other related analyses.

The Ballot: Why it Might Have Been Confusing to Voters

Here's an AP Photograph of the presidential portion of the ballot in Palm Beach County, FL.

Voters are supposed to punch the hole in the middle of the ballot corresponding to one President/Vice-President ticket.

From an HCI perspective, the first thing to point out is that the voter must establish a correspondence between candidate tickets and holes in the middle. Contrast this with a potential selection method where voters would directly manipulate the names. The need to establish a correspondence in the mind of the voter creates the potential for voters to establish an incorrect correspondence. It seems to me that there are at least three ways for a voter to establish a correspondence between candidate tickets and holes in the middle.

Method 1: Visually follow the arrows

In the box for each ticket, near the center, there is an arrow. For example, for the Libertarian ticket there is an arrow at the right that seems to point pretty clearly to a particular punch hole. This method would establish a correct correspondence, so that voter intended for any candidate would actually be counted for that candidate.

Method 2: Count based on the printed numbers

In the box for each ticket, right next to the arrow, there is a number. For example, for the Libertarian ticket the number is 7. It's not obvious where to start counting the holes, but if the top hole is counted as 1, this would cause a voter who intended to vote for the Libertarian candidate to actually vote for the Green party candidate, and voter who intended to vote Democrat to have their vote counted for the Green party candidate.

Method 3: Count based on ordinal position

A voter might count the candidate tickets and infer the ordinal position of their ticket, then count off the holes down the center. One possibility would be for voters to count the position only in the left column, so that Republican would be 1, Democrat 2, Libertarian 3, and so on. This correspondence would cause a voter intending to vote Republican to do so correctly, but a voter intending to vote Democrat to have the vote actually be counted for Reform.

One of the interesting things about Method 3 is that while it seems fairly natural for the first few tickets in the left column, it seems unlikely that a voter would construct a cognitive map based on ordinal position that included all the candidate tickets. Consideration of the right-hand column would probably cause a voter to reevaluate the tentative correspondence he or she had established for the left-hand side.

Evaluation: Were Voters Really Confused and Did that Confusion Lead to Vote Counting that Did Not Match Voter Intentions?

How could you tell if voters were in fact confused and cast ballots that did not reflect their intentions? There are several possible indicators:

During voting, were there a lot questions asked of election officials that reflected voters making mental correspondences with method 2 or 3?
Did voters punch more than one hole, potentially indicating that during the voting process they used one method of establishing the correspondence and then decided that another correspondence was the right one.
After voting, did some voters realize that they had used an incorrect correspondence and express concern to friends, election officials, or reporters?
Did any candidate ticket that would have benefited from incorrect voter correspondences of methods 2 or 3 receive a surprisingly large number of votes?

I discuss each of these indicators below, together with the evidence I've been able to assemble so far on whether they happened or not.

Questions While Voting. I have not seen any reports yet about whether election officials got a lot of questions from voters about the correspondences. Apparently, there was some concern by election officials in advance of the election, and according to Congressman Wexler, a memo was sent to poll workers telling them to remind voters to vote for one candidate and follow the numbered hole (as reported in the Sun-Sentinel newspaper).
Disqualified ballots. Ballots with more than one hole punch are automatically disqualified. Some 19,120 ballots had more than one candidate punched and hence were not counted (based on reporting in the Sun-Sentinel newspaper). This amounted to more than 4% of the total number of ballots that were counted: 461,988 (according to official Palm Beach County results). There are other potential explanations for this large number, including the possibility that voters changed their minds about who to vote for while they were voting, that they thought they should vote separately for President and Vice-President rather than once for the ticket, that they didn't realize they could only vote for one, or that they just had no clue about the whole process. There is no way to tell for sure whether the cause of the double punching was confusion about how to establish a correspondence or one of these other explanations. However, there are ways to make some inferences. A comparison could be made with the percentage of double punches on other parts of this ballot or with the percentage of double punches in previous presidential elections. On this ballot, only 3783 voters double-punched on the Senate portion (based on CNN report), even though almost all cast a vote on that portion (461,978). I have not seen a comparison to the last presidential election, nor have I seen comparisons to the percentage of rejected ballots in other counties.

Perhaps even more instructive would be to examine the nature of the double-punches. If they almost all reflect the same confusion about the correspondence (e.g., if the two punched holes were for Democrat and Reform, which could result from a voter intending to vote Democrat and switching between correspondence 1 and 3 above) rather than being more uniformly distributed, that would provide further evidence for a confusion about correspondences, and in fact evidence for one particular confusion. I have not yet seen exact data on which holes were punched on the double-punched ballots.
Voter complaints. Congressman Wexler reports (RealAudio) that his office received hundreds of complaints from voters on the day of the election, even before it became apparent that the Florida race would decide the election. Many of these voters' explanations of their confusion points to their using correspondence 3 above. For example,
No one has complained about interpreting the ballot according to correspondence 2 above, so that probably wasn't a problem.
Surprising Votes for Buchanan. In the initial count, there were 3,407 votes (.79% of the total) for Patrick Buchanan and the Reform Party ticket. These voters may, of course, have intended to vote for the Reform ticket. Some have suggested that this was unlikely, however, because the area votes heavily Democratic and many of its Jewish voters would never vote for Buchanan. Even statewide, where he might be expected to be more popular, Buchanan's percentage was much smaller (.29%). Greg Adams from Carnegie-Mellon has posted a statistical analysis showing that Palm County was indeed a statistical outlier in its results and that a regression analysis suggests Buchanan should have received about 792 votes if it followed the trends of the rest of the state. Patrick Buchanan himself said, "I don't want to take any votes that don't belong to me," adding that he had not campaigned in Palm Beach and that the majority of those votes probably belonged to Gore. (Sun-Sentinel report). In one of the Congressional races in the area, however, the Reform Party candidate did receive 2,651 votes (2.09% of the total in that race).
In the initial count, there were also 743 votes for the Libertarian ticket. Again, these may have been deliberate, or they could reflect a problem from correspondence 2 above.

The Testing and Approval Process

The two-column ballot was unusual. In prior Florida elections, there have not been so many candidates, and hence two columns were not necessary. In other Florida counties, they either used smaller type and one column, or two separate pages. The two-column ballot was designed by the Palm Beach Supervisor of Elections, Theresa LePore, and was approved by the other two people on the canvassing board. The motivation was to provide larger typeface than would have been possible with a one-column, single page ballot.

No mention has been made of whether there was any user testing of the ballots prior to the election.

"I'll never use facing pages like that (again)," LePore said Wednesday morning, when the impact became clear. "I was trying to make the ballot more readable for our elderly voters in Palm Beach County. I was trying to do a good thing." (Sun-Sentinel report)

Remedies for This Election

It would be nice, of course, to be able to go back to each of the voters who voted for Buchanan or who punched two holes and ask them who they really intended to vote for. Ballots are secret, however, and there is no way to tell at this point which ballot belonged to which voter.

How to resolve the current Presidential election is beyond the scope of this analysis.

Possible Future Improvements: Roles for HCI Principles and HCI Practitioners

One obvious suggestion for future improvements is user testing of all new ballot designs. Subjects could go through a mock voting process, then be debriefed orally about who they intended to vote for. Any pattern of mismatches would then be detected. Note, however, that somewhat large subject samples might be necessary to detect problems. It appears that only about 1 in 20 of the Palm Beach voters were disqualified or cast their ballots unintentionally, though others may have been initially confused but figured it out. This suggests that bringing in half a dozen voters as subjects to test a ballot may not be sufficient to detect the nature and severity of problems.

A number of other HCI evaluation techniques could also be used that involve expert evaluators instead of or in addition to a sample of voters. These techniques include scenario walk-throughs and development of cognitive models of the voter's voting process. For example, the three alternative methods described in the first section of how a user might establish a correspondence between tickets and holes to punch reflect (very) informal cognitive models.

Perhaps the most important principle that could be applied is that of direct manipulation. It really would make more sense for voters to directly punch or mark the candidate's name rather than a hole some distance away that corresponds to that name. The hole punches are an artifact of our vote counting machines. New techniques with scanners and OCR (or touch-screens) should enable a more direct-manipulation interface for voting.

Finally, there should be no need for disqualified ballots. Each voter should get immediate feedback from the voting machine if they have voted improperly, and should be given a chance to correct their ballot, or fill out a new one. In computer interfaces, there has been a general trend towards more and more immediate feedback to users about errors in data entry (even web forms now have embedded JavaScript code to validate data entry). In many cases, computer interfaces have gone one step further to simply not allow invalid data entry in the first place, by having all data entry through menus and buttons. For example, in a computer interface, after one candidate was selected, the other candidates would be "grayed out" so that it would not be possible to vote again.

In any case, it seems pretty clear that HCI professionals should be involved in these decisions in the future, rather than leaving it to the intuitions, however well-intentioned, of election officials. Perhaps universities that provide HCI training to students could band together to offer a free evaluation service to election ballot designers in the future.