Partial failure of automation has already occurred; how to determine if the failure is total

The printing of 50.85 million ballots for the May 10 elections is now complete. The COMELEC said they were two days ahead of schedule. This early finish, however, was achieved at terrible cost: the high-speed printing caused the misalignment of the ultraviolet security mark on the ballot by one to two millimeters. This led the failure of the automated PCOS ballot authentication system, which the COMELEC had to abandon in favor of a manual ballot authentication system. This manual system relies on election inspectors to shine a UV lamp on each ballot to determine ballot authenticity through visual inspection.

The unreliable scanning of UV marks is, in effect, already a partial failure of automation.

But if the UV marks were misaligned due to the high-speed printing, shouldn’t we all be concerned that the ovals themselves have been similarly misaligned? Misaligned ovals would have very serious consequences. In the 1998 automation pilot in ARMM, according to a report on the COMELEC website, similar ballot printing problems forced the COMELEC to manually recount ballots from Sulu and some municipalities of Lanao del Sur. However carefully 2010 voters will shade the ovals, if these ovals are misaligned, then the voters’ marks will also be misaligned. This will make the PCOS unreliable in registering the voters’ choices, in exactly the same way they had become unreliable in detecting the UV security marks.

If HALAL’s concern turns out to be true, that ovals themselves were also misaligned by the same high-speed printing that misaligned the UV security marks, then we cannot rely anymore on the PCOS to accurately register the voters’ choices. An inaccurate PCOS machine would render useless the rest of the automated election system. It will mean a total failure of automation.

Misalignment of ovals is far more serious than misalignment of UV marks for the following reason: misaligned UV marks will lead the PCOS machine to reject valid ballots, an obvious problem which voters will notice immediately and complain about. This failure of automation cannot be hidden from the voters. Thus, the COMELEC has no choice but to correct the problem. This is also true for PCOS machines that stop working. But inaccurate vote counts due to oval misalignment will still be registered by the machine. Voters will never know their votes were falsely registered. This failure of automation can be hidden from voters. Thus, no one will notice, and no one can complain if the COMELEC and Smartmatic opt to hide this problem. Firms or persons may want to do so, if they can be held legally or financially accountable for the failure of a P7.2 billion project.

Earlier field tests and mock elections were announced by COMELEC to be “almost perfect” and by Smartmatic to be “successful” despite numerous media reports of machines rejecting valid ballots and transmission problems even in Metro Manila. These false claims suggest that the COMELEC and Smartmatic are not averse to declaring “success” despite public knowledge of ballot rejections and transmission problems. Thus, it would not be far-fetched for them to likewise claim “successful” automation on May 10, despite inaccurate scans and false machine counts which no voter will notice and complain about.

Indeed, the COMELEC has managed to hide so far the PCOS machine’s true scanning accuracy, which has remained a complete mystery.

Four opportunities to measure the true PCOS accuracy had been lost: 1) the results of the acceptance tests remain inaccessible to the public; 2) the full reports of Systest Labs, which conducted a system audit and source code review, also remain inaccessible to the public (update: Commissioner Larrazabal has agreed to release the Systest Labs report); and 3) the voter verification feature of the PCOS was disabled, preventing voters from knowing if the machine accurately registered their choices; and 4) the COMELEC decided to reject the proposal for a 100% audit of PCOS accuracy, and to stick instead to a random audit that covers only 1.5% of machines.

Unfortunately, the fifth opportunity to measure PCOS accuracy – and the alignment of ovals – is woefully inadequate for detecting an inaccurate machine, which, according to COMELEC specifications, is one with an accuracy rate below 99.995%. Several days before election day, every board of election inspectors (BEI) is supposed to set up the PCOS machines and do a final test. The machines will be fed with ten pre-tabulated ballots and the expected results compared with the machine count. To pass, a machine must count all the test ballots without a single error. COMELEC rules are not clear what the BEI should do if the PCOS makes even a single error. The most logical thing to do – a more rigorous version of this test should have been done months ago – is to return the machine to Smartmatic for replacement. A ten-ballot set is not enough. It will miss 98% of marginally inaccurate machines, and detect only 2%. A test set of 1,762 ballots is needed to screen out 95% of marginally inaccurate machines. A 2,708-ballot test set can screen out 99% of marginally inaccurate machines. though it will still pass 1%, or 822 of the 82,200 PCOS machines. How I got these numbers will require a separate piece. (Google “how to show that a PCOS is accurate enough”)

Instead of testing machines randomly with a 1,762-ballot test set, the COMELEC’s best option might be to heed the following HALAL suggestion: increase the random manual audit (RMA) coverage from 1.5% to 30% of the machines.

A number of strong arguments justify this simple suggestion: 1) the no-legal-basis argument cannot be invoked, because the law provides for the RMA; 2) the COMELEC clearly has the power to change the coverage of the RMA, because it has already raised the coverage from 0.3% (one precinct per district) to 1.5% (five precincts per district); 3) the COMELEC has long announced that it was prepared to do a manual count in 30% of the precincts, so the 30% RMA can simply use the existing paraphernalia for the manual count; no new expenses or preparations are needed; since no precinct knows in advance if it will be drawn for the RMA, every precinct must prepare to conduct an RMA anyway, whether the COMELEC is going to cover 1.5% or 30% of precincts; 4) a 30% RMA may not satisfy all, but it will certainly placate some groups, particularly the business sector, that had wanted a 100% parallel count; the twenty-fold increase in coverage from 1.5% to 30% will surely go a long way in raising the credibility of the electoral process; and 5) auditing 30% of the PCOS machines should be enough to determine the extent of PCOS inaccuracy caused by any misalignment of ovals.

Unless a credible audit is done, we might never know whether the machines counted our votes accurately or not, and whether the failure of automation we suffered was partial or total.

One Comment

  1. Posted May 1, 2010 at 5:12 pm | Permalink

    Here are few definitions to help understand this subject.

    *** ACCESS CODE
    Username and Security Password to use the computers
    PCOS, I believe, has three levels of secured access (user level, power user level, and administrator level)

    *** SOURCE CODE
    what humans type (A-Z, 0-9, etc.) to program computers.

    *** OBJECT CODE
    what computers use (0’s, 1’s) to think and run.

    *** INPUTS
    DATA that is entered (e.g. bubble marks on ballots)

    *** PROCESS
    the program (human source code requiring interpreter OR compiled object code above ready to run) that is
    1) Fetching inputs,
    2) Decoding data,
    3) Executing instructions,
    4) Storing results in memory (RAM).

    *** OUTPUT
    INFORMATION that is readable by humans (e.g. hardcopy printouts or screen displays)

    *** STORAGE
    Firmware (EEPROM) storage of program
    Softcopy digital version of INPUT Data
    Softcopy digital version of OUTPUT Information

    *** GOAL of AUTOMATED ELECTION
    CREDIBILITY of Elections
    in recording, tallying, reporting People’s Choices (Votes)

    *** QUALITY OF INFORMATION Checklist
    C-omplete information – requires system testing
    A-ccurate information – requires unit testing
    R-elevant information – requires datamine testing
    T-imely information – requires speed testing

    *** INITIAL TESTING
    Alpha Testing – done by manufacturer
    Beta Testing – done by selected users

    *** INSTALLATION or IMPLEMENTATION
    –Plunge – Stop Manual system, Run Automation Nationwide
    –Pilot – Try PLUNGE method only on few site like Hong Kong
    –Phase – Some sites Manual, Some sites Automated
    –Parallel – All sites Manual and Automated at the same time

    *** PRO’s (Pluses) for each Automation Implementation
    –Plunge – fastest and most economical
    –Pilot – detect errors on one site before plunge on all sites
    –Phase – allows gradual human familiarity of system
    –Parallel – guaranteed backup plan if automation fails

    *** CON’s (Minuses) for each Automation Implementation
    –Plunge – no backup plan if automation system fails
    –Pilot – extra cost to test on one site maybe unnecessary
    –Phase – non-uniformity, results may be confusing
    –Parallel – most expensive (extra PhP 380+ Million), most time consuming (requires extra 12 hours for full parallel count), danger of invalidating automated tally if it differs with manual tally, confusion in deciding which tally is accurate – manual or automated

    ***** My Recommendation: COMPROMISE *****

    a) run FULL AUTOMATION in ALL Sites Worldwide

    b) do Parallel count only on the PRESIDENTS in all sites

    which should just take an extra 1 hour instead of 12 hours

    c) if automation tally does not match manual tally within acceptable margin of error (e.g. plus or minus 1%), then redo the tally on automation using different machine

    d) if tallies still do not match, then redo manual tally

    e) if tallies still do not match, do manual tally and automated tally using different machine on vice-president along with president

    f) if tallies still do not match, redo tally using different machin on an additional category of single winner category local offices (e.g. governor, vice-governor, district congressman, mayor, or vice-mayor)

    I believe COMELEC said that they have two to four backup smartmatic machines per site. So doing the above should be feasible.

    What we need is to have FAITH in the System, HOPE in the Voters, and TRUST in the COMELEC.

    Let us give the benefit of the doubt to the COMELEC personnel who have been credible, forthcoming and amazingly earnest in fulfilling the wishes of the people for a clean and honest election. Mabuhay ang Pilipinas!

    The Voice of the People is The Voice of God!

    VOX POPULI est VOX DEO!

    +amdg: Ad Majorem Dei Gloriam

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: