Tag Archives: misalignment

HALAL April update: estimated chance of AES success is now 32%; PCOS accuracy remains a mystery

by Halalang Marangal (HALAL)

Last March 2010, Halalang Marangal issued an analysis of the estimated chance of success of the Automated Election System (AES) and put it at 25%. Based on developments in April, we are updating our estimates of the probabilities of success of the sub-projects and the AES itself as follows:

AES Sub-Project March April

  • Hardware 80% 80%
  • Software 70% 70%
  • Logistics 80% 90%
  • Transmission 70% 90%
  • Ballot Printing 80% 70%

Overall AES Project 25% 32%

Note that when estimating the overall chance of success of an entire project, comprising several sub-projects, each of which are essential to the success of the entire project, the individual probabilities of success of the sub-projects must be multiplied together, not averaged. Note too that we are estimating here the success or failure of automation, not the election itself.

In the hardware sub-project, there was no reason to modify our earlier assessment. Much of the hardware were still not fully tested, and neither were any test results made available to the public for scrutiny. Also, the purchase of 21% more memory cards than necessary remained unexplained, raising concerns that these extra memory cards, if they fall in the wrong hands, may be configured with false data and substituted for authentic cards.

In the software sub-project, no new developments occurred either, that might have led us to modify our assessment. The Systest Labs full report on its system audit and source code review remained inaccessible to the public, and no local group still has managed to conduct a source code review. PCOS software remained configured to disable the voter verification feature, an essential feature that enables voters to determine the accuracy of the PCOS with respect to the voters’ actual choices. The digital certification system remains in Smartmatic hands, instead of an independent third-party like the Department of Science and Technology.

In the logistics sub-project, HALAL has since learned that in addition to the three original small firms contracted to make nationwide deliveries of election paraphernalia for the Comelec, better capitalized forwarders like Air21, which have more experience in handling cargo, have also been contracted. This has led us to raise our estimate of this sub-project’s probability of success from 80% to 90%. Ensuring that paired ballots and PCOS machines, which are being delivered separately, will arrive on time in the right precincts remains a huge logistical problem.

In the transmission sub-project, HALAL has since learned that the Comelec will now be providing for 100% coverage of all precincts in terms of transmission capability, while the March 8 full-page ad of Smartmatic only reported enough transmission equipment to cover 70% of all precincts. Thus, we have raised our estimate of the probability of success of this sub-project from 70% to 90%. However, transmission problems even within Metro Manila as well as in remote provinces like Batanes still suggest that similar problems will occur on election day.

Under the ballot printing sub-project, the printing of 50.85 million ballots was reported by Comelec complete two days ahead of schedule.

It seems though that this early finish was attained at terrible cost. The Comelec says that the high-speed printing resulted in the “misalignment by one to two millimeters” of the ultraviolet security mark. The problem was serious enough that it led the Comelec to abandon the automatic PCOS authentication of ballots, in favor of a manual check for authenticity by shining a UV lamp on each ballot and letting the BEI determine ballot authenticity through visual inspection.

HALAL raises this important question: if the UV marks were misaligned due to the high-speed printing, could the ovals themselves have been similarly misaligned? Misaligned ovals would have very serious consequences. In the 1998 automation pilot in ARMM, according to a report on the Comelec website, similar ballot printing problems led the Comelec to manually recount ballots from Sulu and some municipalities of Lanao del Sur. However carefully voters will shade the ovals, if these ovals are misaligned, then the voters’ marks will also be misaligned, which will make the PCOS machine unreliable in scanning and counting the voters’ choices, in the same way it became so unreliable in scanning the UV marks that automatic scanning for ballot authenticity had to be abandoned.

Misalignment of ovals is far more serious than misalignment of UV marks for the following reason: misaligned UV marks will lead the PCOS machine to reject valid ballots, an obvious problem which voters will notice and complain about. Thus, the Comelec has no choice but to correct the problem. But misalignment of ballots leads to inaccurate vote counts, which will still be registered by the machine, although the voters will never know their votes were inaccurately registered. Thus if the Comelec chose to ignore this problem, no one will notice, and no one can complain. Earlier field tests and mock elections were announced by Comelec to be “almost perfect” and by Smartmatic to be “successful” despite numerous media reports of machines rejecting valid ballots and transmission problems even in Metro Manila. If they can make such false claims despite public knowledge of ballot rejections and transmission problems, it would be much easier for them to claim “successful” elections on May 10 despite inaccurate machine counts which no one will notice and complain about.

Because of this uncertain oval alignment, HALAL reduced its estimate of the probability of success of this sub-project from 80% to 70%.

Because of misaligned UV marks, the Comelec decided on its own, without prodding, to shift to a 100% manual audit for authenticity of the ballots, before any winner is proclaimed. The possible misalignment of ovals should have logically led to a similar 100% manual audit of ballots for accuracy of the machine counts, before any winner is proclaimed.

With the Comelec’s decision to reject the proposal for a 100% audit, and to stick instead to a random audit that covers only 1.5% of precincts, we have lost a fourth opportunity to ascertain the accuracy of the PCOS machines. Earlier, three other opportunities had also been lost: 1) the results of the acceptance tests remain inaccessible to the public; 2) the full reports of Systest Labs, which conducted a system audit and source code review, also remain inaccessible to the public; and 3) the voter verification feature of the machines was disabled. Sadly, the ten ballots that will be used by the BEI for testing three days before the elections are too few to reliably screen out inaccurate machines.

We will never know at all, it seems, how accurately these machines counted the voters’ choices.

April 29, 2010

Advertisements

The high-speed printing that misaligned UV marks can misalign the ballot ovals too

The ultraviolet (UV) scanner of the PCOS was disabled because the high-speed printing resulted in the UV mark on the ballot being “misaligned by one to two millimeters”, according to COMELEC Commissioner Larrazabal. As a result, the machine’s UV scanner often missed the mark and many valid ballots being being rejected.

Here’s the big question: If the misalignment of the UV mark was serious enough to make COMELEC turn off the UV scanner, then can’t the ovals be misaligned too? A misaligned oval means a misaligned a vote-mark. Which means the PCOS main scanner may have problems interpreting the voters’ choices.

The Comelec requires from the PCOS an error rate of less than .005%. That means less than five errors for every 100,000 marks. To determine if this Comelec specification is met, each PCOS should have been tested properly. But no test statistics have been released by the Comelec. And if these tests were done at all, they were probably done with perfectly aligned, not misaligned, ovals.

With misalignment, the two types of errors the scanner can make will both get worse: the false positives that register a vote/mark which is not there, and the false negatives that miss a vote/mark which is there. This means some candidates will gain votes (“dagdag”), while other candidates will lose votes (“bawas”). Does that sound familiar?

This problem is made worse by the Comelec decision to, in effect, blindfold voters while the machine is registering their choice. Originally, the PCOS was programmed to display the voter’s choices on its screen, so he can check if his choices were correctly registered by the PCOS and abort the process if the PCOS didn’t. This voter-verification is in fact required by the Automated Election Law (Section 7n): “Provide the voter a system of verification to find out whether or not the machine has registered his choice.” The Comelec ordered Smartmatic to disable this feature. Thus, the PCOS may falsely register voters’ choices, without voters knowing it. This is far worse than a PCOS that stops working or rejects valid ballots, problems which are apparent at once. An inaccurate PCOS will keep scanning and counting happily, with no indication or warning that it is miscounting votes.

According to a former Comelec official, similar problems had also occurred when the Comelec piloted automation in ARMM in 1998. He showed me a Comelec report entitled “Partial Automation of 1998 National and Local Elections”, which is also on the Comelec website. In 1998, said the report, Sulu ballots had to be manually recounted “due to an error in NPO’s printing procedure”. A similar recount was done with Lanao del Sur ballots, “also due to errors in printing of the ballots.”

The Comelec blamed “high-speed printing” for the misalignment. But misalignment is a common printing problem, and good operators know how to correct it. Why would it happen in such an important job as the printing of ballots?

It seems that the ballot printing is being done in such a hurry, that operators are not getting the time needed to stop the high-speed machines and correct any misalignments that may be happening. That the Comelec had insisted on printing ballots even on Maundy Thursday and Good Friday shows how disastrous even a slight delay might be. If they stopped the machines too often to correct for misalignments, they might miss their deadlines. Thus, the UV misalignment problem has gone on uncorrected.

As of April 16, according to Smartmatic, the NPO has printed 43.7 million ballots. If the high-speed printing has misaligned the UV marks, then how many of these ballots have misaligned ovals too? And in these misaligned ovals, how many false “dagdag-bawas” interpretations by the PCOS will occur?

The misalignment of UV marks and possibly the ovals too are the consequences of the Comelec violating the law and using a machine that has neither been piloted nor used widely. The SAES 1800 has never been piloted in the Philippines, as Section 6 of the Automated Election Law requires: “… the AES shall be used in at least two urbanized cities and two provinces each in Luzon, Visayas and Mindanao.” Nor has the PCOS been used widely in any other country, as Section 10 of the law also requires: “… the system procured must have demonstrated capability and been successfully used in a prior electoral exercise here or abroad.” The Supreme Court’s support for the Comelec’s stubborn insistence in using unpiloted Smartmatic machines must have reinforced the Comelec’s sense of impunity in violating provisions of the law.

Because of the lack of pilot, we lost the chance to detect early problems like these. As a result, they have put the entire national elections at risk.

Solution: both the PCOS and their associated ballots must be thoroughly checked for misalignment and accuracy in scanning. The tests must be done not by Smartmatic, but by independent third parties, say, the DOST, and witnessed by all stakeholders.

What if careful testing shows that the PCOS cannot reliably read properly shaded ovals? Then we may have no choice but to manually count the votes again, ballot by ballot.

[Note: This piece is based on the Halalang Marangal April 17 Statement on the problem of potentially-misaligned ovals, which I also drafted. It was published by the Philippine Star, p.18, on April 21, 2010.]

Roberto Verzola has a background in engineering and economics and a passion for social issues. He is recognized by the IT industry as an Internet pioneer in the Philippines and is often tapped by NGOs for technical advice. He currently lectures at the Institute of Mathematics of the University of the Philippines and is a convenor and secretary-general of the election watchdog Halalang Marangal (HALAL).