Partial failure of automation has already occurred; how to determine if the failure is total

The printing of 50.85 million ballots for the May 10 elections is now complete. The COMELEC said they were two days ahead of schedule. This early finish, however, was achieved at terrible cost: the high-speed printing caused the misalignment of the ultraviolet security mark on the ballot by one to two millimeters. This led the failure of the automated PCOS ballot authentication system, which the COMELEC had to abandon in favor of a manual ballot authentication system. This manual system relies on election inspectors to shine a UV lamp on each ballot to determine ballot authenticity through visual inspection.

The unreliable scanning of UV marks is, in effect, already a partial failure of automation.

But if the UV marks were misaligned due to the high-speed printing, shouldn’t we all be concerned that the ovals themselves have been similarly misaligned? Misaligned ovals would have very serious consequences. In the 1998 automation pilot in ARMM, according to a report on the COMELEC website, similar ballot printing problems forced the COMELEC to manually recount ballots from Sulu and some municipalities of Lanao del Sur. However carefully 2010 voters will shade the ovals, if these ovals are misaligned, then the voters’ marks will also be misaligned. This will make the PCOS unreliable in registering the voters’ choices, in exactly the same way they had become unreliable in detecting the UV security marks.

If HALAL’s concern turns out to be true, that ovals themselves were also misaligned by the same high-speed printing that misaligned the UV security marks, then we cannot rely anymore on the PCOS to accurately register the voters’ choices. An inaccurate PCOS machine would render useless the rest of the automated election system. It will mean a total failure of automation.

Misalignment of ovals is far more serious than misalignment of UV marks for the following reason: misaligned UV marks will lead the PCOS machine to reject valid ballots, an obvious problem which voters will notice immediately and complain about. This failure of automation cannot be hidden from the voters. Thus, the COMELEC has no choice but to correct the problem. This is also true for PCOS machines that stop working. But inaccurate vote counts due to oval misalignment will still be registered by the machine. Voters will never know their votes were falsely registered. This failure of automation can be hidden from voters. Thus, no one will notice, and no one can complain if the COMELEC and Smartmatic opt to hide this problem. Firms or persons may want to do so, if they can be held legally or financially accountable for the failure of a P7.2 billion project.

Earlier field tests and mock elections were announced by COMELEC to be “almost perfect” and by Smartmatic to be “successful” despite numerous media reports of machines rejecting valid ballots and transmission problems even in Metro Manila. These false claims suggest that the COMELEC and Smartmatic are not averse to declaring “success” despite public knowledge of ballot rejections and transmission problems. Thus, it would not be far-fetched for them to likewise claim “successful” automation on May 10, despite inaccurate scans and false machine counts which no voter will notice and complain about.

Indeed, the COMELEC has managed to hide so far the PCOS machine’s true scanning accuracy, which has remained a complete mystery.

Four opportunities to measure the true PCOS accuracy had been lost: 1) the results of the acceptance tests remain inaccessible to the public; 2) the full reports of Systest Labs, which conducted a system audit and source code review, also remain inaccessible to the public (update: Commissioner Larrazabal has agreed to release the Systest Labs report); and 3) the voter verification feature of the PCOS was disabled, preventing voters from knowing if the machine accurately registered their choices; and 4) the COMELEC decided to reject the proposal for a 100% audit of PCOS accuracy, and to stick instead to a random audit that covers only 1.5% of machines.

Unfortunately, the fifth opportunity to measure PCOS accuracy – and the alignment of ovals – is woefully inadequate for detecting an inaccurate machine, which, according to COMELEC specifications, is one with an accuracy rate below 99.995%. Several days before election day, every board of election inspectors (BEI) is supposed to set up the PCOS machines and do a final test. The machines will be fed with ten pre-tabulated ballots and the expected results compared with the machine count. To pass, a machine must count all the test ballots without a single error. COMELEC rules are not clear what the BEI should do if the PCOS makes even a single error. The most logical thing to do – a more rigorous version of this test should have been done months ago – is to return the machine to Smartmatic for replacement. A ten-ballot set is not enough. It will miss 98% of marginally inaccurate machines, and detect only 2%. A test set of 1,762 ballots is needed to screen out 95% of marginally inaccurate machines. A 2,708-ballot test set can screen out 99% of marginally inaccurate machines. though it will still pass 1%, or 822 of the 82,200 PCOS machines. How I got these numbers will require a separate piece. (Google “how to show that a PCOS is accurate enough”)

Instead of testing machines randomly with a 1,762-ballot test set, the COMELEC’s best option might be to heed the following HALAL suggestion: increase the random manual audit (RMA) coverage from 1.5% to 30% of the machines.

A number of strong arguments justify this simple suggestion: 1) the no-legal-basis argument cannot be invoked, because the law provides for the RMA; 2) the COMELEC clearly has the power to change the coverage of the RMA, because it has already raised the coverage from 0.3% (one precinct per district) to 1.5% (five precincts per district); 3) the COMELEC has long announced that it was prepared to do a manual count in 30% of the precincts, so the 30% RMA can simply use the existing paraphernalia for the manual count; no new expenses or preparations are needed; since no precinct knows in advance if it will be drawn for the RMA, every precinct must prepare to conduct an RMA anyway, whether the COMELEC is going to cover 1.5% or 30% of precincts; 4) a 30% RMA may not satisfy all, but it will certainly placate some groups, particularly the business sector, that had wanted a 100% parallel count; the twenty-fold increase in coverage from 1.5% to 30% will surely go a long way in raising the credibility of the electoral process; and 5) auditing 30% of the PCOS machines should be enough to determine the extent of PCOS inaccuracy caused by any misalignment of ovals.

Unless a credible audit is done, we might never know whether the machines counted our votes accurately or not, and whether the failure of automation we suffered was partial or total.

HALAL April update: estimated chance of AES success is now 32%; PCOS accuracy remains a mystery

by Halalang Marangal (HALAL)

Last March 2010, Halalang Marangal issued an analysis of the estimated chance of success of the Automated Election System (AES) and put it at 25%. Based on developments in April, we are updating our estimates of the probabilities of success of the sub-projects and the AES itself as follows:

AES Sub-Project March April

• Hardware 80% 80%
• Software 70% 70%
• Logistics 80% 90%
• Transmission 70% 90%
• Ballot Printing 80% 70%

Overall AES Project 25% 32%

Note that when estimating the overall chance of success of an entire project, comprising several sub-projects, each of which are essential to the success of the entire project, the individual probabilities of success of the sub-projects must be multiplied together, not averaged. Note too that we are estimating here the success or failure of automation, not the election itself.

In the hardware sub-project, there was no reason to modify our earlier assessment. Much of the hardware were still not fully tested, and neither were any test results made available to the public for scrutiny. Also, the purchase of 21% more memory cards than necessary remained unexplained, raising concerns that these extra memory cards, if they fall in the wrong hands, may be configured with false data and substituted for authentic cards.

In the software sub-project, no new developments occurred either, that might have led us to modify our assessment. The Systest Labs full report on its system audit and source code review remained inaccessible to the public, and no local group still has managed to conduct a source code review. PCOS software remained configured to disable the voter verification feature, an essential feature that enables voters to determine the accuracy of the PCOS with respect to the voters’ actual choices. The digital certification system remains in Smartmatic hands, instead of an independent third-party like the Department of Science and Technology.

In the logistics sub-project, HALAL has since learned that in addition to the three original small firms contracted to make nationwide deliveries of election paraphernalia for the Comelec, better capitalized forwarders like Air21, which have more experience in handling cargo, have also been contracted. This has led us to raise our estimate of this sub-project’s probability of success from 80% to 90%. Ensuring that paired ballots and PCOS machines, which are being delivered separately, will arrive on time in the right precincts remains a huge logistical problem.

In the transmission sub-project, HALAL has since learned that the Comelec will now be providing for 100% coverage of all precincts in terms of transmission capability, while the March 8 full-page ad of Smartmatic only reported enough transmission equipment to cover 70% of all precincts. Thus, we have raised our estimate of the probability of success of this sub-project from 70% to 90%. However, transmission problems even within Metro Manila as well as in remote provinces like Batanes still suggest that similar problems will occur on election day.

Under the ballot printing sub-project, the printing of 50.85 million ballots was reported by Comelec complete two days ahead of schedule.

It seems though that this early finish was attained at terrible cost. The Comelec says that the high-speed printing resulted in the “misalignment by one to two millimeters” of the ultraviolet security mark. The problem was serious enough that it led the Comelec to abandon the automatic PCOS authentication of ballots, in favor of a manual check for authenticity by shining a UV lamp on each ballot and letting the BEI determine ballot authenticity through visual inspection.

HALAL raises this important question: if the UV marks were misaligned due to the high-speed printing, could the ovals themselves have been similarly misaligned? Misaligned ovals would have very serious consequences. In the 1998 automation pilot in ARMM, according to a report on the Comelec website, similar ballot printing problems led the Comelec to manually recount ballots from Sulu and some municipalities of Lanao del Sur. However carefully voters will shade the ovals, if these ovals are misaligned, then the voters’ marks will also be misaligned, which will make the PCOS machine unreliable in scanning and counting the voters’ choices, in the same way it became so unreliable in scanning the UV marks that automatic scanning for ballot authenticity had to be abandoned.

Misalignment of ovals is far more serious than misalignment of UV marks for the following reason: misaligned UV marks will lead the PCOS machine to reject valid ballots, an obvious problem which voters will notice and complain about. Thus, the Comelec has no choice but to correct the problem. But misalignment of ballots leads to inaccurate vote counts, which will still be registered by the machine, although the voters will never know their votes were inaccurately registered. Thus if the Comelec chose to ignore this problem, no one will notice, and no one can complain. Earlier field tests and mock elections were announced by Comelec to be “almost perfect” and by Smartmatic to be “successful” despite numerous media reports of machines rejecting valid ballots and transmission problems even in Metro Manila. If they can make such false claims despite public knowledge of ballot rejections and transmission problems, it would be much easier for them to claim “successful” elections on May 10 despite inaccurate machine counts which no one will notice and complain about.

Because of this uncertain oval alignment, HALAL reduced its estimate of the probability of success of this sub-project from 80% to 70%.

Because of misaligned UV marks, the Comelec decided on its own, without prodding, to shift to a 100% manual audit for authenticity of the ballots, before any winner is proclaimed. The possible misalignment of ovals should have logically led to a similar 100% manual audit of ballots for accuracy of the machine counts, before any winner is proclaimed.

With the Comelec’s decision to reject the proposal for a 100% audit, and to stick instead to a random audit that covers only 1.5% of precincts, we have lost a fourth opportunity to ascertain the accuracy of the PCOS machines. Earlier, three other opportunities had also been lost: 1) the results of the acceptance tests remain inaccessible to the public; 2) the full reports of Systest Labs, which conducted a system audit and source code review, also remain inaccessible to the public; and 3) the voter verification feature of the machines was disabled. Sadly, the ten ballots that will be used by the BEI for testing three days before the elections are too few to reliably screen out inaccurate machines.

We will never know at all, it seems, how accurately these machines counted the voters’ choices.

April 29, 2010

The printing problem in 1998

The Comelec first tried automation in 1998, based on optical scanners, similar to the technology they are using today.

The Comelec report about that 1998 experience says:

Ballots for the province of Sulu were re-counted at the Philippine International Convention Center (PICC) due to an error in NPO’s printing procedure. Results in several municipalities in Lanao del Sur were re-counted manually also due to errors in printing of the ballots and procedural errors.

In short, printing problems also resulted in the misalignment of bubbles (as they called the ovals then), forcing the Comelec to recount ballots from Sulu and some towns of Lanao del Sur.

In 2010, the misalignment was caused by the high-speed printing, as the Comelec itself acknowledge in the case of the ultraviolet security mark. This forced the Comelec to revert to a manual method of authenticating each ballot using visual inspection by election inspectors wielding ultraviolet lamps. If the UV marks are misaligned, then the ovals themselves can be misaligned too.

If so, then we might have no choice but to count votes again, ballot by ballot.

How to show that a PCOS voting machine is accurate enough

The COMELEC has already rejected 100% manual audit proposal. That should be the end of it.

Yet, it is hard ignore the voices of the country’s major information technology organizations, business oranizations, legal organizations, election watchdogs, church organizations, and other sectors – all counselling the COMELEC to do a 100% manual audit, since this is the first time the country is trying Smartmatic’s SAES 1800 PCOS machine.

Even in banks that have automated their operations, to the extent of using a bill counter that counts bills automatically, tellers still count the bills manually before handing out withdrawals to clients. And clients too would again count the money manually before leaving. When a depositor withdraws from an ATM, she would also manually count the money issued by the ATM machine, before she leaves the premises. We all do this, despite the fact that we have no reason to doubt the accuracy of the bill counters and ATMs.

Since we do have a strong reason to doubt the accuracy of PCOS machines, all the more should we double-check its count. Why do we doubt the accuracy of the PCOS?

The Comelec has already acknowledged the misalignment of ultraviolet security marks due to high-speed printing. Because of this misalignment, the Comelec decided on its own, without prodding, to ask election inspectors to conduct a 100% manual audit of the authenticity of the ballots, by visually determining with a UV lamp whether each ballot is authentic or not.

Since the Comelec already admitted that the security mark was misaligned, then it should, at the very least, double-check the possibility that the ovals in the ballot are likewise misaligned. If some ballots have misaligned ovals, then for exactly the same reason that the PCOS machine became unreliable in detecting the security marks, it will also become unreliable in detecting shaded ovals.

This is why we doubt the accuracy of the PCOS machines: they might be fed ballots with misaligned ovals caused by the high-speed printing.

The Comelec specified in its contract with Smartmatic that the PCOS should be able to read shaded marks correctly at least 99.995% of the time (in other words, that it makes at most one error for every 20,000 marks or .005%).

I propose a way to convincingly show, with 95% confidence, that a PCOS machine is 99.995% accurate. A 95% confidence level is the typical confidence level used in statistical tests, product quality control, and scientific experiments.

At this point, the math will get a little bit heavy. But since we are talking of P7.2 billion of taxpayers’ money, let us all hunker down and take the extra effort.

Let us suppose that Smartmatic supplied us with machines whose accuracy rates are all slightly lower than the Comelec specification of 99.995%, say, they make one error for every 19,999 instead of 20,000 marks. We want a set of test ballots that will detect these below-spec machines, because they will make one or more reading errors. Then we can reject these inaccurate machines and have them replaced by Smartmatic because they are all slightly below Comelec specifications (unless of course the Comelec was willing to adjust its specs downwards to accomodate Smartmatic).

If we feed one of these below-spec machines a test ballot with a single mark, we know that the chances of a correct reading is 19,998/19,999 or 99.9949997%, which is practically 100%. We’d expect the machine to read the ballot without error. If we used this ballot as our test set, this below-spec machine, which should fail the test, will pass it instead, where “fail” means at least one reading error and “pass” means zero reading error.

If we feed a test ballot with two marks into the same machine, the chances of two consecutive successes in reading marks are 99.99449997% x 99.99449997% or 99.990%. Our below-spec machine will most probably pass such a test ballot too.

If we feed a test ballot with three marks into the same machine, the chances of correctly reading all three are 99.99449997% x 99.99449997% x 99.99449997% or 99.985%. The machine will again probably pass the test.

Brace yourself: the above can be written as (99.99449997%)^3 = 99.985%, where “^3” means “multiplied by itself three times”.

A real ballot will contain an average of 34 marks (one each for president, vice president, party-list, congressman, governor, mayor, and so on, 12 for senators, etc.).

Hence the chances that the below-spec machine will accurately read a single test ballot with 34 marks correctly are (99.99449997%)^34 = 99.83%. This is still very high. The machine will most probably read this test ballot correctly too, and pass the test.

For ten ballots (340 marks): the chances of accurately reading all of them are (99.99449997%)^340 = 98.3%. Still very high.

Note that 3-7 days before election day, election inspectors, watchers and watchdogs in every precinct cluster will use a ten-ballot test set to determine if the PCOS machines meet the Comelec specification of 99.995%. This means they should be able to detect our below-spec machine, so that it may be replaced by Smartmatic. However, the chances that our below-spec machine will make no reading error and therefore pass the test is 98.3%. Thus, out of a hundred of our below-spec machines, the ten-ballot test will correctly detect two but will pass 98 machines that should have failed. This is obviously not acceptable.

For thirty ballots (1,020 marks): the chances of accurately reading all are (99.99449997%)^1020= 95.0%

For 408 ballots (13,872 marks): the chances are (99.99449997%)^13,872 = 50.0%. If we use a 408-ballot set to test the below-spec machines, half of them will be detected successfully, but half will still pass the test. Not good.

For 1,000-ballot test set (34,000 marks): the chances of our machines passing are now (99.99449997%)^34,000 = 18.3%. Out of a hundred below-spec machines, 18 will still pass undetected. Not acceptable.

Let me now jump to the magic number: 1,762 ballots (59,908 marks). The chances are now (99.99449997%)^59,908 = 5.0%. Out of a hundred, ninety-five of the below-spec machines will fail and only five will pass a 1,762-ballot test set. (In an earlier computation, I had used 99.9944% and got 1,573. A 1,762-ballot test set will detect marginally inaccurate machines better.)

This 95% confidence level is the most common standard used in statistical testing, product quality control, and scientific experiments. If we wanted even better quality standards, we can adopt a 99% instead of 95% confidence level. This will require a 2,708-ballot test set. Then, out of every hundred below-spec machines that Smartmatic delivers, 99 will make at least one error and will be correctly rejected. Only one will pass the test and be deployed for vote counting.(This would still be 822 below-spec PCOS machines out of the 82,200, but that’s probably a necessary risk, unless we want even stricter testing).

Let me now summarize how to test each PCOS machine for accuracy:

Prepare a test set of 1,762 official ballots with 34 marks each, making sure that every oval on the ballot is marked at least once. Tabulate the votes represented by the marks. The tally must be checked and rechecked for 100% accuracy, because it is the standard against which the accuracy of the machine will be measured.

Configure a PCOS machine to accept the ballots, and feed the test set into the machine. If the machine reads all the ballots without error, then we can be certain at the 95% confidence level that the machine has an accuracy rate of 99.995% or better. (If I were the test engineer, I would then repeat the test on every accepted machine, and I would only use as backup those machines that fail on the second test.)

If the COMELEC had done this acceptance testing when the machines were delivered, any machine which did not meet the 95% confidence level should have been returned to Smartmatic for replacement. Thus every PCOS machine deployed for the elections should have passed this test. Since such a test can take around seven hours and a half (assuming 15 seconds per ballot), and Smartmatic claimed that it was testing PCOS machines at the rate of 2,000 per day, it is highly unlikely that these machines went through this kind of acceptance testing. So, we do not know how many PCOS machines actually meet the Comelec specifications.

If Smartmatic is confident about the accuracy of its PCOS machines, and if the Comelec is confident about the alignment of the ovals in the 50.85 million ballots they just printed, they should welcome this 1,762-ballot test as a chance to prove that the PCOS machines truly meet the Comelec specifications for accuracy.

50.85 million ballots printed: how many have misaligned ovals?

The news just came in: the printing of ballots for the May 10 national elections is now complete — all 50.85 million of them.

We already know, by COMELEC’s own admission, that many of these 50.85 million ballots have misaligned ultraviolet security marks.

The niggling question is: how many of these 50.85 million ballots also have misaligned ovals?

For a more detailed discussion of misaligned ovals, see my earlier post, which was also published on April 21, 2010 by Philippine Star, p.18.

Four opportunities to check PCOS accuracy: all taken away by COMELEC

The Filipino voting public had four opportunities for checking the scanning accuracy of the PCOS machine, but the three were all taken away by COMELEC, while the fate of the fourth still hangs in the balance.

Acceptance testing by COMELEC

Everytime Smartmatic delivered a batch of machines, COMELEC should have tested the machines before accepting them, to screen out the lemons. After all, these machines were made in China in a hurry. But even if they were not. It is simple due diligence. The COMELEC specifications are clear in the contract, including the PCOS minimum accuracy rate of 99.995%, or a maximum error rate of .005%. Any machine that did not meet specs should have been returned for replacement. We lost this opportunity because the full results of the testing, if it was ever done by COMELEC at all, were not given to stakeholders like political parties, election watchdogs and the media.

Systest audit and source code review

One of the things Systest would have tested, because it is the heart of the ballot appreciation process, is the scanning accuracy of the PCOS. I had expected a certification from Systest that the PCOS, for instance, met all minimum functional requirements as specified in the Automated Election Law and the contract between COMELEC and Smartmatic. I had expected that the test procedures would be described in detail in the full report of Systest and the test results would be there too, for the scrutiny of all stakeholders. Finally, I had expected the Systest certifications and well as full report to be available to stakeholders like political parties, election watchdogs and the media. COMELEC claimed on Feb. 9 that Systest had finished its audit/review and given its certification, but released no proper Systest document or full report to back up its claim. We lost this opportunity because of the secrecy that surrounds the certifications and full report from Systest.

Voters verification that the PCOS correctly registered their choices

On election day itself, all voters would be in a position to check for themselves the accuracy of the PCOS, by observing whether their marks are properly interpreted by the machine. The PCOS was required by law to show voters their choices (that is, the names of those they voted for), so that they can verify if their choices were correctly registered on the machine. The PCOS, in fact, had this as a built-in feature, and it could show on its screen those choices. This was an excellect feature that empowered every voter to conduct a real-time audit of the accuracy of the machine. We lost this opportunity because COMELEC ordered Smartmatic to disable this feature.

Manual audit of the machine results

The ballots will be counted manually by an audit team, and this count compared with the machine count. Any discrepancy, after the manual count is double-checked, will mean an inaccurate scan by the PCOS, allowing us to actually measure its accuracy rate. We fear that this opportunity may be lost because COMELEC decided to conduct the manual audit of only 1.5% of precincts, and the machine winners may be proclaimed ation anyway, while the audit is still going on. If the machine winners are proclaimed while the audit is still going on, then the audit results won’t matter anymore. We all know how long election protests take to resolve, if they are ever resolved at all. Who would be interested in a futile exercise that is already moot and academic? The rush to proclaim within 48 hours (by May 13), is misplaced because the term of outgoing officials will end on June 30 yet. The manual audit can be given a June 22 deadline, the winners proclaimed on June 23, and still have one week to spare.

If 100% manual audit of UV marks is ok, why not 100% manual audit of ovals?

The question that bugs me is why would COMELEC plug every loophole to make sure that no test results from acceptance testing or the Systest audit/review can be used to question PCOS accuracy, that no inaccuracies are observable to voters on election day, and that when the inaccuracies are eventually discovered through a manual audit, it will be too late to correct them because the machine winners have been proclaimed.

When COMELEC found out that the ultraviolet security marks were misaligned, it ordered a 100% manual audit of every ballot before winners are proclaimed. COMELEC did so on its own, without any prodding.

If the ballot ovals are as misaligned as the ultraviolet security marks, by the same logic, the only solution is a 100% manual audit of every ballot before winners are proclaimed.

Three things the COMELEC can do to assuage our concerns

This is the latest letter of HALAL to the COMELEC. The HALAL suggestions are important and urgent. Let us all add our voices to these suggestions.

Halalang Marangal

April 20, 2010

The En Banc

COMMISSION ON ELECTIONS

Palacio del Gobernador

Intramuros, Manila

Hon. Chairman JOSE A. R. MELO

Hon. Commissioner RENE V. SARMIENTO

Hon. Commissioner NICODEMO T. FERRER

Hon. Commissioner LUCENITO T. TAGLE

Hon. Commissioner ARMANDO C. VELASCO

Hon. Commissioner ELIAS R. YUSOPH

Hon. Commissioner GREGORIO Y. LARRAZABAL

Gentlemen:

We are deeply concerned that the risk of failure of the Automated Election System (AES) remains unacceptably high. But it is very late in the day to return to the old manual system, which may not save us either. It will simply mean business-as-usual for the cheats, who have mastered the manual system so well that they can manipulate its results in their sleep.

Halalang Marangal (HALAL) proposes three simple things for the COMELEC to do that can help lead us out of this dilemma.

THREE THINGS WE ASK FROM THE COMELEC TO ASSUAGE PEOPLE’S CONCERNS ABOUT THE AES

ONE. We ask the COMELEC to assign every scanning machine and its associated precinct cluster a simple unique identifier and widely disseminate this information to the public.¹

Why? Because this will make the Precinct Count Optical Scanner (PCOS) machines more accountable, just as vehicles with license plates, or policemen with visible name plates, are more accountable. It empowers the voting public better when they know exactly the individual machine they are dealing with, making it easy to identify errant machines, send reports, complaints, etc. The unique identifiers of all machines in a polling center must be conspicuously displayed outside the center, in big characters. The ID of each PCOS must also be conspicuously displayed in the room where the machine is located. A unique identifier, known by all, is a fundamental requirement in any automation project.

These unique PCOS IDs must be quickly added to the COMELEC Project of Precincts (POP), which is the master list of all polling precincts. The updated POP must be made available to the media and the public on a compact disc (CD), for quick copying and dissemination.

TWO. We ask the Comelec to authorize election inspectors to print before transmission 29 of the 30 Election Returns (ERs).

Why? To make as many copies of the precinct results as possible, before the connection with a central server and its associated possibility that the authentic PCOS data may be overwritten with false data from the server during transmission. Also, to remove any cause for PPCRV and NAMFREL, Nacionalistas and Liberals, or national and local candidates of the same party, to dispute who should get ER originals. There may not be enough with eight copies, there will be enough with 29 copies. With so many copies of the original ER circulating, suppressing the truth will be much harder.

The printed ER should contain the machine’s unique ID. The ER originals can now go to more political parties, election watchdogs, media organizations, and other accredited civic groups.

The 30^th copy of the ER, printed after transmission, should be immediately read aloud by the BEI so that everyone who had received an earlier copy may make sure that their copy is identical with the 30^th copy. If not, then the transmission has compromised the election results in the PCOS memory as well as the 30^th copy. These, and the electronic ER copy just transmitted to the central servers, must now be presumed to contain false data and must be questioned. This must be noted in the minutes. But the 29 original ERs still contain the authentic data.

THREE. We ask the COMELEC to authorize a 100% manual audit of the votes for president.

Why? As a final check on the accuracy of the PCOS count, for the same reason that a bank teller still manually counts bills after they have been counted by a machine. The COMELEC reverted to a 100% manual audit of ballot authenticity after it was discovered that the high-speed printing caused 1-2 mm misalignment which resulted in the inaccurate scanning of the UV security mark. For exactly the same reason, we need a 100% manual audit of the votes, to find out if a similar misaligment of ovals due to high-speed printing also resulted in the inaccurate scanning of marks. Without the PCOS feature that allows each voter to verify if his choices were correctly registered by the machine, a 100% manual audit is our only remaining option to check the scanning and counting accuracy of the PCOS. Earlier proposals that have been raised for a 100% manual audit covering three positions only were in the right direction. Our counsel to limit further the coverage of the audit to the president is meant to streamline the process even more, by making possible a very simple manual method as follows:

The BEI will first count the ballots, and then sort them into separate stacks, one stack per presidential candidate. The stacks should be double-checked by individual watchers and elections watchdogs.² The BEI will then count in public, aloud, the ballots in each stack. If the counts are correct, then the total of the counts will equal the number of ballots. If no discrepancy is found, we estimate that this method will take no more than one hour. If a discrepancy is found, it may take another half hour to confirm the accuracy of the manual count.

We emphasize that extra care must be taken to make this manual audit 100% accurate, because it will serve as the standard against which the accuracy of the PCOS machine will be measured. The stacks and the final vote counts must be checked several times to detect and correct any errors. The results of this manual count of votes for president, including the total votes and any discrepancy in the totals, must be appended to the printed ERs and countersigned by the BEI. The results of the manual count will also be entered in the minutes. They should be taken into account before any winner is proclaimed. After all, the term of outgoing elected officials will end only in June 30. There no need to rush any proclamation.

Airplanes and ships are required by law to carry life vests and life boats, even if these are superfluous most of the time because air and sea accidents are rare. They are there not only to assuage passenger concerns, but also because of the lives they will save in those rare events when they are needed.

Similarly, we ask the COMELEC for the following safety mechanisms: unique PCOS identifiers made public, 29 ERs printed before any transmission, and a 100% manual audit of the votes for president before proclaiming any winner.

We hope the COMELEC will consider these HALAL recommendations for the 2010 elections, which we are now formally submitting to the COMELEC in the spirit of supporting the Commission’s efforts to protect the integrity of the ballot and the sanctity of the people’s voice.

Very truly yours,

Halalang Marangal Convenors/Board of Directors

WIGBERTO TAÑADA Chairman, HALAL Former Senator Senate of the Philippines	MEHOL K. SADAIN Former Commissioner Commission on Elections	FRANCISCO GUDANI President, HALAL Retired General Armed Forces of the Philippines
ISAGANI SERRANO President Philippine Rural Reconstruction Movement	Sr. MARY JOHN MANANZAN Association of Major Religious Superiors	MA. PAZ LUNA TOYM Awardee
	ROBERTO VERZOLA Secretary-General, HALAL

1If a unique ID per PCOS already exists, then the COMELEC need only publicize them widely. Smartmatic is apparently using a Stock-Keeping Unit (SKU) system that uses seven digits for internal control. This is enough. The important thing is that the identifiers are unique per machine, that they are consistently used in all official lists and ERs, and that they are publicly known and, on election day, conspicuously displayed outside polling centers and in the precincts.

2This is how some countries do their manual count. After voting closes, all ballots for one electoral jurisdiction are transported to a big hall or covered court, authenticaled, and then dumped on the floor. “Scrutineers” then sort the ballots into stacks, one candidate, one stack. Double-checked, of course. If the contest is not very close, the heights of the stacks will show before the counting starts – even before the sorting is over – who won as member of parliament in that jurisdiction. On the same night, winners in most juristictions would be known, as well as the majority party if any, and then of course the prime minister – a clean and honest count, without automation.

The issue: failure of automation, not failure of election

The issue is not a failure of election, but a failure of automation.

“Failure of election” is a narrow legal term describing a rare situation. The Omnibus Election Code defines it as a situation in which “the election in any polling place has not been held on the date fixed, or had been suspended before the hour fixed by law”. The suspension may also occur “after the voting and during the preparation and the transmission of the election returns”. The definition further requires that the failure “would affect the result of the election”, or “results in a failure to elect”. It has occurred in barangays or towns but never on a national scale.

Although some have raised its possibility in 2010, they were probably using the term loosely and were not aware of its legal definition.

Thus, Chairman Melo could say, with a straight face, that failure of election was “pure fantasy”. He is using its narrow legalistic definition. If voters were able to cast their votes and Comelec proclaimed a winner, there was no failure of election.

The issue in 2010 is the high risk of a failure of automation. This is what was raised by Halalang Marangal, an election watchdog which includes former Senator Wigberto Tañada, former Comelec Commissioner Mehol Sadain, and retired General Francisco Gudani among its convenors. We had in fact estimated the probability of failure as of March 8 at 75%, and we have seen no reason to substantially revise that estimate. We still consider the risk of failure “unacceptably high”.

Let me define what we mean by a failure of automation.

Election automation is a failure if the time it takes to determine the winners in the election is not significantly shorter than the manual method, or if the fraud that has chronically attended our elections is not significantly reduced.

Let me review the basis of our assessment that the election automation had a 25% chance of success. (You can find the details in Jarius Bondoc’s April 5, 7 and 9 columns in this paper.)

A March 8 full-page ad by Smartmatic in major national dailies had claimed “a vote of confidence” on the election automation project it was implementing in the Philippines. Smartmatic had claimed substantial achievements in the five sub-systems that comprised the whole Automated Election System (AES).

But when we scrutinized carefully the Smartmatic ad, we found gaps, delays, problems and at least one glaring false claim (“successful field tests and mock elections”).

In the Hardware sub-system, Smartmatic claimed they have completed the delivery of the machines, but glaringly omitted reference to testing. Clearly they have not tested the machines thoroughly. Neither did they have the time to do so. Former Comelec commissioner Mehol Sadain tells us that in 2004, it had taken them three months to fully test 1,990 automated counting machines. If deployed, some of the partially-tested machines are bound to cause problems on election day. We also found that Smartmatic had bought 21% more memory cards than necessary. In the wrong hands, these could be loaded with false precinct results and substituted for authentic cards. Because of these and other problems, we estimated the probability of success of this sub-system at 80%.

In the Software sub-system, we noted that no local stakeholders had managed to conduct a proper review of the source code, because of the Comelec’s obstinate refusal to implement the clear intent and letter of the law. We also noted that the Comelec released no certification documents or full report that would support its Feb. 9 claim that Systest Labs had completed its audit/review of the AES. Since time had run short for a thorough review, we estimated the probability of success of this sub-system at 70%.

For the Logistics sub-system, we cited media findings about the questionable capacity of the forwarders chosen by Smartmatic to deliver election paraphernalia throughout the Philippines. We estimated the Logistics probability of success at 80%.

For the Transmission sub-system, we cited among other things the 70% signal coverage in the Philippines, as Smartmatic itself found out. Smartmatic had transmission problems even within Metro Manila, suggesting poor quality of transmission equipment. We gave it 70%.

For the Ballot Printing sub-system, we cited the confidential Comelec memo which warned that it was impossible to finish ballot printing on time, given the rate they were printing them. We gave it 80%. The Comelec subsequently brought in a fifth ballot printer, raising its capacity by 20%, and making it possible – if no further glitches happened and the printing went on non-stop – to meet its April 25 deadline.

To get the overall probability of success of the entire AES project, the sub-system probabilities of success must be multiplied together. Yes, multiplied together, not averaged. And not just the lowest figure – the weakest link – either. Check it with your calculator: .8 x .7 x .8 x .7 x .8 = .25 or 25%. Note that we see the glass as one-fourth full, not three-fourths empty. We are optimists to a fault, not doomsayers.

So many things can go wrong with the AES that Murphy’s Law is bound to kick in. Like a toss of two coins, Chairman Melo is betting that two heads will come up. He bet P7.2 billion of the taxpayer’s money. If you count the whole election budget, P11 billion, all in.

Chairman Melo’s bet will lose 75% of the time. That makes failure of automation the issue in 2010.

The time is very short, but we still have a possible solution. Details in subsequent posts.

The high-speed printing that misaligned UV marks can misalign the ballot ovals too

The ultraviolet (UV) scanner of the PCOS was disabled because the high-speed printing resulted in the UV mark on the ballot being “misaligned by one to two millimeters”, according to COMELEC Commissioner Larrazabal. As a result, the machine’s UV scanner often missed the mark and many valid ballots being being rejected.

Here’s the big question: If the misalignment of the UV mark was serious enough to make COMELEC turn off the UV scanner, then can’t the ovals be misaligned too? A misaligned oval means a misaligned a vote-mark. Which means the PCOS main scanner may have problems interpreting the voters’ choices.

The Comelec requires from the PCOS an error rate of less than .005%. That means less than five errors for every 100,000 marks. To determine if this Comelec specification is met, each PCOS should have been tested properly. But no test statistics have been released by the Comelec. And if these tests were done at all, they were probably done with perfectly aligned, not misaligned, ovals.

With misalignment, the two types of errors the scanner can make will both get worse: the false positives that register a vote/mark which is not there, and the false negatives that miss a vote/mark which is there. This means some candidates will gain votes (“dagdag”), while other candidates will lose votes (“bawas”). Does that sound familiar?

This problem is made worse by the Comelec decision to, in effect, blindfold voters while the machine is registering their choice. Originally, the PCOS was programmed to display the voter’s choices on its screen, so he can check if his choices were correctly registered by the PCOS and abort the process if the PCOS didn’t. This voter-verification is in fact required by the Automated Election Law (Section 7n): “Provide the voter a system of verification to find out whether or not the machine has registered his choice.” The Comelec ordered Smartmatic to disable this feature. Thus, the PCOS may falsely register voters’ choices, without voters knowing it. This is far worse than a PCOS that stops working or rejects valid ballots, problems which are apparent at once. An inaccurate PCOS will keep scanning and counting happily, with no indication or warning that it is miscounting votes.

According to a former Comelec official, similar problems had also occurred when the Comelec piloted automation in ARMM in 1998. He showed me a Comelec report entitled “Partial Automation of 1998 National and Local Elections”, which is also on the Comelec website. In 1998, said the report, Sulu ballots had to be manually recounted “due to an error in NPO’s printing procedure”. A similar recount was done with Lanao del Sur ballots, “also due to errors in printing of the ballots.”

The Comelec blamed “high-speed printing” for the misalignment. But misalignment is a common printing problem, and good operators know how to correct it. Why would it happen in such an important job as the printing of ballots?

It seems that the ballot printing is being done in such a hurry, that operators are not getting the time needed to stop the high-speed machines and correct any misalignments that may be happening. That the Comelec had insisted on printing ballots even on Maundy Thursday and Good Friday shows how disastrous even a slight delay might be. If they stopped the machines too often to correct for misalignments, they might miss their deadlines. Thus, the UV misalignment problem has gone on uncorrected.

As of April 16, according to Smartmatic, the NPO has printed 43.7 million ballots. If the high-speed printing has misaligned the UV marks, then how many of these ballots have misaligned ovals too? And in these misaligned ovals, how many false “dagdag-bawas” interpretations by the PCOS will occur?

The misalignment of UV marks and possibly the ovals too are the consequences of the Comelec violating the law and using a machine that has neither been piloted nor used widely. The SAES 1800 has never been piloted in the Philippines, as Section 6 of the Automated Election Law requires: “… the AES shall be used in at least two urbanized cities and two provinces each in Luzon, Visayas and Mindanao.” Nor has the PCOS been used widely in any other country, as Section 10 of the law also requires: “… the system procured must have demonstrated capability and been successfully used in a prior electoral exercise here or abroad.” The Supreme Court’s support for the Comelec’s stubborn insistence in using unpiloted Smartmatic machines must have reinforced the Comelec’s sense of impunity in violating provisions of the law.

Because of the lack of pilot, we lost the chance to detect early problems like these. As a result, they have put the entire national elections at risk.

Solution: both the PCOS and their associated ballots must be thoroughly checked for misalignment and accuracy in scanning. The tests must be done not by Smartmatic, but by independent third parties, say, the DOST, and witnessed by all stakeholders.

What if careful testing shows that the PCOS cannot reliably read properly shaded ovals? Then we may have no choice but to manually count the votes again, ballot by ballot.

[Note: This piece is based on the Halalang Marangal April 17 Statement on the problem of potentially-misaligned ovals, which I also drafted. It was published by the Philippine Star, p.18, on April 21, 2010.]

Roberto Verzola has a background in engineering and economics and a passion for social issues. He is recognized by the IT industry as an Internet pioneer in the Philippines and is often tapped by NGOs for technical advice. He currently lectures at the Institute of Mathematics of the University of the Philippines and is a convenor and secretary-general of the election watchdog Halalang Marangal (HALAL).

NEVER AGAIN releases its first presentation

This quickly-done presentation, Never Again How-To, summarizes the concept of a national election vigil. You can freely use it to explain to others the objectives and activities of the movement.

The National Election Vigil over ERs Against Cheating (NEVER AGAIN) is being initiated by Halalang Marangal (HALAL), and everyone is welcome to join.

We will improve the presentation as we go along. You are also welcome to
add improvements. If you do, please share with us your improved version, so we can also update the file.

If some of you are good at making animations or video, you may want to help us turn this into a video, which we can upload in YouTube.

The Comelec 100% manual audit of ballot authenticity

On its own and without any prodding, the Comelec decided to conduct a manual audit of ballots to determine their authenticity. The Board of Election Inspectors (BEI) will do the audit by shining an ultraviolet (UV) lamp on each ballot and deciding, based on their visual appreciation of the ballot, whether it is authentic or fake. This audit is not random, but 100%. It is not automated, but manual. It will be conducted not after, but before proclaiming winners.

What made the Comelec decide to conduct this 100%, manual, pre-proclamation audit of ballots for authenticity? They discovered that the high-speed printing had caused the UV security mark on the ballots to be “misaligned by one to two millimeters”. As a result, the voting machine’s UV scanner was inaccurately reading the misaligned mark and making wrong decisions whether the ballot was authentic or fake. For instance, here’s an April 5 report from GMA News:

The Commission on Elections (Comelec) on Monday denied that poll machine provider Smartmatic-Total Information Management (TIM) failed to supply the correct ink for the ultraviolet (UV) security markings that are being printed on the ballots for the May polls.

“There was no admission on the part of the Comelec,” said Comelec Commissioner Gregorio Larrazabal, who heads the poll body’s steering committee on automation.

Earlier, the Comelec said it will be using hand-held UV lamps to verify the authenticity of the ballot after the high-speed printing of the ballots caused the UV marks to misalign, making it hard for the Precinct Count Optical Scan (PCOS) machine to read the security feature.

Thus, the Comelec shifted to a 100% manual audit of the ballot’s authenticity.

If the ultraviolet mark was misaligned by one to two millimeters due to high-speed printing, then, for exactly the same reason, shouldn’t we be concerned that the ballot ovals may be misaligned by one to two millimeters too?

However carefully voters will shade the ovals, if these ovals are misaligned, then the voters’ marks will be misaligned too. Just as misalignment caused the UV marks to be registered unreliably, misaligned shaded ovals, representing the voters’ choices, will also be scanned and registered unreliably.

An unreliable scanner makes two kinds of mistakes. The first is a false positive, when it reads a mark that is actually not there. The second is a false negative, when it misses a mark that is actually there. A false positive is, in local parlance, a “dagdag” and a false negative is a “bawas”.

As designed, the machine allowed the voter to verify first, if the machine correctly registered his choice. This would have enabled voters to catch any inaccuracy by the voting machine in registering their choices – a requirement of the law, in fact. But the Comelec has disabled this built-in feature of the machine.

The misaligned UV mark could be detected by voters, because the machine rejected the ballot, which alerts the voter to the problem. The Comelec therefore had no choice but to correct what would have been very obvious to voters, watchers and election inspectors: machines rejecting too many valid ballots.

But misaligned marks will not be detected by voters, because the machine will not inform voters of their choices, but will simply thank them and then record any “dagdag-bawas” that has occurred as if these were the voters’ choices. If the Comelec chooses to ignore this problem, no one will notice on election day.

According to the Comelec, the printing of the 50.7 million ballots is nearly done. Ahead of schedule, the Comelec says. That’s because no untoward incident delayed the process. The printing was such “high-speed” that it misaligned the UV marks, which will now have to be detected through a 100% manual pre-proclamation audit. But how do we distinguish now between, on one hand, the good ballots and, on the other hand, the bad ballots with ovals in varying degrees of misalignment?

One solution is to re-enable the voter verification feature. But it is probably too late for that. In fact, it was too late to fix the UV scanning problem, the Comelec said, so they simply disabled the scanner.

And they switched to a 100% manual pre-proclamation audit to determine the authenticity of each ballot using hand-held UV lamps.

Isn’t the solution to a similar misalignment of ovals obvious?

PCOS machines in Philippine automated elections: failure rates, error rates

According to the news, two of the twenty PCOS machines in Hongkong stopped working for a while. That is a 10% failure rate.

Cesar Flores of Smartmatic claims they expect a PCOS failure rate of 0.3- 0.5%. However, vendor claims must be taken with a grain of salt, more so if their goods were hurriedly made in China. The claim is also belied by Smartmatic’s own plans: they are deploying 8% of the total machines for backup. So, they must be expecting up to 8% of the machines to fail, which is more consistent with the failure rates in Hongkong.

The actual PCOS failure rate is, in fact, a big unknown.

First, it appears that Smartmatic had done most of the testing, not COMELEC. Due diligence requires COMELEC to do acceptance testing. Any buyer must double-check delivered goods before signing a receipt acknowledging that it was received in good working condition. Especially since the Smartmatic deliveries involved P7.2 billion of taxpayers’ money, the machines should have been independently tested if they meet COMELEC specifications as detailed in their contract. Those that didn’t meet specs should have been returned for replacement. If deployed, these can cause trouble during election day itself, as they did in Hongkong.

The 0.3-0.5% PCOS failure rate that Smartmatic claims is not backed by properly-witnessed test stats and is contradicted by the 10% failure rate reported in Hongkong and Smartmatic’s own preparations to replace up to 8% of machines that may fail on election day.

Second, the test stats have remained inaccessible to third parties like political parties, election watchdogs and media. Transparent test stats minimize potential insider collusion (as in the ballot secrecy folder contract), which can result in overpricing or payments for sub-standard equipment. Transparency also minimizes the possibility that insiders will selectively assign machines, depending on their quality, in order to cause trouble in targetted regions or provinces in favor one candidate or another. Just imagine if problemmatic machines or modems are selectively assigned to Aquino, Villar, Estrada, or Teodoro bailiwicks — whoever are disfavored.

At least five PCOS test results are so important that they should be publicly known:

• Mean time between failures (MTBF). This is the average time a PCOS machine stays operational. Knowing the MTBF and the mean time to repair (or replace), we can determine the average failure rate. Instead of actual statistics, we have today media-reported field anecdotes and unsubstantiated vendor claims.

• Average rejection rate of valid ballots. This is a specific but important case, when the PCOS stays operational but rejects a valid ballot. In Smartmatic demos, field tests and mock elections, the rejection rates were inordinately high, far above COMELEC specs.

• Scan error rate. Just as PCOS machines fail, they make mistakes too. A PCOS scanning error can be a false positive (registering a vote that is not there) or a false negative (missing a vote that is there). When the PCOS is adjusted to read lighter shades, false positives increase because even a slight smudge may be falsely registered as a vote. When adjusted to read darker shades only, false negatives increase, because lightly or partially shaded ovals may be missed by the PCOS. Each machine has to be calibrated towards that ideal spot which minimizes the total errors from both false positives and false negatives. Based on COMELEC specs, this total should be lower than .005%, or five scanning errors for every 100,000 marks (at most one error per 1,000 ballots). Unfortunately, the calibration may change in transit or under environmental stresses like heat, humidity, or mechanical shocks. A PCOS machine that rejects valid ballots has, in effect, very high false negatives, because it misses all the shaded ovals, each representing one vote, in those rejected but valid ballots.

• Transmission error rate. Because of ambient electrical and electronic noise, transmission is more susceptible to error than scanning, and therefore demands high quality equipment. That Smartmatic modems had transmission problems even within Metro Manila does not speak well of their quality. A poor quality modem is hopeless and should be replaced.

• Battery backup life. COMELEC specified at least 16 hours of backup. A good quality control engineer would insist on batteries lasting up to 20 hours under test, a 25% margin for coping with unexpected operating and environmental extremes.

Smartmatic had earlier claimed it was testing 2,000 machines a day. Compare this to the three months it took COMELEC to thoroughly test some 1,900 automated counting machines in 2004. Even granting that the 2004 testing was done at a leisurely pace, the huge difference still makes one wonder how thorough the PCOS testing was.

In particular, the PCOS scan error rate is very important. If the error rate is, say, 5%, and the presidential winner’s margin is less than 5%, then we will again find ourselves in political limbo. In 2004, GMA’s supposed margin over FPJ was 3.48%. COMELEC specified .005%, which is quite low. But it doesn’t look like COMELEC actually measured each machine’s error rate. That is not possible when testing 2,000 machines a day (the necessary statistical test requires more than 1,700 test ballots per PCOS).

Each PCOS should pass various COMELEC tests before it is accepted, paid for, and deployed to a polling place. And stakeholders should have access to all the test statistics, including the number of machines that stopped operating, the number of valid ballots rejected, and the actual number of falsely registered voter choices, to prevent insiders from accepting bad machines and selectively assigning these to targetted areas.

Without the test statistics, we can only guess which is closer to the truth, the 10% failure rate shown by the machines in Hongkong, or the 0.3-0.5% failure rate claimed by Smartmatic.

It is not too late. COMELEC can still order the release to media of these test statistics, and improve its credibility before the voting public.

COMELEC officials learn from GMA

If there one lesson Comelec officials could have learned from GMA, it is this:

When caught in the act, brazen it out.

GMA did so when caught in the act, on tape, micro-managing the cheating in Mindanao.She did it again when caught with her fingers on the ZTE pie.

With the help of an improbable rebellion charge, the Ampatuans are trying to do the same.

Comelec officials linked to the aborted contract to buy overpriced “ballot secrecy folders” at P380 each are taking the same tact.

Admit nothing. Give no quarters. The public has a short memory. Everything will soon be forgotten. Wait for the brouhaha to die down. It will soon be business as usual.

A culture that has gradually become worse among wrongdoers since GMA grabbed power is the culture of impunity.

And it is rooted in GMA’s power grab. That she could get away with it in violation of the constitution. That she could get caught planning a one-million vote padding operation, including the kidnapping of a minor election clerk, and get away with it. Caught on tape, her very own grating voice captured for posterity forever. What more evidence can a prosecutor ask for? But she got away with it. That’s impunity.

And that’s what keeps those Comelec grafters in their positions, despite having been caught in the act.

Front side of the official ballot for the May 10 elections in the Philippines

Here’s a copy of the front side of the official ballot that will be used for the May 10, 2010 national elections in the Philippines, for everyone’s scrutiny.

By the way, the Comelec is being accused of bias in the ballot design. Disqualified candidate Vetellano Acosta (#1 on the list of presidential candidates) is still listed in the ballot. The Comelec had approved Acosta’s candidacy earlier, then subsequently disqualified him. As a result of Acosta’s inclusion, LP candidate Noynoy Aquino, who would have been first on the list, is now a less conspicuous #2, and NP candidate Manny Villar is now all alone in the fourth column.

You decide for yourself, whether this is bias or not on the part of the Comelec.

It’s a long ballot. Be careful when filling it up. You cannot ask for an extra copy.

More stink at the Comelec: 1.68 million folders for P0.638 billion?

And I thought P1 billion for some 80,000 UV lamps stank so much that you could smell the overprice a mile away.

This one, you could smell 10 miles away!

[UPDATE: A more recent story by Malaya says the Comelec has rescinded the contract. Just like that? Nobody will be prosecuted for approving and awarding the contract?]

The full report by GMA News is here. This is the gist:

Election officials had bought nearly two million ballot secrecy folders for the May 10 elections at an overpriced rate of P380 each, Senate Minority Leader Aquilino Pimentel Jr. alleged on Sunday.

Citing information from a confidential source, Pimentel claimed there had been no bidding for the folders, which will be used to cover the ballot during voting. He declined to name the company that had bagged the contract from the Commission on Elections (Comelec).

“I want Chairman [Jose] Melo to investigate this matter because this will taint the integrity of the Comelec. It looks like more than P300 per piece is too much… I cannot name my source because doing so will endanger his life,” Pimentel told GMANews.TV in Filipino.

Under Comelec Resolution 8786, there should be 22 folders per clustered precinct. This means the poll body must shell out P638 million for 1.68 million folders for the 76,340 clustered precincts in the upcoming elections.

May 10 election automation: beware of selective deployment of good and bad equipment

Election Day is fast approaching, yet we have gotten zero statistics so far on the PCOS machine error rates, in particular:

• the average number of operating hours before a machine fails
• the average number of valid ballots the machine rejects
• the average scan accuracy rate of the machine
• the average failure rate of the transmission equipment

These secrets appear to be so well-guarded that no media, political parties, election watchdogs, much less Comelec or Smartmatic has cited or released credible statistics about these test results.

Yet, they must exist. After all, due diligence requires that Smartmatic do these tests thoroughly before accepting the PCOS machines from Chinese factories that manufactured them. Due diligence also requires that the Comelec test these machines independently, before accepting them from the vendor Smartmatic. Where are the results of all these tests? We want to know if indeed the machines meet Comelec specifications, or if they are of poor quality, as some made-in-China products are. We do know that Smartmatic field tests have been plagued with quality problems: its machines reject too many valid ballots; its modems have transmission problems even in areas with strong signals, like Metro Manila.

Why do we want these tests made public?

• We don’t want the Filipino taxpayers money wasted on lemons. If a machine does not meet Comelec specifications and deemed of poor quality, what’s the point shipping it to Batanes or Tawi-tawi, where it will only cause trouble? In fact, why should the Comelec pay for an off-spec machine, which is worse than useless, because it can disenfranchise voters or even misrepresent their intent?
• If Smartmatic doesn’t have enough good quality machines (after all, they were made in China a hurry), we should know early enough, so we can take precautionary measures and prepare a manual backup system.
• We don’t want Smartmatic or Comelec selectively deploying good or bad machines to certain regions or provinces in a way that can bias the election in favor of some candidates, against others.

A second chance to determine the accuracy of these machines would have been the system audit and source code review conducted by Systest Labs. Smartmatic and the Comelec claim that the audit/review is done, and they have obtained the necessary certification from Systest. However, they have released neither the proper certification documents nor the full text of the Systest report, leading us to believe, that the audit/review is either not complete yet, or it is unable/unwilling — perhaps due to some adverse finding — to give the full certification to the Smartmatic machines and software as required by law.

If Smartmatic and the Comelec continue to keep results of these machine tests and third-party audits/reviews secret, there would have still been a third chance to determine to accuracy of these machines — on election day itself. By law, voters have a right to see whether the machine interpreted their choices, as reflected in their marks, properly. The machine should therefore show on screen the names of candidates which the machine interprets to be the voter’s choices, based on its scan. This way, every voter would have had an idea if the machine was scanning the marks accurately. The Comelec, however, disabled this feature, depriving every voter of their lawful right, and keeping all of us in the dark, as of election day, regarding the true accuracy of the machines.

The last line of defense — our fourth and final chance — in trying to determine the accurate these optical scanners would have been the post-election random manual audit, where an audit team will manually count the votes and compare their results with the machine count. Should there be any discrepancy, the team has to determine whether the error is in the manual count or the machine count. Though it will occur after the elections, we would still get an idea how accurate the machines have been. But again, the Comelec now indicates that it wants the random manual audit conducted after the proclamation of winners, which makes the audit moot and academic.

Seeing that the Comelec and Smartmatic have done everything in their power to keep the public in the dark about the true accuracy of the PCOS machines, leads us to very strong suspicions that:

• They simply do not want the public to know the true accuracy of the machines
• Given our limited anecdotal experience with these machines, some of them do not meet the minimum Comelec specifications. Yet, by accepting them, the Comelec is binding the Philippine government (and Filipino taxpayer) to pay for them.
• A sinister purpose drives this secrecy because it gives those who know which machines are good and which are bad the power to influence the outcome of the elections by selectively deploying good and bad machines to favor some candidates over others.

These are very bad signs indeed.

May 10 election automation: watch out for “ghost precincts”, “ghost machines”

One method of election cheating that can be implemented either under manual or automated elections, and is hard to detect, is what we in Halalang Marangal (HALAL) call “ghost precincts” (under an automated system, “ghost machines”).

Imagine a typical school with, say, 20 classrooms, each one turned into a voting precinct. On election day, if one took the effort of counting all the precincts in that school, making sure none was missed, 20 precincts would be counted. Each precinct will have its share of election inspectors, watchers, voters, kibitzers, etc. All these live bodies would be visible on election day.

After the voting is over, the polls close, the votes are counted (manually or by machine), and the Election Returns (ERs) are sent to municipal or city canvassing centers (through government vehicles or electronic transmission) for canvassing and consolidation.

What if you kept careful track of every ER coming to the city or municipal canvassing center and inexplicably counted 25 instead of 20 precincts coming from the same school. Each of the extra five ERs would likewise have its share of inspectors, watchers and voters — every signature in order. Yet, you are sure you counted only 20 precincts when you checked every room in the school. (Or are you in fact sure? Is it possible you missed a few rooms somewhere?)

This is the problem of ghost precincts. In an automated setting, ghost machines need not even physically “arrive” at the city or municipal canvassing center. They will just pop up in the consolidated tally, their results having been transmitted electronically. These ghost machines could be housed anywhere, spread out in a number of safehouses, or housed in a central facility, sending authentic-looking reports, backed up with possibly authentic ballots in authentic ballot boxes. Where could these machines have come from? Possibly from the Chinese manufacturer, or possibly from the vendor. Certainly from insiders.

They may even turn up already grouped into “ghost voting centers”, when they integrate themselves among the living during the canvassing and consolidation process.

There is only one way to detect their presence (actually, their absence): on election day, one must explore every nook and cranny of every voting center, and count every precinct and machine that is actively processing voters and subsequently counting votes. One must then compare this list against the list of precincts (or machines) that report results during the canvassing and consolidation process.

Any extra precinct or machine is a ghost.

These ghosts are hard to detect because the window of opportunity for detecting them is a narrow one: only on election day. In addition, few political parties especially in national contests have the organizational capacity to actually double-check the physical existence of every precinct and school in every city and municipality. There are always gaps in their reach. In these gaps, the cheats can operate, creating ghost precincts and possibly even ghost schools, to produce thousands of fraudulent votes for their candidates.

But there is no other way. To detect ghost precincts, one must be armed with the complete Project of Precincts (the official Comelec listing of every precinct in the country — which may in fact include ghost precincts) and be patient enough to confirm the existence of each one on election day.

If you find a precinct (or school!) that does not come alive on election day, you’ve busted a ghost.

Get yourself a CD copy of the Project of Precincts, and on May 10, try ghost-busting.

May 10 election automation: can data-substitution happen during transmission?

There are several entry points for election cheats under the election automation project of the Comelec. We will focus here on the transmission phase.

Every precinct counting machine (PCOS) is supposed to transmit its electronic Election Return (e-ER) to the three upstream servers: the municipal canvassing server, the KBP-PPCRV-political parties server, and the Comelec central server, in that order.

The risk of reverse data flow (RDF)

Why does the municipal server come first, and the Comelec central server last? Here’s the risk if the Comelec central server comes first: suppose that during transmission a reverse flow of data actually occurs? That is, instead of receiving data, the central server instead sends to the PCOS, overwriting the latter’s authentic election results with fraudulent data coming from the central server. We will call this the Reverse Data Flow (RDF) risk. It can only happen if both the PCOS and the central server had earlier been programmed to do so, upon receipt a certain command (for instance, if the PCOS receives a certain string of characters from the central server). We know that Smartmatic machines have such capability and can be commanded to accept incoming data, because it happened during the 2008 pilot in ARMM. (Those in the industry call this the Wao incident. Wao is a town in Lanao del Sur Province.)

If this RDF risk materializes, then, when the PCOS subsequently connects to the municipal and the KBP-PPCRV servers, it will now be uploading not authentic data but the fraudulent data it received from the central server. To be effective, RDF needs to occur on the first connection to the outside (presumably with the central server). Why? Suppose the authentic data from the PCOS manages to get out on the first connection to the municipal or KBP-PPCRV server. If the central server subsequently manages to load the PCOS with fraudulent data through RDF, then discrepancies will show up between the municipal and central data files that will be harder to cover up.

RDF can also occur between a PCOS and a municipal server, but this means the cheats would have to take control of many municipal servers, instead of a single central server, to achieve a similar impact. Thus, RDF through the central server is simpler and easier to cover up, if cheats were to attempt it. This is why it is extremely important for the PCOS to connect to the municipal server first, and the Comelec central server last.

How to make the PCOS connect to the central server first

A security flaw in the implementation of the transmission sequence exists, that can be exploited by cheats. In its Resolution No. 8739, the Comelec instructs the Board of Election Inspectors that if the PCOS is unable to connect with the municipal server after three tries, then the BEI should try sending to the KBP-PPCRV server instead. And if that doesn’t work either, they should try sending to the Comelec central server next. Then, back to the municipal server, in round-robin fashion. The revised general instructions (Comelec Resolution No. 8786) keeps this round-robin approach.

Hence, if there’s a way to intentionally block municipal servers and the KBP-PPCRV server from receiving a PCOS transmission for a while, then the PCOS will end up connecting with the Comelec server first, setting up the conditions for the RDF problem.

Remember the 5,000 cellphone jammers reportedly imported into the country? They suit this purpose perfectly. The 5,000 are more than enough to cover the 1,631 city/municipal servers throughout the country, plus the KBP-PPCRV server. If this method of cheating were to be attempted, the cheats will probably not operate in every city and municipality but only in selected municipalities where they can achieve maximum impact with minimum of disruption. Areas where there is no credible opposition might be good candidates for such an operation.

Another possibility is swamp the target server with a Denial of Service (DoS) attack through the Internet, long enough to get the PCOS to try the central server first. After the RDF operation, which will only take the expected few minutes (except that data will be flowing in the opposite direction), things can go back to normal at the municipal and KBP-PPCRV servers.

How can this method of cheating be prevented?

Several measures are necessary to prevent or detect this method of cheating:

1. The source code of the PCOS as well as the servers must be opened for scrutiny and review. RDF can only happen if there are programs in the PCOS and the server instructing them to make it happen. A thorough code review may be able to determine if such rogue programs exist, as long as they are not camouflaged or hidden very well.

2. The Comelec central server must be accessible for close observation at all times, to all stakeholders, especially political parties and non-partisan election monitors such as media and citizens’ groups. This will make it more difficult to set up the Comelec server for an RDF operation or to install new software at the last minute for doing so.

3. The BEI must be under strict instructions not to attempt connection to the Comelec central server until the data has been transmitted to their upstream municipal server and the KBP-PPCRV server.

4. Print more than 8 ERs before any transmission is attempted, to give more minority parties access to one of the pre-transmission ERs. Under current Comelec instructions, only 8 ER copies will be printed before transmission and the remaining 22 after transmission, so only the dominant majority and minority parties (the dominant minority designation is still being contested between the NP and the LP) get a copy each of the pre-transmission ER. Another copy goes to PPCRV, which however has announced no concrete plan so far to do a parallel count, and still another gets posted on a conspicuous place at the precinct level. These first 8 copies are extremely important for detecting RDF.

5. Specifically instruct the BEIs and official watchers to ensure that the ERs printed before and after transmission are identical and to record this fact as well as any discrepancy in the BEI minutes.

Buying P1 billion worth of UV lamps for the May 10 automated elections

[UPDATE 1: According to a more recent report, the Comelec is appropriating P30 million, not P1 billion, for 77,000 UV lamps. That price (P389 per lamp) sounds more reasonable. Now the UV lamp is only as expensive as one Ballot Secrecy Folder, which Sen Pimentel says, may cost the Comelec P385 per,]

[UPDATE 2: According to the same report above, the Comelec has abandoned its “wrong ink” theory to explain why the PCOS machine is unable to scan the UV marks on the ballot. Could it be the PCOS itself that is the problem then?]

It has been reported (see for yourself here, down towards the end of the long report) that the Comelec has appropriated P1 billion to buy the ultra-violet (UV) lamps that will be used by election officials in every precinct to manually check if a ballot is authentic or not.

I can only gape in disbelief. This is what Jun Lozada calls “bumubukol” (roughly translated: excessively bloated)

On one hand, the counting machines themselves (one per precinct) cost P4 billion. These are complex high-tech equipment with all kinds of electronic sensors, motors, devices, chips, and software inside.

On the other hand, think of a table lamp (or flashlight), and replace the bulb with one that gives off ultra-violet light (like those they use in some parties). That’s a UV lamp. They also want one per precinct. (See some examples below.) The Comelec is going to spend P1 billion for that? Some businessmen are going to make a lot of money. And some bureaucrats are going to get a lot of commissions.

I will say it again, election fraud is usually an inside job.

This one involves not votes but equipment. But you can smell the stink of overpricing a mile away.

Let us review the whole story:

Comelec and Smartmatic knew since January that their PCOS machines were rejecting too many valid ballots. We all read the media reports during their field tests and mock elections. Some of us have participated in a few demos. The machines do reject too many valid ballots.

Halalang Marangal (HALAL) wrote the Comelec on Feb. 9 that we wanted to know if the Comelec thoroughly tests the machines before accepting (and paying for) them. We also wanted to get a copy of the test results, so we can analyze these ourselves. No response.

Then we kept hearing about these printing problems — that it was the poor quality of printing which made the PCOS machine reject the valid ballots.

Comelec and Smartmatic eventually gave two explanations. But the explanations are conflicting, so we have not heard the final word on this yet:

• One (see this story for details of the “misalignment” theory by Smartmatic), the printing of the UV security marks by the National Printing Office (NPO) are a millimeter or so off their expected position, and the misalignment is causing the PCOS UV scanner to miss some of the marks. Ok, that sounds reasonable. However, it makes us worry: what if the ovals are likewise misaligned, won’t the PCOS machines misread the votes too? We will set aside this very important question for later. Back to the UV marks first.
• Two (see this story for details of the “wrong ink” theory by Comelec), Smartmatic gave NPO bad ink, and the UV mark was not readable enough by the machine. Ok, that sounds reasonable too. In the past, the Comelec has purchased bad indelible ink too. They must have caught Smartmatic doing the same thing for the UV ink. However, this explanation conflicts with the first. Is it misalignment in the printing, or bad ink? I was ready to believe one explanation or the other. But two conflicting explanations make me ask, what else are these people not telling us?

Anyway, that’s where the UV lamps come in. The PCOS machine was designed to look for some UV mark in the right place. If the mark is not exactly there, the stupid machine concludes that the ballot is fake, and spits it out. This must be the reason for all those rejections of valid ballots.

So, the Comelec says, let us disable the UV scanning feature of the PCOS (which we Filipino taxpayers paid for, by the way), and buy instead a billion pesos of UV lamps that the BEI will use to detect fake ballots manually. Great.

But what if it is neither the alignment nor the ink? What if the problem is with the PCOS itself? Remember that the manufacture of the PCOS machines was transferred at the last minute to factories in China, which made them in a rush to meet the extended deadlines in Manila.

What if the made-in-China PCOS machines themselves are the problem: perhaps due to poor accuracy or generally poor quality? (After all, experts originally estimated the AES project to cost at least P11 billion, and Smartmatic underbid everyone else with their lower P7.2 billion bid.)

Before the Comelec buys P1 billion worth of UV lamps because P4 billion worth of vote counting machines are unable to read UV marks, shouldn’t the Comelec make sure that the problem is truly in the UV ink or its alignment and not in the PCOS machine itself?

There’s only one way to make sure: test the PCOS machines thoroughly. Smartmatic claims they are testing 2,000 machines daily. In 2004, the Comelec took 3 months to test 1,990 counting machines. Smartmatic is testing a similar number of machines in just one day? This can only mean one thing: the Smartmatic tests are neither as complete nor as thorough. They could be omitting some important tests, or some important data.

In fact, there seems to be a Comelec-Smartmatic blackout on the results of these PCOS tests. I have not seen anywhere, and I’ve been looking for it since January, a compilation of test results regarding PCOS failure rates, rate of ballot rejection, and scan accuracy rates. HALAL couldn’t get the statistics out of Comelec either. Yet, the Comelec must know these things, before signing those Smartmatic delivery documents which will more or less say, “This is to certify that the following items were received in good condition,” which is the basis for setting into motion the process of payment. Comelec should not take the vendor’s word for it, but check the quality of the goods themselves. Due diligence.

We will get defrauded not once but twice, if we spend another P1 billion to correct what is purported to be an ink or misalignment problem, should it turn out that the true cause of the problem is the P4 billion worth of PCOS machines which the Comelec has not thoroughly tested for workmanship, performance, and conformity to specifications.