Too soon to call 2010 elections successful

It is too soon to declare the 2010 elections a success.

People want a successful election so badly, that it is easy to get carried away by flood of incoming election returns. Many want to believe that a clean and honest election has finally happened, at last.

But the vice-presidential election is yet to be settled. The contest between the 12^th and 13^th places in the senatorial race still has to be settled too. Hundreds, perhaps thousands, of local races also await to be settled.

Already news is coming in about delayed Election Returns (ERs), malfunctioning, missing or otherwise questionable memory cards, and other indicators of potential or emerging problems.

As in the manual system, the precinct level count is always the fastest. Even when election inspectors, watchers and the public counted votes by hand, most of the election results had always been available past midnight or early morning. Even under the manual method, the biggest challenge has always been at the municipal level and higher, where wholesale cheating operations occurred.

In fact, the automated election system failed spectacularly its first truly public test a week before election day, when many candidates got zero – a “bawas” — and some got more than the votes actually cast for them – a “dagdag”. The results were worse than most manual counts. Fortunately, the failures in the machine count were so obvious that the election inspectors and watchers noticed them immediately. An embarrassed Comelec quickly called off the public test, and traced the problem to misaligned ovals on the ballot. Because of a last-minute change from single-spacing to double-spacing in the ballot layout for local candidates, their oval locations did not anymore match the coordinates stored in a configuration file in a memory card within the PCOS machine.

Reconfiguring the memory cards was somewhat easier than reprinting ballots, so that is what the Comelec and Smartmatic tried to do.

Smartmatic only had 18,000 spare memory cards and there was little time to recall the rest, so in addition to the spares, Smartmatic recalled the cards that could still be recalled; imported the rest from Hongkong and Taiwan; edited each of the 1,631 ballot layout configuration files (unique for every town); programmed these configuration files into 76,340 memory cards (one for each machine); delivered the 76,340 newly reconfigured memory cards to the waiting machines all over the archipelago; found the right machines for the right memory cards; replaced the misconfigured memory card; and conducted a second round of public testing and sealing of the PCOS machines. All within a span of five days – 120 hours. Aside from some 400 machines that malfunctioned, the rest of the 76,340 machines worked fine and gave the country its first successful automated elections. So they say.

Can we now trust the machine results?

These machines had grievously failed to count a few days earlier. This was followed by a mad rush of recalls, importations, file reconfigurations, card reprogramming, deliveries, reinstallation, and a second round of testing and sealing. In the mad rush, were security procedures and chain of custody considerations still observed? Did anyone see an election inspector with an ultraviolet lamp to check for authentic ballots, for instance? (We have not found anyone who did.) Suppose there were also more subtle problems that a ten-ballot test set was insufficient to detect – ovals that were misaligned by only one or two millimeters, for example, just as the security marks were, or oval coordinates that were purposely changed slightly to shave votes from targetted candidates. Were tests done at all for these potential problems?

Suppose an ATM had earlier given you only half the money than it deducted from your account, and the bank tells you the machine is now ok. Wouldn’t you count your money at least once in subsequent withdrawals? Suppose most ATMs of a bank network shortchanged its clients, wouldn’t they demand that every ATM of that network be carefully tested and recertified for its counting accuracy?

For exactly the same reason, every candidate who lost – and won – in the machine-counted 2010 elections should demand thorough post-election testing and audit for accuracy of every counting machine and its results.

Losing candidates should demand it, because they might have actually won.

Winning candidates – especially those who lead by a huge margin – should demand it, because the gross machine errors a few days earlier and subsequent doubts about machine accuracy have devalued their victory.

Apparent president-elect Noynoy Aquino should demand it, if only for the sake of his running-mate Mar Roxas, who sacrificed his own presidential ambitions to give way to Noynoy.

There was no time for proper testing in the mad rush to the May 10 elections because few wanted the elections postponed. But we have fifty days before June 30, when the new set of elected officials are scheduled to take over. We still have enough time check, double-check, and be sure about the results of the 2010 elections.

In the meantime, the Comelec and local election authorities should not be in a hurry to proclaim winners and declare the elections a success.

Expect a flood of demands for recount from losing candidates

The PCOS fiasco a few days before the May 10 elections has shattered the credibility not only of election automation, but of the entire electoral process itself. The entire process hinges on an accurate count by the PCOS machine. In fact, COMELEC specifications require at least 99.995% accuracy, or at most one error for every 20,000 marks or around 600 ballots. Today, we have very little idea how accurately the PCOS will count our votes.

One minor mistake – changing from a single-spaced to a double-spaced layout – has created a crisis of credibility for the entire elections. Who will accept at face value the PCOS machine counts now? Even a candidate who wins with a narrow margin may suspect that the machine might have trimmed his margin. Certainly, losing candidates can now be expected to demand a manual recount, so they can see for themselves how they actually fared. No assurance from Smartmatic or the Comelec will now suffice, because everyone has seen how the PCOS machine made gross errors during the final testing and sealing.

Many may not realize it yet, but we are in a different ballgame now. May 10 election victories — whoever wins, in whatever position – have just suffered a major devaluation.

The PCOS fiasco has robbed the winners of the May 10 elections of a clear victory. It has created not only a cloud of doubt about the machine results, but also last-minute disruptions in the crucial final days before the elections that will surely create confusion and chaos in many areas.

Deliveries had to be suspended, because the PCOS machines had to stay put in the hubs or sub-hubs, so they can await the CF memory card replacements. The final testing and sealing had to be called off, to avoid further embarassment and damage to the credibility of the automation project. When the replacement memory cards eventually arrive for installation in the machines, only then can these machines leave the distribution hubs and sub-hubs for final deployment.

Designing machines and ballot sets that will only work with each other, but delivering them separately had earlier created a “logistical nightmare”. The nightmare just became worse, because machines, ballot sets, and memory cards are now being delivered separately, and under even greater time pressure.

If the PCOS machine, its associated set of ballots, and its associated memory card, do manage to find themselves reunited in their destination precinct cluster, on time for the May 10 elections, they still need to be tested again. It was obvious from the fiasco that the accuracy of the machines were not checked before they were deployed, so the final testing is our only chance to determine the accuracy of the PCOS. This final testing cannot be dispensed with.

If the final testing with a ten-ballot test set shows even a single error, then we cannot guarantee with 95% confidence that the machine’s error rate is lower than 1%. There’s a good chance it is higher than 1%. Then, the board of election inspectors must either ask for a replacement, or go ahead and use an inaccurate machine.

Where machines don’t arrive, stop working, reject too many valid ballots, or otherwise fail, the board of election inspectors must resort to a manual count. The Comelec says it has prepared the paraphernalia for up to 30% of the precincts. But the precinct election inspectors have ask for the paraphernalia to be delivered first. More delays. Given the confusion, more than 30% of precincts may need to resort to a manual count.

If the paraphernalia for a manual count don’t arrive, the ballot boxes will have to be sealed and watched over, to be counted later.

So, expect confusion and chaos in many areas on election day. Yet, even where everything works smoothly, the machine counts will remain under a cloud of doubt, and losing candidates will surely demand a recount, citing as reason the errors made by the PCOS machines earlier. Who can blame them?

All because of an innocuous-looking suggestion to change from a single-spaced to a double-spaced ballot layout.

The origin of this suggestion must be traced. Who first raised it? Why did Comelec approve it? Why didn’t Smartmatic or Comelec test the accuracy of the PCOS machine with the new layout?

If anyone had intentionally wanted to disrupt the elections and discredit its results, he – or she – couldn’t have done it any better.

HALAL analysis of recently-released SysTest source code review

Halalang Marangal (HALAL) recently obtained a copy of the SysTest report on the source code review of the Smartmatic software that it conducted Oct. 26, 2009 to Feb. 9, 2010. This review was the basis for the Comelec concluding that the Smartmatic Automated Election System will count our May 10 votes properly, securely and accurately. The SysTest report and related documents may be downloaded here.

HALAL’s conclusion, after scrutinizing the SysTest report, is that the Smartmatic software should NOT have been certified. We should not have put our national elections at risk given the clear warnings of SysTest about problems in the Smartmatic software.

You may download the HALAL analysis here.

Partial failure of automation has already occurred; how to determine if the failure is total

The printing of 50.85 million ballots for the May 10 elections is now complete. The COMELEC said they were two days ahead of schedule. This early finish, however, was achieved at terrible cost: the high-speed printing caused the misalignment of the ultraviolet security mark on the ballot by one to two millimeters. This led the failure of the automated PCOS ballot authentication system, which the COMELEC had to abandon in favor of a manual ballot authentication system. This manual system relies on election inspectors to shine a UV lamp on each ballot to determine ballot authenticity through visual inspection.

The unreliable scanning of UV marks is, in effect, already a partial failure of automation.

But if the UV marks were misaligned due to the high-speed printing, shouldn’t we all be concerned that the ovals themselves have been similarly misaligned? Misaligned ovals would have very serious consequences. In the 1998 automation pilot in ARMM, according to a report on the COMELEC website, similar ballot printing problems forced the COMELEC to manually recount ballots from Sulu and some municipalities of Lanao del Sur. However carefully 2010 voters will shade the ovals, if these ovals are misaligned, then the voters’ marks will also be misaligned. This will make the PCOS unreliable in registering the voters’ choices, in exactly the same way they had become unreliable in detecting the UV security marks.

If HALAL’s concern turns out to be true, that ovals themselves were also misaligned by the same high-speed printing that misaligned the UV security marks, then we cannot rely anymore on the PCOS to accurately register the voters’ choices. An inaccurate PCOS machine would render useless the rest of the automated election system. It will mean a total failure of automation.

Misalignment of ovals is far more serious than misalignment of UV marks for the following reason: misaligned UV marks will lead the PCOS machine to reject valid ballots, an obvious problem which voters will notice immediately and complain about. This failure of automation cannot be hidden from the voters. Thus, the COMELEC has no choice but to correct the problem. This is also true for PCOS machines that stop working. But inaccurate vote counts due to oval misalignment will still be registered by the machine. Voters will never know their votes were falsely registered. This failure of automation can be hidden from voters. Thus, no one will notice, and no one can complain if the COMELEC and Smartmatic opt to hide this problem. Firms or persons may want to do so, if they can be held legally or financially accountable for the failure of a P7.2 billion project.

Earlier field tests and mock elections were announced by COMELEC to be “almost perfect” and by Smartmatic to be “successful” despite numerous media reports of machines rejecting valid ballots and transmission problems even in Metro Manila. These false claims suggest that the COMELEC and Smartmatic are not averse to declaring “success” despite public knowledge of ballot rejections and transmission problems. Thus, it would not be far-fetched for them to likewise claim “successful” automation on May 10, despite inaccurate scans and false machine counts which no voter will notice and complain about.

Indeed, the COMELEC has managed to hide so far the PCOS machine’s true scanning accuracy, which has remained a complete mystery.

Four opportunities to measure the true PCOS accuracy had been lost: 1) the results of the acceptance tests remain inaccessible to the public; 2) the full reports of Systest Labs, which conducted a system audit and source code review, also remain inaccessible to the public (update: Commissioner Larrazabal has agreed to release the Systest Labs report); and 3) the voter verification feature of the PCOS was disabled, preventing voters from knowing if the machine accurately registered their choices; and 4) the COMELEC decided to reject the proposal for a 100% audit of PCOS accuracy, and to stick instead to a random audit that covers only 1.5% of machines.

Unfortunately, the fifth opportunity to measure PCOS accuracy – and the alignment of ovals – is woefully inadequate for detecting an inaccurate machine, which, according to COMELEC specifications, is one with an accuracy rate below 99.995%. Several days before election day, every board of election inspectors (BEI) is supposed to set up the PCOS machines and do a final test. The machines will be fed with ten pre-tabulated ballots and the expected results compared with the machine count. To pass, a machine must count all the test ballots without a single error. COMELEC rules are not clear what the BEI should do if the PCOS makes even a single error. The most logical thing to do – a more rigorous version of this test should have been done months ago – is to return the machine to Smartmatic for replacement. A ten-ballot set is not enough. It will miss 98% of marginally inaccurate machines, and detect only 2%. A test set of 1,762 ballots is needed to screen out 95% of marginally inaccurate machines. A 2,708-ballot test set can screen out 99% of marginally inaccurate machines. though it will still pass 1%, or 822 of the 82,200 PCOS machines. How I got these numbers will require a separate piece. (Google “how to show that a PCOS is accurate enough”)

Instead of testing machines randomly with a 1,762-ballot test set, the COMELEC’s best option might be to heed the following HALAL suggestion: increase the random manual audit (RMA) coverage from 1.5% to 30% of the machines.

A number of strong arguments justify this simple suggestion: 1) the no-legal-basis argument cannot be invoked, because the law provides for the RMA; 2) the COMELEC clearly has the power to change the coverage of the RMA, because it has already raised the coverage from 0.3% (one precinct per district) to 1.5% (five precincts per district); 3) the COMELEC has long announced that it was prepared to do a manual count in 30% of the precincts, so the 30% RMA can simply use the existing paraphernalia for the manual count; no new expenses or preparations are needed; since no precinct knows in advance if it will be drawn for the RMA, every precinct must prepare to conduct an RMA anyway, whether the COMELEC is going to cover 1.5% or 30% of precincts; 4) a 30% RMA may not satisfy all, but it will certainly placate some groups, particularly the business sector, that had wanted a 100% parallel count; the twenty-fold increase in coverage from 1.5% to 30% will surely go a long way in raising the credibility of the electoral process; and 5) auditing 30% of the PCOS machines should be enough to determine the extent of PCOS inaccuracy caused by any misalignment of ovals.

Unless a credible audit is done, we might never know whether the machines counted our votes accurately or not, and whether the failure of automation we suffered was partial or total.

How to show that a PCOS voting machine is accurate enough

The COMELEC has already rejected 100% manual audit proposal. That should be the end of it.

Yet, it is hard ignore the voices of the country’s major information technology organizations, business oranizations, legal organizations, election watchdogs, church organizations, and other sectors – all counselling the COMELEC to do a 100% manual audit, since this is the first time the country is trying Smartmatic’s SAES 1800 PCOS machine.

Even in banks that have automated their operations, to the extent of using a bill counter that counts bills automatically, tellers still count the bills manually before handing out withdrawals to clients. And clients too would again count the money manually before leaving. When a depositor withdraws from an ATM, she would also manually count the money issued by the ATM machine, before she leaves the premises. We all do this, despite the fact that we have no reason to doubt the accuracy of the bill counters and ATMs.

Since we do have a strong reason to doubt the accuracy of PCOS machines, all the more should we double-check its count. Why do we doubt the accuracy of the PCOS?

The Comelec has already acknowledged the misalignment of ultraviolet security marks due to high-speed printing. Because of this misalignment, the Comelec decided on its own, without prodding, to ask election inspectors to conduct a 100% manual audit of the authenticity of the ballots, by visually determining with a UV lamp whether each ballot is authentic or not.

Since the Comelec already admitted that the security mark was misaligned, then it should, at the very least, double-check the possibility that the ovals in the ballot are likewise misaligned. If some ballots have misaligned ovals, then for exactly the same reason that the PCOS machine became unreliable in detecting the security marks, it will also become unreliable in detecting shaded ovals.

This is why we doubt the accuracy of the PCOS machines: they might be fed ballots with misaligned ovals caused by the high-speed printing.

The Comelec specified in its contract with Smartmatic that the PCOS should be able to read shaded marks correctly at least 99.995% of the time (in other words, that it makes at most one error for every 20,000 marks or .005%).

I propose a way to convincingly show, with 95% confidence, that a PCOS machine is 99.995% accurate. A 95% confidence level is the typical confidence level used in statistical tests, product quality control, and scientific experiments.

At this point, the math will get a little bit heavy. But since we are talking of P7.2 billion of taxpayers’ money, let us all hunker down and take the extra effort.

Let us suppose that Smartmatic supplied us with machines whose accuracy rates are all slightly lower than the Comelec specification of 99.995%, say, they make one error for every 19,999 instead of 20,000 marks. We want a set of test ballots that will detect these below-spec machines, because they will make one or more reading errors. Then we can reject these inaccurate machines and have them replaced by Smartmatic because they are all slightly below Comelec specifications (unless of course the Comelec was willing to adjust its specs downwards to accomodate Smartmatic).

If we feed one of these below-spec machines a test ballot with a single mark, we know that the chances of a correct reading is 19,998/19,999 or 99.9949997%, which is practically 100%. We’d expect the machine to read the ballot without error. If we used this ballot as our test set, this below-spec machine, which should fail the test, will pass it instead, where “fail” means at least one reading error and “pass” means zero reading error.

If we feed a test ballot with two marks into the same machine, the chances of two consecutive successes in reading marks are 99.99449997% x 99.99449997% or 99.990%. Our below-spec machine will most probably pass such a test ballot too.

If we feed a test ballot with three marks into the same machine, the chances of correctly reading all three are 99.99449997% x 99.99449997% x 99.99449997% or 99.985%. The machine will again probably pass the test.

Brace yourself: the above can be written as (99.99449997%)^3 = 99.985%, where “^3” means “multiplied by itself three times”.

A real ballot will contain an average of 34 marks (one each for president, vice president, party-list, congressman, governor, mayor, and so on, 12 for senators, etc.).

Hence the chances that the below-spec machine will accurately read a single test ballot with 34 marks correctly are (99.99449997%)^34 = 99.83%. This is still very high. The machine will most probably read this test ballot correctly too, and pass the test.

For ten ballots (340 marks): the chances of accurately reading all of them are (99.99449997%)^340 = 98.3%. Still very high.

Note that 3-7 days before election day, election inspectors, watchers and watchdogs in every precinct cluster will use a ten-ballot test set to determine if the PCOS machines meet the Comelec specification of 99.995%. This means they should be able to detect our below-spec machine, so that it may be replaced by Smartmatic. However, the chances that our below-spec machine will make no reading error and therefore pass the test is 98.3%. Thus, out of a hundred of our below-spec machines, the ten-ballot test will correctly detect two but will pass 98 machines that should have failed. This is obviously not acceptable.

For thirty ballots (1,020 marks): the chances of accurately reading all are (99.99449997%)^1020= 95.0%

For 408 ballots (13,872 marks): the chances are (99.99449997%)^13,872 = 50.0%. If we use a 408-ballot set to test the below-spec machines, half of them will be detected successfully, but half will still pass the test. Not good.

For 1,000-ballot test set (34,000 marks): the chances of our machines passing are now (99.99449997%)^34,000 = 18.3%. Out of a hundred below-spec machines, 18 will still pass undetected. Not acceptable.

Let me now jump to the magic number: 1,762 ballots (59,908 marks). The chances are now (99.99449997%)^59,908 = 5.0%. Out of a hundred, ninety-five of the below-spec machines will fail and only five will pass a 1,762-ballot test set. (In an earlier computation, I had used 99.9944% and got 1,573. A 1,762-ballot test set will detect marginally inaccurate machines better.)

This 95% confidence level is the most common standard used in statistical testing, product quality control, and scientific experiments. If we wanted even better quality standards, we can adopt a 99% instead of 95% confidence level. This will require a 2,708-ballot test set. Then, out of every hundred below-spec machines that Smartmatic delivers, 99 will make at least one error and will be correctly rejected. Only one will pass the test and be deployed for vote counting.(This would still be 822 below-spec PCOS machines out of the 82,200, but that’s probably a necessary risk, unless we want even stricter testing).

Let me now summarize how to test each PCOS machine for accuracy:

Prepare a test set of 1,762 official ballots with 34 marks each, making sure that every oval on the ballot is marked at least once. Tabulate the votes represented by the marks. The tally must be checked and rechecked for 100% accuracy, because it is the standard against which the accuracy of the machine will be measured.

Configure a PCOS machine to accept the ballots, and feed the test set into the machine. If the machine reads all the ballots without error, then we can be certain at the 95% confidence level that the machine has an accuracy rate of 99.995% or better. (If I were the test engineer, I would then repeat the test on every accepted machine, and I would only use as backup those machines that fail on the second test.)

If the COMELEC had done this acceptance testing when the machines were delivered, any machine which did not meet the 95% confidence level should have been returned to Smartmatic for replacement. Thus every PCOS machine deployed for the elections should have passed this test. Since such a test can take around seven hours and a half (assuming 15 seconds per ballot), and Smartmatic claimed that it was testing PCOS machines at the rate of 2,000 per day, it is highly unlikely that these machines went through this kind of acceptance testing. So, we do not know how many PCOS machines actually meet the Comelec specifications.

If Smartmatic is confident about the accuracy of its PCOS machines, and if the Comelec is confident about the alignment of the ovals in the 50.85 million ballots they just printed, they should welcome this 1,762-ballot test as a chance to prove that the PCOS machines truly meet the Comelec specifications for accuracy.

50.85 million ballots printed: how many have misaligned ovals?

The news just came in: the printing of ballots for the May 10 national elections is now complete — all 50.85 million of them.

We already know, by COMELEC’s own admission, that many of these 50.85 million ballots have misaligned ultraviolet security marks.

The niggling question is: how many of these 50.85 million ballots also have misaligned ovals?

For a more detailed discussion of misaligned ovals, see my earlier post, which was also published on April 21, 2010 by Philippine Star, p.18.

Four opportunities to check PCOS accuracy: all taken away by COMELEC

The Filipino voting public had four opportunities for checking the scanning accuracy of the PCOS machine, but the three were all taken away by COMELEC, while the fate of the fourth still hangs in the balance.

Acceptance testing by COMELEC

Everytime Smartmatic delivered a batch of machines, COMELEC should have tested the machines before accepting them, to screen out the lemons. After all, these machines were made in China in a hurry. But even if they were not. It is simple due diligence. The COMELEC specifications are clear in the contract, including the PCOS minimum accuracy rate of 99.995%, or a maximum error rate of .005%. Any machine that did not meet specs should have been returned for replacement. We lost this opportunity because the full results of the testing, if it was ever done by COMELEC at all, were not given to stakeholders like political parties, election watchdogs and the media.

Systest audit and source code review

One of the things Systest would have tested, because it is the heart of the ballot appreciation process, is the scanning accuracy of the PCOS. I had expected a certification from Systest that the PCOS, for instance, met all minimum functional requirements as specified in the Automated Election Law and the contract between COMELEC and Smartmatic. I had expected that the test procedures would be described in detail in the full report of Systest and the test results would be there too, for the scrutiny of all stakeholders. Finally, I had expected the Systest certifications and well as full report to be available to stakeholders like political parties, election watchdogs and the media. COMELEC claimed on Feb. 9 that Systest had finished its audit/review and given its certification, but released no proper Systest document or full report to back up its claim. We lost this opportunity because of the secrecy that surrounds the certifications and full report from Systest.

Voters verification that the PCOS correctly registered their choices

On election day itself, all voters would be in a position to check for themselves the accuracy of the PCOS, by observing whether their marks are properly interpreted by the machine. The PCOS was required by law to show voters their choices (that is, the names of those they voted for), so that they can verify if their choices were correctly registered on the machine. The PCOS, in fact, had this as a built-in feature, and it could show on its screen those choices. This was an excellect feature that empowered every voter to conduct a real-time audit of the accuracy of the machine. We lost this opportunity because COMELEC ordered Smartmatic to disable this feature.

Manual audit of the machine results

The ballots will be counted manually by an audit team, and this count compared with the machine count. Any discrepancy, after the manual count is double-checked, will mean an inaccurate scan by the PCOS, allowing us to actually measure its accuracy rate. We fear that this opportunity may be lost because COMELEC decided to conduct the manual audit of only 1.5% of precincts, and the machine winners may be proclaimed ation anyway, while the audit is still going on. If the machine winners are proclaimed while the audit is still going on, then the audit results won’t matter anymore. We all know how long election protests take to resolve, if they are ever resolved at all. Who would be interested in a futile exercise that is already moot and academic? The rush to proclaim within 48 hours (by May 13), is misplaced because the term of outgoing officials will end on June 30 yet. The manual audit can be given a June 22 deadline, the winners proclaimed on June 23, and still have one week to spare.

If 100% manual audit of UV marks is ok, why not 100% manual audit of ovals?

The question that bugs me is why would COMELEC plug every loophole to make sure that no test results from acceptance testing or the Systest audit/review can be used to question PCOS accuracy, that no inaccuracies are observable to voters on election day, and that when the inaccuracies are eventually discovered through a manual audit, it will be too late to correct them because the machine winners have been proclaimed.

When COMELEC found out that the ultraviolet security marks were misaligned, it ordered a 100% manual audit of every ballot before winners are proclaimed. COMELEC did so on its own, without any prodding.

If the ballot ovals are as misaligned as the ultraviolet security marks, by the same logic, the only solution is a 100% manual audit of every ballot before winners are proclaimed.

Three things the COMELEC can do to assuage our concerns

This is the latest letter of HALAL to the COMELEC. The HALAL suggestions are important and urgent. Let us all add our voices to these suggestions.

Halalang Marangal

April 20, 2010

The En Banc

COMMISSION ON ELECTIONS

Palacio del Gobernador

Intramuros, Manila

Hon. Chairman JOSE A. R. MELO

Hon. Commissioner RENE V. SARMIENTO

Hon. Commissioner NICODEMO T. FERRER

Hon. Commissioner LUCENITO T. TAGLE

Hon. Commissioner ARMANDO C. VELASCO

Hon. Commissioner ELIAS R. YUSOPH

Hon. Commissioner GREGORIO Y. LARRAZABAL

Gentlemen:

We are deeply concerned that the risk of failure of the Automated Election System (AES) remains unacceptably high. But it is very late in the day to return to the old manual system, which may not save us either. It will simply mean business-as-usual for the cheats, who have mastered the manual system so well that they can manipulate its results in their sleep.

Halalang Marangal (HALAL) proposes three simple things for the COMELEC to do that can help lead us out of this dilemma.

THREE THINGS WE ASK FROM THE COMELEC TO ASSUAGE PEOPLE’S CONCERNS ABOUT THE AES

ONE. We ask the COMELEC to assign every scanning machine and its associated precinct cluster a simple unique identifier and widely disseminate this information to the public.¹

Why? Because this will make the Precinct Count Optical Scanner (PCOS) machines more accountable, just as vehicles with license plates, or policemen with visible name plates, are more accountable. It empowers the voting public better when they know exactly the individual machine they are dealing with, making it easy to identify errant machines, send reports, complaints, etc. The unique identifiers of all machines in a polling center must be conspicuously displayed outside the center, in big characters. The ID of each PCOS must also be conspicuously displayed in the room where the machine is located. A unique identifier, known by all, is a fundamental requirement in any automation project.

These unique PCOS IDs must be quickly added to the COMELEC Project of Precincts (POP), which is the master list of all polling precincts. The updated POP must be made available to the media and the public on a compact disc (CD), for quick copying and dissemination.

TWO. We ask the Comelec to authorize election inspectors to print before transmission 29 of the 30 Election Returns (ERs).

Why? To make as many copies of the precinct results as possible, before the connection with a central server and its associated possibility that the authentic PCOS data may be overwritten with false data from the server during transmission. Also, to remove any cause for PPCRV and NAMFREL, Nacionalistas and Liberals, or national and local candidates of the same party, to dispute who should get ER originals. There may not be enough with eight copies, there will be enough with 29 copies. With so many copies of the original ER circulating, suppressing the truth will be much harder.

The printed ER should contain the machine’s unique ID. The ER originals can now go to more political parties, election watchdogs, media organizations, and other accredited civic groups.

The 30^th copy of the ER, printed after transmission, should be immediately read aloud by the BEI so that everyone who had received an earlier copy may make sure that their copy is identical with the 30^th copy. If not, then the transmission has compromised the election results in the PCOS memory as well as the 30^th copy. These, and the electronic ER copy just transmitted to the central servers, must now be presumed to contain false data and must be questioned. This must be noted in the minutes. But the 29 original ERs still contain the authentic data.

THREE. We ask the COMELEC to authorize a 100% manual audit of the votes for president.

Why? As a final check on the accuracy of the PCOS count, for the same reason that a bank teller still manually counts bills after they have been counted by a machine. The COMELEC reverted to a 100% manual audit of ballot authenticity after it was discovered that the high-speed printing caused 1-2 mm misalignment which resulted in the inaccurate scanning of the UV security mark. For exactly the same reason, we need a 100% manual audit of the votes, to find out if a similar misaligment of ovals due to high-speed printing also resulted in the inaccurate scanning of marks. Without the PCOS feature that allows each voter to verify if his choices were correctly registered by the machine, a 100% manual audit is our only remaining option to check the scanning and counting accuracy of the PCOS. Earlier proposals that have been raised for a 100% manual audit covering three positions only were in the right direction. Our counsel to limit further the coverage of the audit to the president is meant to streamline the process even more, by making possible a very simple manual method as follows:

The BEI will first count the ballots, and then sort them into separate stacks, one stack per presidential candidate. The stacks should be double-checked by individual watchers and elections watchdogs.² The BEI will then count in public, aloud, the ballots in each stack. If the counts are correct, then the total of the counts will equal the number of ballots. If no discrepancy is found, we estimate that this method will take no more than one hour. If a discrepancy is found, it may take another half hour to confirm the accuracy of the manual count.

We emphasize that extra care must be taken to make this manual audit 100% accurate, because it will serve as the standard against which the accuracy of the PCOS machine will be measured. The stacks and the final vote counts must be checked several times to detect and correct any errors. The results of this manual count of votes for president, including the total votes and any discrepancy in the totals, must be appended to the printed ERs and countersigned by the BEI. The results of the manual count will also be entered in the minutes. They should be taken into account before any winner is proclaimed. After all, the term of outgoing elected officials will end only in June 30. There no need to rush any proclamation.

Airplanes and ships are required by law to carry life vests and life boats, even if these are superfluous most of the time because air and sea accidents are rare. They are there not only to assuage passenger concerns, but also because of the lives they will save in those rare events when they are needed.

Similarly, we ask the COMELEC for the following safety mechanisms: unique PCOS identifiers made public, 29 ERs printed before any transmission, and a 100% manual audit of the votes for president before proclaiming any winner.

We hope the COMELEC will consider these HALAL recommendations for the 2010 elections, which we are now formally submitting to the COMELEC in the spirit of supporting the Commission’s efforts to protect the integrity of the ballot and the sanctity of the people’s voice.

Very truly yours,

Halalang Marangal Convenors/Board of Directors

WIGBERTO TAÑADA Chairman, HALAL Former Senator Senate of the Philippines	MEHOL K. SADAIN Former Commissioner Commission on Elections	FRANCISCO GUDANI President, HALAL Retired General Armed Forces of the Philippines
ISAGANI SERRANO President Philippine Rural Reconstruction Movement	Sr. MARY JOHN MANANZAN Association of Major Religious Superiors	MA. PAZ LUNA TOYM Awardee
	ROBERTO VERZOLA Secretary-General, HALAL

1If a unique ID per PCOS already exists, then the COMELEC need only publicize them widely. Smartmatic is apparently using a Stock-Keeping Unit (SKU) system that uses seven digits for internal control. This is enough. The important thing is that the identifiers are unique per machine, that they are consistently used in all official lists and ERs, and that they are publicly known and, on election day, conspicuously displayed outside polling centers and in the precincts.

2This is how some countries do their manual count. After voting closes, all ballots for one electoral jurisdiction are transported to a big hall or covered court, authenticaled, and then dumped on the floor. “Scrutineers” then sort the ballots into stacks, one candidate, one stack. Double-checked, of course. If the contest is not very close, the heights of the stacks will show before the counting starts – even before the sorting is over – who won as member of parliament in that jurisdiction. On the same night, winners in most juristictions would be known, as well as the majority party if any, and then of course the prime minister – a clean and honest count, without automation.

PCOS machines in Philippine automated elections: failure rates, error rates

According to the news, two of the twenty PCOS machines in Hongkong stopped working for a while. That is a 10% failure rate.

Cesar Flores of Smartmatic claims they expect a PCOS failure rate of 0.3- 0.5%. However, vendor claims must be taken with a grain of salt, more so if their goods were hurriedly made in China. The claim is also belied by Smartmatic’s own plans: they are deploying 8% of the total machines for backup. So, they must be expecting up to 8% of the machines to fail, which is more consistent with the failure rates in Hongkong.

The actual PCOS failure rate is, in fact, a big unknown.

First, it appears that Smartmatic had done most of the testing, not COMELEC. Due diligence requires COMELEC to do acceptance testing. Any buyer must double-check delivered goods before signing a receipt acknowledging that it was received in good working condition. Especially since the Smartmatic deliveries involved P7.2 billion of taxpayers’ money, the machines should have been independently tested if they meet COMELEC specifications as detailed in their contract. Those that didn’t meet specs should have been returned for replacement. If deployed, these can cause trouble during election day itself, as they did in Hongkong.

The 0.3-0.5% PCOS failure rate that Smartmatic claims is not backed by properly-witnessed test stats and is contradicted by the 10% failure rate reported in Hongkong and Smartmatic’s own preparations to replace up to 8% of machines that may fail on election day.

Second, the test stats have remained inaccessible to third parties like political parties, election watchdogs and media. Transparent test stats minimize potential insider collusion (as in the ballot secrecy folder contract), which can result in overpricing or payments for sub-standard equipment. Transparency also minimizes the possibility that insiders will selectively assign machines, depending on their quality, in order to cause trouble in targetted regions or provinces in favor one candidate or another. Just imagine if problemmatic machines or modems are selectively assigned to Aquino, Villar, Estrada, or Teodoro bailiwicks — whoever are disfavored.

At least five PCOS test results are so important that they should be publicly known:

• Mean time between failures (MTBF). This is the average time a PCOS machine stays operational. Knowing the MTBF and the mean time to repair (or replace), we can determine the average failure rate. Instead of actual statistics, we have today media-reported field anecdotes and unsubstantiated vendor claims.

• Average rejection rate of valid ballots. This is a specific but important case, when the PCOS stays operational but rejects a valid ballot. In Smartmatic demos, field tests and mock elections, the rejection rates were inordinately high, far above COMELEC specs.

• Scan error rate. Just as PCOS machines fail, they make mistakes too. A PCOS scanning error can be a false positive (registering a vote that is not there) or a false negative (missing a vote that is there). When the PCOS is adjusted to read lighter shades, false positives increase because even a slight smudge may be falsely registered as a vote. When adjusted to read darker shades only, false negatives increase, because lightly or partially shaded ovals may be missed by the PCOS. Each machine has to be calibrated towards that ideal spot which minimizes the total errors from both false positives and false negatives. Based on COMELEC specs, this total should be lower than .005%, or five scanning errors for every 100,000 marks (at most one error per 1,000 ballots). Unfortunately, the calibration may change in transit or under environmental stresses like heat, humidity, or mechanical shocks. A PCOS machine that rejects valid ballots has, in effect, very high false negatives, because it misses all the shaded ovals, each representing one vote, in those rejected but valid ballots.

• Transmission error rate. Because of ambient electrical and electronic noise, transmission is more susceptible to error than scanning, and therefore demands high quality equipment. That Smartmatic modems had transmission problems even within Metro Manila does not speak well of their quality. A poor quality modem is hopeless and should be replaced.

• Battery backup life. COMELEC specified at least 16 hours of backup. A good quality control engineer would insist on batteries lasting up to 20 hours under test, a 25% margin for coping with unexpected operating and environmental extremes.

Smartmatic had earlier claimed it was testing 2,000 machines a day. Compare this to the three months it took COMELEC to thoroughly test some 1,900 automated counting machines in 2004. Even granting that the 2004 testing was done at a leisurely pace, the huge difference still makes one wonder how thorough the PCOS testing was.

In particular, the PCOS scan error rate is very important. If the error rate is, say, 5%, and the presidential winner’s margin is less than 5%, then we will again find ourselves in political limbo. In 2004, GMA’s supposed margin over FPJ was 3.48%. COMELEC specified .005%, which is quite low. But it doesn’t look like COMELEC actually measured each machine’s error rate. That is not possible when testing 2,000 machines a day (the necessary statistical test requires more than 1,700 test ballots per PCOS).

Each PCOS should pass various COMELEC tests before it is accepted, paid for, and deployed to a polling place. And stakeholders should have access to all the test statistics, including the number of machines that stopped operating, the number of valid ballots rejected, and the actual number of falsely registered voter choices, to prevent insiders from accepting bad machines and selectively assigning these to targetted areas.

Without the test statistics, we can only guess which is closer to the truth, the 10% failure rate shown by the machines in Hongkong, or the 0.3-0.5% failure rate claimed by Smartmatic.

It is not too late. COMELEC can still order the release to media of these test statistics, and improve its credibility before the voting public.

More stink at the Comelec: 1.68 million folders for P0.638 billion?

And I thought P1 billion for some 80,000 UV lamps stank so much that you could smell the overprice a mile away.

This one, you could smell 10 miles away!

[UPDATE: A more recent story by Malaya says the Comelec has rescinded the contract. Just like that? Nobody will be prosecuted for approving and awarding the contract?]

The full report by GMA News is here. This is the gist:

Election officials had bought nearly two million ballot secrecy folders for the May 10 elections at an overpriced rate of P380 each, Senate Minority Leader Aquilino Pimentel Jr. alleged on Sunday.

Citing information from a confidential source, Pimentel claimed there had been no bidding for the folders, which will be used to cover the ballot during voting. He declined to name the company that had bagged the contract from the Commission on Elections (Comelec).

“I want Chairman [Jose] Melo to investigate this matter because this will taint the integrity of the Comelec. It looks like more than P300 per piece is too much… I cannot name my source because doing so will endanger his life,” Pimentel told GMANews.TV in Filipino.

Under Comelec Resolution 8786, there should be 22 folders per clustered precinct. This means the poll body must shell out P638 million for 1.68 million folders for the 76,340 clustered precincts in the upcoming elections.