Tag Archives: systest

HALAL Statement: Smartmatic ballot production software was not certified


Last April 30, in response to a request by candidate Joey de Venecia III, the Comelec made public a set of documents relating to the source code review of the Smartmatic software conducted by SysTest Labs Inc. One of the documents was “Certification Test Summary for AES May 2010 Rev. 1.00”, dated March 8, 2010. It summarized another SysTest report “Final AES Certification Test Report for the Smartmatic Automated Electon System (AES)”, which the COMELEC has not made public yet.

The Summary may help explain the recent spate of PCOS failures to read local votes which, Smartmatic admits, is due to their error in configuring the ballot design. Citing the problems SysTest had earlier found in its source code review, the Summary listed several “compensating controls” that were essential in mitigating the Smartmatic software problems that SysTest had identified.

In one compensating control, SysTest was very explicit: “The Ballot Production tool was not subjected to the full certification process; therefore it should not be utilized in the May 10, 2010 election process.” (Summary, p.6) Given the ballot printing problems of the COMELEC, from the misalignment of the ultraviolet security mark to the misconfiguration of the ballot design, HALAL asks the COMELEC and Smartmatic to clarify if they utilized Smartmatic’s Ballot Production tool despite the explicit warning of SysTest.

HALAL notes that the March 8 SysTest summary only gave a conditional endorsement of the Smartmatic software. HALAL further notes that the summary was submitted one month past the AES Law deadline for the legally-required certification “categorically stating that the AES … is operating properly, securely and accurately”. This was the SysTest recommendation in the summary (p.7): “Assuming the abovementioned [compensating] controls are put into practice and that the AES is properly configured, operated and supported, SysTest Labs finds the Smartmatic Automated Election System to be capable of operating properly, securely and accurately and therefore recommends the system for certification and use in the May 10, 2010 election.”

Instead of the categorical statement required by the AES Law R.A. 9369, SysTest’s conditional endorsement was premised on the crucial assumption that all “controls are put into practice”.

According to the AES Law R.A. 9369, the COMELEC Technical Evaluation Committee must “certify, through an established international certification entity, … categorically stating that the AES, including its hardware and software components, is operating properly, securely, and accurately, in accordance with the provisions of this Act based, among others, on the following documented results: 1) … ; 2) … ; 3) The successful completion of a source code review; 4) … “

Given all the problems cited in the Feb. 9 SysTest report (HALAL’s analysis of this report is attached), and the explicit warning in the Mar. 8 SysTest summary report against using Smartmatic’s ballot production tool, it is clear that no certification should have been issued to the Smartmatic software because it would put our national elections at an unacceptably high risk.

HALAL analysis of recently-released SysTest source code review

Halalang Marangal (HALAL) recently obtained a copy of the SysTest report on the source code review of the Smartmatic software that it conducted Oct. 26, 2009 to Feb. 9, 2010. This review was the basis for the Comelec concluding that the Smartmatic Automated Election System will count our May 10 votes properly, securely and accurately. The SysTest report and related documents may be downloaded here.

HALAL’s conclusion, after scrutinizing the SysTest report, is that the Smartmatic software should NOT have been certified. We should not have put our national elections at risk given the clear warnings of SysTest about problems in the Smartmatic software.

You may download the HALAL analysis here.