Why risk-limiting audits are risky.
It's a bad idea to use software to audit software.
As an election watcher, I have witnessed many risk-limiting audits (RLAs) in Larimer County. I was immediately struck by how inadequate they seemed because the most important audit step is hidden within the RLA tool and is not observable by the audit board or election watchers.
After reading the so-called “gentle introduction” to RLAs, I quickly learned that my skepticism was justified. The problem is not with the RLAs per se, but that Colorado is not doing the RLAs correctly and as a result cannot guarantee that the election outcomes are correct.
In this “extra gentle introduction” to RLAs, I explain what’s wrong with Colorado’s RLAs and how we can fix them. Even without all the technical details, it’s easy to see why Colorado’s RLA process can’t be trusted. We should be doing a hand audit with paper ballots and not using software to audit software. You might also be interested in other problems with Colorado’s RLA discussed in my post “Risk-limiting audits in Colorado have little chance of finding and fixing incorrect election outcomes.”
In most Colorado counties, ballots are scanned and tabulated using computers, creating a digital Cast Vote Record of each scanned ballot. The Cast Vote Record is simply a very wide spreadsheet, with one line listing all the votes for a single ballot card. For the 2024 November election in Larimer County, we had 2 ballot cards per voted ballot. When a batch of ballot cards are scanned, the scanner number, batch number, and position in the batch are printed on each card. This number series is included on the Cast Vote Record, which allows a specific randomly chosen ballot card to be retrieved and audited.
After the election is certified, counties publish the Cast Vote Record. However, if the published Cast Vote Record had any errors, there is no way for the public to know. A valid RLA is the only way to assure the public that the Cast Vote Record is correct.
The right way to conduct an RLA: a manual audit of paper ballots
According to RLA best practices, a bipartisan audit board should compare each paper ballot to its Cast Vote Record by hand. If any errors in the Cast Vote Record are found during the audit, additional ballots might have to be audited to determine if there are enough errors to change the outcome of the election. If enough errors are found in the Cast Vote Record, a full hand audit is possible.
The wrong way to conduct an RLA: using software to compare the Cast Vote Record to digital ballots
In Colorado, the audit board does not manually compare the Cast Vote Record to the paper ballot. The audit board does not even have access to the Cast Vote Record at any time during the audit! Instead, the audit board enters the votes from each paper ballot into the CO-RLA Tool. The software tool compares the digital ballot entry to the Cast Vote Record. The audit board does not even know if the Cast Vote Record matches the digital ballot.
Using software to audit software is a bad idea since software is vulnerable to hacking and coding errors. Furthermore, using software for the audit is not transparent because neither the audit board nor the election watchers can independently verify that the Cast Vote Record is correct.
Compare the above diagram to how Colorado conducts their RLAs in the diagram below. In Colorado, the audit board’s primary role is to ensure that the correct randomly selected ballot is being audited and that it matches the scanned ballot image. After they enter all the ballot info into the CO-RLA Tool, they patiently wait for the Secretary of State to tell them if they passed the audit or if they need to audit an additional random sample of ballots. Once the RLAs are completed, the results are published on Colorado’s audit website, but it is virtually impossible for anyone to independently confirm that the audit results are correct. The Secretary of State controls the entire RLA process and prevents full transparency.
How to fix Colorado’s RLA: provide the Colorado audit boards with the Cast Vote Record
It’s super easy to fix this problem and allow the audit board to do their job correctly. Simply provide them with the Cast Vote Record for every ballot they audit. This would allow them to check whether the Cast Vote Record is correct before they enter the ballot votes into the CO-RLA Tool. It would take mere seconds per ballot to add this step. It would also allow election watchers to independently verify that the Cast Vote Record for each audited ballot is correct.
When Colorado’s election rules make it difficult for the public to verify that election outcomes are correct, we can’t help but wonder: what are they trying to hide?
Using software to audit software is not transparent. It’s time to Make Elections Transparent Again.