This
was too good not to translate. Bruni performs an analysis of the
fingerprint machines, as applied in the Venezuelan election and gets an answer
that I have always suspected was the reason why we never heard any evaluation
of the performance of this very expensive fingerprint capturing system in our elections:
The Devil is in the details, Chapter
IV. One voter, one vote: indelible ink or fingerprint machines?
The first
time I heard about the fingerprint capturing machines I was surprised at the
large number of technical and logistical challenges that would have to be
overcome in order to install such a system and I asked myself if the
possibility of multiple votes in Venezuela, justified the purchase
and installation of such an expensive and complicated system. In fact, in a pamphlet
by COGENT systems, the winner of the bidding process, it is specified that never
before had such a system been implemented, with so many technical challenges,
as the one that was installed in Venezuela[1].
But, independently of whether it is,
or not, technically feasible to put such a system in practice with success, the
question that has been going around in my head is if the Directors of the CNE [2]
that defend the fingerprint machines are right, if, effectively, the
fingerprint machines guarantee the premise of
One voter, one vote
I decided then to investigate a
little bit about systems for the recognition of fingerprints
I found that such systems are used
specially for criminal identification and that there are two types of problems
that an be tackled.
The first one is called “1:1” or “one-to-one”,.
In this one, the fingerprint is compared
with another one that is believed to be that of the same person. For example
the fingerprint of Pedro Perez is compared with that which the authorities have
of Pedro Perez and it is determined whether it happens to be the same fingerprint
or not.
The second problem is the “1:N” or “one
to many”. In this case you want to know if the fingerprint of Pedro Perez is
found among a set of fingerprints stored by the authorities. It is obvious that
the second case is more complicated and that it can yield a higher percentage
of errors than the first.
Logically, in an
election, I told myself, both types of recognitions have to be performed to
determine if Pedro Perez is who he says he is and later determine if he already
voted.
I could not obtain official information
about this, but I have been told that in the Venezuelan elections only the second
type of verification was performed, that is, the “one to many”, while the
determination if the person was Pedro Perez was done with the National ID card,
called cedula.
Following this, I tried
to learn more about the ways to verify such systems. I found that the then
National Institute for Standards (NIST)
of the US Government, performs
tests to determine the precision of various commercial systems, including among
them, the systems made by Cogent Systems.
The evaluation is made following
two complementary criteria, the TAR and the FAR. According to one of the NIST
reports [3], the
TAR (True Accept Rate) is defined as the fraction of correct identifications by
the identity algorithm, while the FAR (False Accept Rate) is defined as the
fraction of false positives in recognizing an identity.
Now, even thought the Cogent
systems received excellent reviews in the independent tests that were
performed, the accuracy rates were not 100%
Let’s see, for example, the results
relative to the identification systems of individuals presented by NIST at the Biometrics
Congress in 2004 [4]
(see page 16). According to the presentation, it was found that in the tests for
the identification of visitors, Cogent’s technology had a TAR of 98% when databases
of high quality fingerprints were used and it could go down as low as 47% when databases
with low quality fingerprints were used.
In both cases, a value of FAR (false positives) of 0.01% was found.
Let’s set aside these numbers in our minds
for the moment and let’s make an analysis of the possible results of the
application of fingerprint capturing machines in the Venezuelan elections.
When a vote arrives at the
fingerprint machine, there are two possibilities: that he is an honest voter
(He has not voted yet) or he is a voter that cheats (He already voted and wants
to vote again). On the other hand, the verification system for the fingerprint
capturing system can respond correctly or erroneously if the voter already
voted or not or even may not find the fingerprint or take longer than the time
required to do it. We then have the following possibilities:
|
|
True state of the voter
|
System Response
|
Interpretation f the result
|
What does the law say in this
case? [5]
|
|
Case1
|
Did not vote
|
Did not vote
|
correct
|
Allows vote
|
|
Case2
|
Did not Vote
|
Voted
|
error
|
Does not allow vote
|
|
Case3
|
Did not vote
|
Can’t find it
|
error
|
Allows vote
|
|
Case4
|
Voted
|
Voted
|
correct
|
Does not allow vote
|
|
Case5
|
Voted
|
Did not vote
|
error
|
Allows vote
|
|
Case6
|
Voted
|
Can’t find it
|
error
|
Allows vote
|
As you
can see, it is a system much more complex than a simple system to identify
Pedro Perez whether individually or with a database of many fingerprints. Thus
if you were to design tests to evaluate the trustworthiness of the answers of such
systems, the levels of precision have to
be much tighter than those found in identification systems.
Now, suppose for a moment that we
can apply the TAR given above for our system. That is, let’s say that the TAR
is 98% for excellent fingerprint databases and goes down to 47% for low quality
fingerprints. The TAR gives the rate for a good performance which, in our
system, consists of cases 1 and 4. Let’s say also that only 25% of the
fingerprints stored in the database of the CNE are of low quality and finally,
let’s say that there are 10 million voters. In this case, we would obtain that only 8.52 million voters are in the
category of “One Person, one vote”, the other 1.48 million missing would
fall under the category of errors. According to the law, in cases 3,5 and 6
they are allowed to vote and, among them, we don’t know if there are any cases
of multiple votes.
Of course, if the CNE were a
serious organization, it would have already informed us of how many cases there
were of multiple votes and false positive recognitions by the little machines. After
spending so many millions on them, Venezuelans deserve to know what are the TAR
and other statistical errors of such an onerous system. No?
One thing is certain. The CNE is
NOT right: the fingerprint machines DO NOT GUARANTEE the principle of “One Voter,
one vote”
….and the indelible ink is much cheaper
and much faster….
References
[1]
COGENT document, “One Voter one Vote”.
[2]
El Nacional, 30 de Julio, page A2. Reference to CNE Directors Lucena y Hernández.
[3] Fingerprint Vendor
Technology Evaluation 2003, Análisis Report. National Institute of Standards
and Technology.
[4]
Wilson, C.L., “NIST Patriot Act Biometric Testing”, Biometrics Conference,
2004.
[5] CNE,
Resolución N° 041022-1621, “Normas sobre el procedimiento de captación de
huellas dactilares y garantía del principio de un elector un voto en las
elecciones regionales 2004”.
Note: after publishing this post,
a reader indicated that my sentences about the CNE not publishing the data was
not accurate since the information on the number of “cheaters” in the elections have been
published in a table of an Ultimas Noticias article of July 30, 2006 (page30).
According to that table, between the Revocatory Referendum and the governors
elections there has been a total of 53 cheaters.
Such a value shows that not only the system is not 100% reliable and
produces mistrust among the voters, but its cost and the political anxiety it
has created cannot be justified by the abysmally low cheating statistics.
In the same table, I discovered some data called “grey zones” that show the
number of voters that could not be
properly identified by the system. The numbers shown are quite high and seem to
confirm even more strongly that the “one voter, one vote” principle cannot be
guaranteed.
Another reader indicated that in the CAPEL report there was information
about the digital fingerprints. A quick review made me realize that my
hypothesis of 25% of bad quality fingerprints was optimistic. Therefore, the
errors produced due to the imprecision of the fingerprints is even higher.
In other words, the more I learn details about this system, the more I like
the indelible ink.
That is why I say that the Devil is always in the details.