Being a masochist, I have read most of the final Carter Center report on Venezuela and the recall referendum. I will not go through the agony of commenting on all of the things I disagree with, but I did find it a little too self-serving.
However, there is one part that I definitely have to talk about, because I have devoted lots of time to it in my personal life and in this blog: Chapter 13, in which the report talks about an “independent” panel on the allegations of statistical evidence for fraud. (pages 127-134).
First on the Panel. There were four people in the panel who had the background that qualified them to be on it. However, the report says that they were “independent” experts who had not been involved in the Venezuelan recall referendum. I disagree. Prof. Jonathan Taylor was used as a consultant by the Carter Center the week after the recall vote and his work was subject to criticism by the Venezuelan scientific community, which led him change his work and his conclusions. So, I would take exception to his presence in the panel as an “independent”.
In fact, it appears to me that by “independent” the Carter Center seemed to imply that none of them were Venezuelan. A truly “independent” panel should not have included anyone that was involved, worked on or published on the subject of the Venezuela recall in the days following the vote. This would have disqualified both Prof. Rubin and Prof. Taylor.
Now, as a scientist I find the whole procedure absurd anyway. Scientific ideas need to be discussed and debated, the most useful panel would have been one of independents in which all of those that did some work on the subject are invited to defend, discuss and debate their work and that of others.
The Carter Center has four conclusions. The first one agrees with the work of two of the “independent” panelists, but disagrees with the work of Jimenez and Valladares discussed in my section rrStudies and included in the discussion. Once again, Taylor’s work is quoted but no mention is ever made of why Jimenez is wrong.
The second conclusion refers to the work of Hausmann and Rigobon. The conclusion is that there are “other” reasons why these results were obtained, which have not been tested.
The third one refers to another conclusion by Hausmann and Rigobon and it is said that “the panel has attempted to replicate” the results and concluded that the anomaly is small. Maybe this should have been a reason to have someone like Rigobon on the panel. After all, many people have been unable to replicate Taylor’s work, but that does not mean that either side is right or wrong.
Finally, the panel concludes that there is insufficient evidence that Benford’s law applies to election results. Now, here I must ask: How many undergraduates does it take to screw this particular light bulb?
First of all, the data for the Yes vote in the recall does follow Benford’s Law. Second, the data from the 200 Venezuelan Presidential election also follows Benford’s law. So, why didn’t the independent panel hire a couple of undergraduates to reproduce the work of Pericci and Mykoss on the recall vote and the Venezuelan Presidential elections, instead of using data from Chicago or the results for the Valladares’s model? Now, that seems too involved and convoluted to me! In fact, I think the Carter Center should have looked at the last three or four Presidential elections and referenda in Venezuela. By doing that simple study, which can probably be done on a weekend by a couple of undergraduates, then instead of saying that there is “insufficient” evidence, they could have reached a very solid conclusion and would not have to use the word “insufficient”.
In fact, if you look at the first graph that Mykoss has here, you would be hard pressed to say that the result of the 2000 Presidential election does not follow Benford’s law and that was an election in Venezuela, not Chicago! The same could be said of Pericci and Torres, how do you explain that the Si follows Benford’s law, but the No does not?
In fact, in my old days as a Professor and Researcher, if this were the conclusions of a lab report by a student or a paper submitted to me as a referee, I would have given them a very low grade or rejected the paper. More likely, I would have sent them back to start all over again from the beginning, telling them that their procedures for setting up the panel were not scientific and it was biased. I would have also suggested that they needed to do a lot more work to support their conclusions, before their lab report or paper is completed.
Finally, I can only express my amazement that no mention is made of the Exit poll studies that question the results of the recall vote. In fact, the term Exit poll is never mentioned in the text! While there can be errors in the techniques used to make these polls, the study by Sanso and Prado has some really strong conclusions, which would be difficult to explain away even if the exit poll techniques were far from perfect. In fact, since it is never mentioned, I can only wonder if it was simply set aside for the reason that they could not explain it away…
As we say in Spanish “Piensa mal y Acertaras”