"Africa is, indeed, coming into fashion." - Horace Walpole (1774)


looking at elections: the bradley effect

(The latest in a series of posts aimed at explaining election stuff to all you lucky non-political scientists)

Not 20 minutes after the networks called it for Hillary Clinton in New Hampshire, I started to see mentions of the "Bradley Effect" on political blogs and news sites. The Bradley Effect refers to a phenomenon in which pre-election polls inaccurately predict an African-American or other minority candidate's percentage of the vote because voters being polled want the person asking the question to believe that they would vote for a minority candidate even though they won't.

In the case of this week's vote, this would cause us to infer that the Bradley Effect was the reason Barack Obama enjoyed a strong advantage over Hillary Clinton in the polls leading up to Tuesday's primary but still lost to Clinton. New Hampshire voters, the thesis goes, want to be seen by pollsters as progressive, open-minded people, but in reality, some of them were lying and don't actually intend to vote for the African-American candidate for whatever reason.

(A side argument on this says that perhaps that's why Obama did so well in Iowa: the social pressure and public nature of the caucuses meant that people had to take a public stand. If they want to be perceived as progressive, they had to back up what they told pollsters with their actual vote.)

I'm inclined not to buy the thesis that the Bradley Effect explains why pollsters were wrong about New Hampshire. Why? Because, as Charles Franklin eloquently explains over at Mystery Pollster, the polls were actually pretty accurate when it came to predicting how much of the total vote Obama would receive. As Franklin notes, "The standard trend estimate for Obama was 36.7%, the sensitive estimate was 39.0% and the last five poll average was 38.4%, all reasonably close to his actual 36.4%."

The mistake, therefore, was not in the predictions as to how Obama would do, but rather pollsters failed to capture how much support Hillary Clinton gained. And I don't think there's much they could have done to mitigate that; the exit poll data made it clear that a lot of voters made up their minds at the very end of the race. Among the 17% of Democratic primary voters who decided on election day, Clinton got 39% to Obama's 36%. I think those 19,000 or so votes (almost 2,000 more than Obama), added to those who decided long ago that they would vote for her, were probably enough to push her over the top. Clinton won by less than 8,000 votes.

While I have no doubt that there are many Americans who won't vote for Obama because of the color of his skin, in this case, I just don't think that's what happened. South Carolina may be more interesting in this regard, but there, I'm not sure there's an incentive for voters to lie to pollsters. Half of South Carolina's Democratic primary voters are African-American, and there's less social pressure among South Carolina whites to be seen as politically and socially progressive.


Blogger euphrony said...

"There are lies and there are damn lies. And then there are statistics."

I think every mention of polls by the media should be prefaced with this quote. And I say that having a good friend whose PhD is in statistics. Used well, it can provide valuable insights. Or, used poorly, it can twist data to say whatever you want it to say. My old boss always said he could take some bit of research data and argue at least three mutually exclusive conclusions from it. I've been around long enough to see that is so true.

You have to be much more than smart, you have to be honest and open to all facts, with no attachments to any particular hypothesis. Therein, I think, lies a big problem with polls. They want to lead a greater audience to a story, so subtleties may be getting missed. what do you think?

By the way, the Wikipedia article on the Bradley Effect already mentions Obama in New Hampshire. They're fast over there.

Thursday, January 10, 2008 3:56:00 PM

Blogger texasinafrica said...

Oh, no question. It's very, very difficult to conduct an unbiased, accurate poll. That said, several of my colleagues/friends do this for a living, and they really do work hard to control for possible biases and trick people into being honest whether they want to or not.

I never trust polls conducted by candidates or anyone else with a vested stake in the outcome of an election, nor do I trust those like Zogby and Rasmussen, which use questionable methods to get a sample of the population. But polls conducted by universities and some professional firms are usually pretty reliable. Even in this case, they were right on the Republicans, and they're right more often than they're wrong. Note that the only poll that accurately predicted Clinton's win was a university poll in NH.

As for statistics, we have like to quote an old saying around here: "Statistics are like prisoners. If you torture them long enough, they'll say anything."

Thursday, January 10, 2008 5:56:00 PM

Anonymous Michael Westmoreland-White said...

I agree with your analysis, but then, today, Andrew Cuomo endorsed H.R. Clinton with a not very subtle racial code phrase about Obama, saying he was trying to "shuck and jive" voters. Clinton may wish Cuomo hadn't said a word!

Thursday, January 10, 2008 7:09:00 PM


Post a Comment

<< Home