Election junkies (and even casual users) have been keeping their eyes on polls for weeks now. As the election approaches, news outlets are coming out with new polls every day, and sites like Real Clear Politics and Five Thirty Eight are spreading the poll fix. Right now, all the numbers are overwhelmingly in Obama’s favor, so based purely on statistics, this election might as well be over. But just how accurate are those polls?
One of the most memorable fiascoes in the history of fiascoes was the “Dewey Defeats Truman” headline after the 1948 election. Since The Hall of Presidents doesn’t have a creepy animatronic Dewey statue, you know that the headline was incorrect. Based on Gallup’s supposedly infallible exit poll numbers, the editors of the “Chicago Daily Tribune” figured that Dewey would win, so they put up the headline and called it a night. And, of course, we all remember what happened the last time the news jumped the gun with some poll numbers.
The numbers we’re seeing may not be as accurate as they seem. Here’s why:
The Cell Phone Problem (aka nobody calls me anymore)
Land lines are going out of style quickly, a fact that nobody realizes more than students. More and more households are becoming cell-phone only, largely among young people. So why does that matter to pollsters? Because many of them don’t call cell phones.
Recent data shows that about 16 percent of homes are cell-only, a number that could rise by the end of the year. And a good number of those were young people — Pew estimates that 46 percent of the cell-only users were ages 18-29 and another 36 percent were 30-49. Additionally, more minorities were cell-only. Those are groups that have been swinging Obama’s way, especially the young people.
Not surprisingly, excluding cell phones is skewing the polls. Calling cells is expensive and time-consuming because pollsters have to dial the numbers by hand. Plus, it’s so much easier to ignore a pollster if their number shows up on your screen. It’s estimated that leaving out cell numbers reduces Obama’s numbers by about 3 percent and inflates McCain’s by 1 percent. Obviously, that doesn’t change a lot in the grand scheme of these polls, but it still shows how skewed the results can be. Ignoring cell-only users means that pollsters are ignoring a lot of the under-30 set, including those first-time voters that are supposed to control this election.
The Bradley Effect (aka nobody wants to look racist)
News flash: there’s a black guy running for president. And that might be tipping the polls.
One of the unspoken problems in this election is that voters might actually be racist. But nobody wants to seem racist, so when talking to a pollster, people might be more inclined to say they’re voting for Obama, even though they agree with everything on that “New Yorker” cover.
It’s called the Bradley Effect, named after Tom Bradley, a black man who ran for California governor in 1982. Even though polls showed him with a significant lead heading into the election and even exit polls projected him winning, he ended up losing the race to his white opponent. Douglas Wilder experienced a similar dip after exit polls and he ended up winning his race for the 1989 Virginia governorship by less than half a percentage point.
Pollsters (and Democrats) are worrying that Obama might fall victim to the Bradley effect. There is little evidence that the effect has been hurting black candidates in the past 15 years, but none of the races have been as big as the presidency. It’s hard to quantify how much the Bradley effect might be skewing the polls, but it might be enough to negate those enormous leads Obama has been racking up.
The Aggregators Take Over (aka fuzzy math)
This election has seen a big digital transformation, whether it’s McCain hitting Facebook or Obama texting his supporters. But it’s also seen the rise of websites that mash poll numbers together. These aggregators, like Five Thirty Eight and Real Clear Politics, take polls from various sources and average them using fancy formulas to give a more accurate poll. They’ve become increasingly popular among people looking for a quick and easy way to see how the race stands. But are they actually accurate?
Five Thirty Eight has gotten props for its system of weighing polls, which supposedly give a more accurate result than just a simple average. And the founder, Nate Silver, knows his stats — he used to work for Baseball Prospectus, the team that predicted a winning season for the Tampa Bay Rays. These poll aggregators often show different numbers than the mainstream media and are more apt to call tossup races.
But no matter how sophisticated the formula, there’s still the issue that these aggregators are performing science on numbers (that might not be all that accurate in the first place) to come up with new numbers. The numbers haven’t been too far off, but it’s still easy to think that statisticians can play with numbers any way they want. And as these aggregators become more and more of a force in polls, it might be worth taking a second look.
This is all to say that polls shouldn’t dominate the coverage. But while they may be accurate and may show how the race is leaning, they don’t show the whole story. The only way to know for sure is to wait for Election Night.