One of the best pieces to be published in this campaign season has just popped up on the New Republic’s website by Michael Crowley: “Survey Says: How Many Pollsters Does It Take To Screw Up An Election?”
Crowley offers a pitch-perfect portrait of the way in which polling has overwhelmed this race and the political class in general; once an obsession solely of professionals, polling has become grist for the mills of dozens of websites and tens of millions of people, just at the moment at which polling itself has become statistically questionable at best owing to cultural changes that polling can’t properly take account of — like increased cell phone use, fewer people at home in the evenings and on weekends, more resistance to calls from strangers, and the difficulty in determining both the size and composition of the actual electorate:
The glut of new polls–and vast spectrum of quality–has created a Darwinian environment in which pollsters and watchdogs attack one another with nerdy ferocity. (One pollster described a firm he considers disreputable to me as “a street gang with a calculator.”) Mark Blumenthal and his colleagues at Pollster. com routinely flag suspect polls, calling out their authors when they don’t disclose crucial information like the wording of their questions and their demographic weighting. Before the Iowa caucus, for instance, Blumenthal challenged pollsters to explain how they were screening for “likely voters” in that unusual contest; five pollsters refused to respond and others grudgingly provided incomplete answers–a fact Blumenthal publicized in an angry New York Times op-ed column….
Almost anyone with an Internet connection and an interest in cross-tabs can become an ombudsman–much like Alan Abramowitz, a political science professor at Emory University. After the longtime Democratic pollster Celinda Lake co-produced national polling numbers showing surprising weakness for Obama earlier this month, for instance, Abramowitz sent Lake a tart e-mail, blind cc’d to several other recipients, demanding to see her raw data. Unappeased, he followed up a few days later: “Celinda–do you believe your own poll?” During this campaign, Abramowitz has badgered several other major pollsters this way, all for the benefit of the fellow academics and journalists he copies on his e- mails.
Sometimes it seems that pollsters spend nearly as much time arguing with their critics as they do actually gathering data. As a result, pollsters face the same fate as other traditional voices of political authority–not least the mainstream media. The more they bash one another in the public eye, the less the public trusts the objectivity of their work.
The interesting problem is that campaign polling was once a tool to help campaigns see their own weaknesses with clarity — a difficult thing to do in a human enterprise and one aided immeasurably by the use of “data” rather than analysis so that the ox being gored on a campaign wasn’t being gored by someone else’s ox, but rather by an empirical challenge. The useful questions that used to be asked were ones with yes-no answers or no more than two choices. Now, owing to the very fact that polls include dozens of questions and have response rates of something like 10 to 20 percent (meaning that most people hang up on poll calls), the likelihood that they mean anything is very low.
That’s why Real Clear Politics began aggregating blogs and coming up with an average — the simple idea being that while no one poll was trustworthy, perhaps mixing them all together would cancel out partisan leanings and response rate weaknesses and allow for a clearer, if vaguer, overall picture.
That picture indicates that Obama, today, enjoys a lead outside the margin of error, nearing 7 points. Which could mean Obama is up 11 or up 3. The only thing we probably do know is that if the election were held today, McCain would lose. His chance for victory lies with Obama’s lead being 3; if it’s 11, he can’t catch up.
One thing is for sure, as Crowley’s very fine piece makes clear: The polls are not going to help us with this, and neither will the exit polls. Only the vote totals.