Columns » John Brummett

Poll critique — a partisan affair

by and

comment

My recent history of poll criticism compels me to shut my mouth on the subject. But I’ll go ahead and type a few words, anyway. Otherwise the space would be left blank or, worse, somebody else might fill it.

Last fall the Arkansas Poll put out by the political science professionals at the University of Arkansas in Fayetteville declared George W. Bush ahead of John Kerry by nine points in Arkansas. I beheld this survey and found it an outrage.

They called people over a 15-day period ending Oct. 20, for goodness sakes, thereby surrendering any hope of doing what a poll was supposed to do, which was reflect movement and take a snapshot. Mixing political opinions of Oct. 5 with those of Oct. 20 over presidential debate season was like making a sandwich with one slice of fresh bread and one slice of molded. Or so I penned. Well, I didn’t actually use the bread metaphor, it having occurred to me just now.

Then we voted the first Tuesday of last November and Bush beat Kerry in Arkansas, by, uh, I believe it was nine points, like the poll said.

About the same time last year, television station KTHV, Channel 11, in Little Rock contracted with that nationwide robot outfit, Survey USA, which could do its polls cheap because it was a robot outfit and didn’t have to pay human beings to make the calls.

I was similarly outraged. Valid polls required live human screening, I wrote. Calling somebody with a robot and programming the robot to ask the respondent to punch one for Bush and two for Kerry risked taking the political temperature of a mere child or well-trained Labrador retriever. Or so I penned.

Survey USA put Bush ahead of Kerry in Arkansas by five points once and six points another time — not right on, it turned out, unless you threw in the margin of error, in which case it was, in fact, right on.

We should all consider whether my personal and freely acknowledged preference for Kerry over Bush — not so much on account of Kerry as on account of Bush — influenced either subconsciously or blatantly my critical analysis of these polls telling me something I didn’t enjoy hearing.

On that point, we have two new polls of the Arkansas governor’s race. One is another Arkansas Poll, and it shows Mike Beebe, the Democrat, leading Asa Hutchinson, the Republican, by seven points. Republicans are outraged, accusing the UA’s chief pollster, Dr. Janine Parry, of Democratic bias. They perhaps do not remember her poll’s dead-on forecast of their boy George’s big win a year ago.

They cite the discrepancy in sample sizes among the four congressional districts. They perhaps do not know that you could tinker with the poll to equalize respondents among those districts, or even weight them slightly to Northwest Arkansas because of growth and turnout trends, and you’d change the statewide numbers by less than a percentage point.

The other poll is by nationally-regarded Zogby for The Wall Street Journal’s online edition. It shows Hutchinson ahead of Beebe by a slightly higher margin than the Arkansas Poll shows Beebe ahead of Hutchinson. Democrats are outraged, explaining that Zogby’s poll, being an online sample, is weighted to the kinds of people who spend a lot of time online. Young Republicans, for example. They perhaps were not aware that Zogby was nailing the Virginia governor’s race, won Tuesday by a Democrat.

Both these polls know what they’re doing. The variation reflects statistical margins of error and sample differences amplified by the natural volatility occurring because this is a race in its embryonic state, not yet even advertised on television.

They’ll get together next October, I’ll wager.

This reminds me of sports fans who argue on Tuesday about what their heroes are going to do on Saturday, as if fans had anything to say or do about it, and as if we weren’t all going to find out the facts soon enough.

Add a comment

Clicky