UC's Bishop, UNH's Moore Team Up on Critique of Pollsters' Failings

Anyone who was within earshot of a television for any length of time in the past year is likely aware that this past election cycle proved to be a particularly difficult time for professional pollsters. With hundreds of polls to choose among then, who distinguished themselves not only by missing the mark but doing it in eye-catching style?

Establishing an answer to that question was the goal for two of the nation’s most respected and seasoned polling experts, the University of Cincinnati’s George Bishop and the University of New Hampshire’s David W. Moore, as they compiled the first annual Top Ten Dubious Polling Awards, released this week on stinkyjournalism.org.

Serious purpose lay behind the exercise’s light-hearted title – Bishop and Moore hope to call attention to the scientific validity and utility of polls.

So who topped their list? That "honor" is shared by all the major media polls, who jointly (but wrongly) portrayed Rudolph Giuliani as the early frontrunner for the 2008 Republican Presidential nomination. In the paper published along with the awards, Bishop and Moore condemn pollsters for portraying Giuliani as the frontrunner "relentlessly, and without regard for any semblance of political reality."

That projection earned all the major pollsters the list’s highest indicator – a designation of five crossed fingers, or the polling equivalent of a "Hail Mary pass."

"Every year, poll watchers are confronted with poll results and commentary that defy either logic or science," says Bishop, professor of political science at UC and author of The Illusion of Public Opinion (Rowman & Littlefield, 2005). "When pollsters focus on what they believe is entertaining and appealing to the audience rather than an accurate reflection of public opinion, they manipulate public opinion."

New Hampshire’s Moore is a senior fellow of the Carsey Institute and a former senior editor of the Gallup Poll, where he worked for 13 years.

Other "Dubious Polling" achievements from the last year singled out for recognition by the pair include:

• Loopiest Poll: Pew Research Poll, for weekly pre-election polls in October that showed wild swings in Obama’s lead.
• Shooting Yourself in the Foot Award: Gallup Poll, for publishing two polls on Feb. 25, 2008, that contradicted each other.
• Over-the-Top Gloating Award: Gary Langer, polling director of ABC, for writing that "What I liked best about the final New Hampshire pre-election polls [which erroneously predicted Obama to win] is that I didn’t do any of them."
• 180 Degree Award: CBS News/New York Times and USA Today/Gallup polls, for coming to opposite conclusions about the controversy over Rev. Jeremiah Wright.
• Waiting for Godot Award: The American Association for Public Opinion Research Committee that still has not issued a report on the erroneous predictions in the N.H. Democratic Primary.
• Who Knows? Award: Pew Research, ABC News/Washington Post, and Los Angeles Times/Bloomberg polls for contradictory conclusions about public support for Wall Street bailout.
• The Wake-Me-Up-When-It’s-Over Award: NPR, Kaiser Family Foundation, and the Harvard School of Public Health Survey for a vague 131-word question.
• Flip-Flop Award: CNN for two December polls that showed opposite results of the public’s support for the auto bailout.
• For Sale! Award: Peter D. Hart Research Associates, for their General Motors-sponsored poll that found overwhelming public support of the auto industry bailout.

The complete article by Bishop and Moore is available on www.stinkyjournalism.org, a site that advocates for more rigorous and scientific journalistic methodology.