What Can We Learn From The Latest Polling Debacle in KY?
On Tuesday night in the Kentucky gubernatorial election, Republican Matt Bevin comfortably defeated his Democratic opponent, Jack Conway, by nine points, defying recent polls that predicted (albeit by a slim margin) a Conway victory. Coupled with a string of equally glaring misfires in last year’s U.S. Senate elections, there’s bound to be a slew of postmortems in the coming days that question the effectiveness of survey research in an era where landlines are disappearing and pollsters are still figuring out best practices for internet and cell phone surveys.
We won’t get into what went wrong with polling models heading into the KY election here. For some insights on that, check out Harry Enten’s excellent analysis on Nate Silver’s FiveThirtyEight. Instead, now is a good time to step back and look at what lessons can be imparted when using public opinion research for public affairs and advocacy campaigns.
Survey research can be an invaluable tool for campaigns. But if it’s not approached with sound strategy and a fundamental understanding of its utility and limitations, it can be a costly waste of time and resources. With that in mind, here are a few guidelines for ensuring your next polling project will yield actionable data for your campaign:
Hire a reputable pollster: This should go without saying, but if you want your poll to be taken seriously, hire a pollster with experience in your issue area and a solid reputation, someone who will not only produce reliable data but help you explain it in the best terms. There are many top-tier Republican and Democratic pollsters as well as nonpartisan firms who do great work. Campaign budgets are always a factor, but if you’re trying to define and shape public opinion on a critical issue, perhaps that free SurveyMonkey poll isn’t the best option.
Know why you’re conducting research: There are usually two main objectives for deciding to do survey research: for public release to elevate awareness about an issue, or for internal use to help inform strategy and messaging. Is your poll for public release? If so, your methodology had better be rock-solid. Otherwise, a smart reporter will call you out and you’ll spend the next couple days dealing with process stories that will reflect poorly on your organization.
Avoid confusing questions and language: Many public affairs campaigns involve complex, nuanced issues that are difficult to decipher even by policy wonks and may not be easily understood by the average voter. For polling data to be reliable, there must be a high degree of certainty that respondents understand what you’re asking.
Always test intensity: Asking whether voters care about your issue isn’t enough. To get to the heart of what sways public opinion, we need to know how much they care, and even more critically, how much they care compared to other issues that are also influencing their behavior. To develop messaging that resonates and moves the needle on your issue, you need to understand and be able to tap into what’s really motivating your audience.
GIGO: The old “garbage in, garbage out” axiom in computer programming is equally apt for survey research. If your methodology and questions are poorly constructed or misleading, the results will be worthless.
Don’t forget about qualitative research: Polling can tell us a lot of things, but at the end of the day, a poll is merely a statistical snapshot of public opinion at the time it was taken. Oftentimes, utilizing a combination of quantitative and qualitative research, like focus groups and online analytics, can help us better define our understanding of public opinion among target audiences and develop language and messaging that can actually change minds.
Polling is a science, but as we’ve seen with Bevin’s “upset” victory in Kentucky, the incorrect application of that science can yield some embarrassing results. To avoid falling into this trap on your next research project, keep the guidelines above in mind.