Ever since its creation in 1935, Gallup has been hailed as the leader of polling in the United States. Beginning with its prediction in 1936 that FDR would defeat Alfred Landon for the U.S. presidency, Gallup has reported on every presidential race since. So when Gallup announced it would not be reporting on the 2016 horse race, the world was shocked.
Gallup’s polling has been off recently. In 2012, while Nate Silver predicted Obama’s win, Gallup’s findings placed Romney in a dead heat with Obama, a non-definitive answer that tarnished its reputation. Gallup studied the numerical mishaps in 2012 and found several sources of error, such as underrepresenting Eastern time zones in Midwest countries, and polling too many older, more Republican voters.
While Gallup’s inaccuracies came as a shock to the political community, they are representative of a bigger problem: the inaccuracies of phone polling, which have only gotten worse. Traditional polling by landline phone has grown more difficult as fewer people are responding to phone surveys; yet, there are still no solid alternatives to phone polling.
Today, landline responses have dropped to eight percent, Considering that only 18 years ago the response rate was 36 percent, this decline in response rate is disturbing. One poll can cost up to five figures, a price that is only going up as response rates go down. According to Cliff Zukin, professor of political science at Rutgers University and former president of the American Association for Public Opinion Research, the lack of response is growing because people are “tired of being solicited on the phone. … As more and more people have grown up and only known cell phones, they figured that if the landline rings in their household, it can’t be for them, it’s a solicitation.” The low response rate biases results towards groups of people who tend to answer phone surveys—mostly seniors. Polling more senior citizens than young Americans makes it harder to determine the opinions of the younger generation and the public as a whole.
Zukin went on to explain to the HPR that to compensate for current biases, “every poll that comes out of the field is weighted. That means it’s adjusted for people we don’t get. … If for example we’re supposed to have 200 people polled with less than a high school education and we only get 100, we will tell the computer to count each person as if they were two, and that brings them up to the correct population level.”
Adjusting polls for those who don’t pick up the phone has its own implications. Weighted polls assume that people from the same demographic have similar political views, but this assumption is not always true: two white 25-year old male high school graduates may be similar, but they are not identical. As the number of people who pick up the phone decreases, polls grow reliant on a smaller and smaller number of people to speak for the general public.
Further, with the rise of cell phones, polls tend to overlook younger and low-income houses without landlines. This oversight is due to the 1991 Telephone Consumer Protection Act, which banned automatic dialing systems to mobile phones. This law exists partially so that people who have to pay for incoming calls are not charged for phone solicitation. However, autodialing programs allow for easy, rapid phone calls. “If that act was repealed, people could use automatic dialers to call cell phones and that would bring the cost down,” Zukin said. “So if the cost was down, then more people would be able to do better work.” Repealing the act would lead to a higher response rate and more accurate polling, but regulation has moved in the opposite direction—in 2012, the FCC took even more strict steps to strengthen limits on polling via cell phones. Considering how many people use cell phones, the move was a significant loss for polling.
With the growing flaws of phone polling, online polling may provide a solution. In 2012 Internet polls were on average more accurate than phone polls, and could cover more people. However, pollsters dispute the proper methods for Internet polls, and internet users tend to be younger. Weighting could resolve the lack of representation in older populations, but this is not the perfect solution. So, until pollsters come up with a cohesive approach for internet surveys, we are stuck with costly and potentially inaccurate phone polls.
Many pundits have successfully used online polls despite their flaws. Nate Silver’s FiveThirtyEight consistently uses polls—including Internet polls—to accurately predict elections. Some of these success are due to the fact that websites like FiveThirtyEight combine multiple polls so the strengths of one poll balance the weaknesses of another. However, this method still does not address the inaccuracies of standalone polls. As is, online polls may be reliable when they are aggregated, but likely not when they stand alone.
Polling is struggling across the board. Gallup’s refusal to poll on the horserace demonstrates the large-scale battle for accurate polls that can’t keep up with the change in technology. With these issues in mind, what does polling mean for the 2016 election?
It is unlikely that standalone Internet polls rather than aggregate polls will have a solidified methodology until past the 2016 election, so the current primary season will be struck with the current polling dilemmas. Given the skewed data in polls, the American public should not treat the numbers as gospel so far from the election. Primary polls since 2000 have had an average error of 7.7 percentage points. Considering how tight races can be up to Election Day, 7.7 percent is the difference between a crushing loss and a landslide victory. With races ending within a margin of one to two percentage points, this margin of error is unacceptable.
So when CNBC recently reported that Hillary Clinton reached 50 percent support in an online poll conducted by Surveymonkey, the public needs to understand that several groups─such as the elderly─are likely underrepresented. Surveymonkey does weight on age, but as Gallup’s experience has shown, weighting is not always accurate in practice. Clinton could be above or below 50 percent approval, depending on how the online poll was conducted.
When it comes to polls in 2016 and beyond, all voters must view the numbers with a skeptical eye. Without a clear concept of who is in the lead in polls, the election is anyone’s game.
Image credit: Fir0002/Wikimedia Commons