The Gallup Organization has identified a series of factors that contributed to its overestimation of support for Mitt Romney in the 2012 presidential election, including problems with its survey sample and methodology, according to an in-depth analysis of its procedures released Tuesday.
Gallup was widely criticized about the results of its pre-election polls, and the review found that the way likely voters were defined, the telephone numbers that were called as well as their geographic distribution, and how people were asked about their race adversely affected Gallup’s presidential surveys.
At a meeting of journalists and pollsters on Tuesday, Frank Newport, the editor in chief of Gallup, outlined the testing the company had done with assistance from methodologists at the University of Michigan, which is known for its survey research program. Mr. Newport conceded that “there is something going on in the industry, and Gallup was at the bottom of that†in the 2012 election. Gallup has said that it is committed to transparency as the organization continues to test the way it conducts polls.
The review, led by Mr. Newport and Michael Traugott of the University of Michigan, was conducted by a team of statisticians, methodologists and analysts from Gallup, assisted by outside consultants. After identifying more than 20 possible elements that could have been related to the Republican skew in Gallup’s polls, the team isolated four factors as the likely causes.
Gallup’s model in identify those most likely to vote â€" a series of seven questions â€" seemed to have failed in 2012, and the organization is re-evaluating its formula for ranking voters who will turn out.
Just as technology has changed the way campaigns work, it has altered the way survey researchers gather data. A change in the way that Gallup called respondents on land lines may have been a contributing factor that led to its sample to be older and more Republican. Half of their respondents, however, were reached on cellphones â€" a proportion that is at or above industry standards.
In addition, there were too many respondents from certain time zones. In the South and the Midwest, there were more respondents from the Central Time Zone, which tends to be more Republican, than the Eastern Time Zone, which skews Democratic.
Finally, the way Gallup asked callers about their race overrepresented some groups.
Individually, none of the four problems is considered to have been a significant factor in the results, but taken together, they helped lead to an overestimation of Mr. Romney’s share of the vote. Gallup plans to use the governor’s races this fall in New Jersey and Virginia to test new election polling methods.
“We hope to be at the accurate end of the spectrum in the future rather than the uncomfortable position that Gallup was in last year,†Mr. Newton said.
