Pollsters admit they were 'egregiously' wrong about 2020: Five Democratic firms say they widely underestimated Trump and turnout - and still don't know how to fix their broken system
A group of five prominent Democratic polling firms has conducted a deep dive into errors that caused surveys to fall widely off the mark in 2020 for the second consecutive presidential election – admitting the data were often 'egregiously' wrong in Republican states.
The assessment comes five months after an election where pollsters had Democrat Joe Biden leading Donald Trump by 10 percentage points leading up to the election, for a race where Biden's actual margin was 4 percentage points.
Just as consequentially, a range of pollsters had Democrats dominating key House and Senate races – missing how Democrats saw their House majority narrow. Democrats took over the Senate only after prevailing in two runoff elections in Georgia.
A group of five Democratic polling firms has taken a deep look-back at what went wrong in 2020, after surveys underestimated a support for Donald Trump and failed to anticipate that 'low propensity' voters who backed the Republican would show up on Election Day
Having taken the unusual step to share data in order to identify structural flaws, the pollsters say that in 2020 'our industry saw major errors and failed to live up to our own expectations.'
Data was off more in Republican states
'We saw that in more Democratic states and districts, and some closely divided states like Georgia and Arizona, the data were quite good. But in more Republican areas, the data were often wrong, sometimes egregiously so,' they wrote in the assessment.
The group of top pollsters, including leading pollster for the Biden campaign ALG polling, identify several factors contributing to the industry-wide whiff.
Pollsters had difficulty reaching 'low propensity' voters who supported President Donald Trump, seen here on election night at the White House
The polling firms shared data and conducted their own analysis to try to find out why they missed the mark
Pre-election polls overstated Biden's margin and under-estimated Trump's. Some pre-election polls had Biden holding a 10-point edge. Biden beat Trump 51 to 47, winning by about 7 million votes in the popular vote
Pollsters struggled once again to predict which voters would turn out. They say they were hampered by Trump supporters who were skeptical of pollsters
Pre-election polls consistently had Joe Biden ahead, with some pollsters showing him with a 10-point lead in the popular vote. The final result, a 51 to 47 per cent win for Biden, was tighter.
Trump supporters were skeptical of institutions – including pollsters
Some difficulties are grounded to changes in the nation's technology habits: more people are using cell phones, shedding landlines, and refusing to pick up unfamiliar numbers. But pollsters have ways to account for such factors – as well as adjusting turnout models to tweak for the electorate they believe will actually show up.
The group also homes in on polling challenges specific to Trump supporters, who may be more skeptical about institutions like government, the media, and polling firms. The pollsters say Trump may himself have exacerbated the situation himself.
'Trump may have helped turn this into a problem for pollsters by attracting distrustful voters and making his most ardent supporters even more distrustful of other people, of the media, and perhaps even polling itself,' they write. 'That, in turn, could have made his supporters less likely to answer polls.'
Participating in the look-back are ALG Research – which did polling for the Biden campaign, Garin-Hart-Yang Research, GBAO Strategies, Global Strategy Group and Normington Petts.
''Trump went after the polls,' one pollster told Politico. 'He was really pretty overt to those that were listening about some of his distrust of polls or media.'
Another told the publication 2020 was an 'Oh, s---' moment for all of us,' adding: 'I think that we all kind of quickly came to the point that we need to set our egos aside. We need to get this right.'
Models were wrong on 'low propensity voters' – underestimating white rural turnout – The pollsters also hashed out the problem teaching low propensity voters, those who are unlikely to show up even if they tell a pollster they have a preference.
They found that their models 'consistently overestimated Democratic turnout relative to Republican turnout' for such low propensity voters. Among such voters, 'the Republican share of the electorate exceeded expectations at four times the rate of the Democratic share,' they conclude.
'This turnout error meant, at least in some places, we again underestimated relative turnout among rural and white non-college voters, who are overrepresented among low propensity Republicans,' they write.
Pandemic factor
Polling the electorate amid a global pandemic would be challenging under any circumstances. But the pollsters hypothesize the pandemic could have worked to further skew data.
'Perhaps voters with more progressive attitudes on COVID-19 were not only more likely to wear masks and stay at home, but also more likely to answer our poll calls while conservatives remained harder to reach,' they write.
Late-break not considered controlling
The authors reference theories about a similar polling error in 2016, which may have missed a late-break among voters that handed Trump a surprise win over Hillary Clinton. But they note the stability of polling in 2020, and say this was not likely the driver this time.
How to avoid making the same mistakes again
The pollsters put forward a few potential solutions, but don't attempt to put forth a detailed solution that would fix polling if Donald Trump runs again in 2024, as he might. One idea is to go back to door-to-door surveys of an earlier era. They also talk about paying survey subjects to induce participation, and developing 'multi-modal research tools.'
They point out that the error factor may be partly specific to Trump, so they caution against overcorrecting against someone they say will 'hopefully' never be on the ballot again.
No comments