Remember the chaotic election last November? How could we forget! If you followed the polls and listened to pundits, you could be forgiven for expecting Hillary Clinton to win by a large margin. But that’s not what happened, and the outcome surprised even some established pollsters, prompting an investigation of the election polls, which six months later has now been completed.

The new report released today [I’ll include a link here] by the American Association for Public Opinion Research finds that, no, the polls aren’t “broken.” National polls fared pretty well, while Midwestern state polls missed a big segment of voters leaning toward Donald Trump. The letter from FBI director James Comey about the Clinton email probe appeared to cause a small temporary shift, and quite a few undecided voters broke for Trump at the last minute, but despite that Clinton won the national vote by 2.1%, almost as much as forecast (3%) and well within the uncertainties. But Trump’s victory in states like Wisconsin, Michigan, and Pennsylvania, where non-college educated suburban white people were politicized and turned out in large numbers, caught experts off guard. (Polls gave Clinton the edge in those three states, and had she won all of them, she would’ve won with 278 vs 260 electoral votes.)

The popular vote went for Clinton, but in the end, that doesn’t matter. Pollsters need to spend more time and effort on state polls, especially in “battleground” states, to improve those polls and more accurately get the voting preferences of all segments of the population of likely voters, including those who don’t respond as often to pollsters’ phone calls or online surveys. State polls throughout the upper Midwest seem to have missed non-college educated and working-class whites in suburban and rural areas, who turned out the vote more than expected and preferentially for Trump.

There’s a couple other findings worth noting, but I’d like to make a more important point about how polls are interpreted. Many people seem to have banked on Clinton’s narrow national lead, which did not translate into an Electoral College win. It wasn’t even close. “The horse race becomes interesting news fodder, but it’s misleading,” Michael Link, past president of AAPOR, tells me.

My sources at Ipsos and USC/LA Times concur. Polls involve collecting survey data, including data that point to whether people are likely to vote, weighting the numbers to account for who’s over- and under-represented, and estimating an error. It’s possible for many polls to miss the same portions of the population and then they’d all be biased. In any case, turning poll numbers into a forecast is a whole new ball game, and what you get is just a probability. If Clinton had a 60% chance of winning, that means Trump had a 4 out of 10 chance, which is very high. I wouldn’t bet on that. Even a 3% chance of losing is just a roll of the dice (1/36).

The fact is, lots of people drank their own Kool-Aid. They believed or wanted to believe Clinton would win, and couldn’t conceive of the possibility that the tide might turn toward Trump. When day after day you hear about Clinton leading, and some self-declared experts claim that she’s a shoo-in, it’s hard to remain skeptical. We have to be more careful and give those polls and surveys a sober look.

For more information about the changing nature of polling, check out my feature story and my news coverage on election night. Courtney Kennedy of Pew, lead author of this polling postmortem, and Lee Miringoff of Marist, a co-author, were among my sources.

Here’s some other misconceptions highlighted by the report:
1. Despite some claims (such as by Nate Silver), the Comey letter probably did not cost Clinton the election. First, it’s hard to distinguish the effects of an individual event. Second, the small apparent shift in the polls didn’t last, and support for Clinton ticked up right before the election. Based on all the data, the report concludes, “There is at best mixed evidence to suggest that the FBI announcement tipped the scales of the race.”
2. There’s no evidence of a large number of “shy Trump” voters. The idea is that Trump’s controversial statements could’ve made it uncomfortable for some people to disclose their support for him to a pollster. But unlike in the UK (where occasionally people observe a “shy Tory” effect), for example, in the US, conservatives don’t seem any less likely than liberals to tell someone their true voting preferences. The fact that live phone polls and online polls had similar numbers confirms this. The report concludes, “There is no consistent partisan favoritism in recent US polling.”
3. Other factors, like a change in turnout between 2012 and 2016, and ballot order effects in some states, don’t explain polling errors.