The Unreliability of Election Polls

Utkarsh Tandon

     For the past several days we have been nervously checking election predictions to prepare ourselves for the announcement of America’s new president. Now we finally have the results — but were they at all what we expected?

     Prediction websites made broad claims such as, “Clinton has a 72.54% chance of winning,” but they also backed these numbers with detailed analysis of swing states and which ones would make the definitive difference. Now that the election has completely unfolded, it is possible to analyze the accuracy of these predictions — not necessarily whether the winner was correctly predicted, but whether the number crunching behind these swing state projections was accurate.

     To begin, the diagram below depicts what analysts at FiveThirtyEight (statistics website) predicted a couple hours before polling closed

   

Courtesy of FiveThirtyEight

     The above infographic clearly shows that Clinton had all the swing states under her belt: Michigan, Pennsylvania, North Carolina and even Florida. But when polls closed and results poured out, she lost each and every one of those states. Although Clinton won a few predicted tight swing states such as Nevada, she lost in states such as Wisconsin and Michigan — states that FiveThirtyEight gave low weightage in deciding the election. However, in reality these states were the ones that made all the difference in the end. Predictions were thoroughly different from the actual outcome; the two below graphs illustrate the extent to which these differences materialized. Graph 1 delineates the discrepancies in predicted swing state importance to the election, and Graph 2 depicts the overall error in ballot count predictions over the 50 states.

Wisconsin, as well as Michigan, turned out to be highly important in tipping the election, which few predicted since Obama easily won the states in the ‘08 and ‘12 elections. FiveThirtyEight gave Wisconsin a small 4.8% chance in tipping the election as it originally gave Clinton a 85.3% advantage in winning the state. In reality however, Trump won Wisconsin with a slim one point lead, which defied all predictions. Additionally, Pennsylvania’s outcome was a major surprise, and had a similarly significant influence in tipping to the election in Trump’s favor. One interesting trend seen in Graph 2 is that FiveThirtyEight accurately predicted vote distributions to the exact percent for many states that Clinton was assured to win. But for even slightly contested states and states that Trump was assured to win, they were significantly incorrect in their predictions. This shows a potential reason for error — that analysts weighted predictions from previous year Democrat successes more heavily than current policies and regional demographics. Although error to some extent is expected, analysts displayed an inability to classify relative swing state importance and had a sizable 7.23 percent error in predictions across the 50 states. This effectively shows that current systems are inadequate in providing accurate predictions, especially in unique and ground-breaking elections such as this one.

     The primary reason the election went in Trump’s favor was because many previous Obama voters shifted their opinion from a structured policy to a more revolutionary way of running the country due to higher levels of unemployment among voters. Many felt that radical change would be the only method of bringing their jobs back —  something that Trump strongly pushed for in his campaign.

     Only two out of dozens of well regarded newspapers predicted a Trump victory. Most sources had given Clinton over a 70% chance of winning the election, and the highly regarded Nate Silver, who had accurately predicted all 50 states in the previous presidential election, was considerably wrong on four major swing states. This election clearly goes down as one of the most unexpected Presidential victories in U.S. history. What Trump will do in the future is unclear, but the 2016 election sheds important light on the volatile nature of our country’s opinions — to the extent that even the most reputable statisticians cannot accurately predict its end result.