• <div class="header-image" style="background-image: url(/live/image/gid/4/2801_V6N8_Header.rev.1537380633.jpg);">​</div><div class="header-background-color"/>

Forecasting the 2016 Election Event Recap

October 25, 2016
On October 25th, the Penn Wharton Public Policy Initiative welcomed David Rothschild to discuss his research and the science of polling as it relates to the 2016 Presidential Election. 

An economist at Microsoft Research and the creator of the event forecasting tool PredictWise, Rothschild’s primary body of work focuses on forecasting and understanding public interest and sentiment. He has written extensively, in both the academic and popular press, on polling, prediction markets, social media and online data, and predictions of upcoming events, especially elections. Since joining Microsoft in 2012, he has been building prediction and sentiment models, and organizing novel/experimental polling and prediction games. This work has been utilized by Bing, Cortana, and Xbox. He has a Ph.D. in applied economics from the Wharton School of the University of Pennsylvania.

David began by explaining how he thinks about using empirical research to inform policy, and how the election cycle gives us a unique way to look back on how data has informed past policies, too. First, he emphasized the importance of understanding where public policy and opinion come from. The idea of traditional polling is built on the concept of a target population of likely voters within a country or state. There must be a non-zero, known probability of contacting anyone within that target population, and theoretically, after taking a small sample, the result you want to predict can be determined within a few percentage points of accuracy.

David Rothschild Forecasting the 2016 Election

Unfortunately, this polling method faces more complications in reality. In the case of polling for the election, many people who are likely voters are unlikely to be found – they have no landlines, no cell phones, and can’t be identified via an online poll. In addition to this coverage error, non-response error can also skew results. If individuals are randomly selected but choose not to answer, this can have an impact on the poll results.

David illustrated his point through the specific example of the Sanders vs. Trump vs. Bloomberg electoral race. When Mayor Bloomberg was deciding whether he should run for President, he hired pollster Douglas Schoen to find out his probability of winning. Schoen produced results saying Bloomberg would win, but he was fundamentally flawed in his conclusions. David reminded us that we will never know the likelihood of what will happen for an event that has never happened! There is no way to judge its accuracy. 

David Rothschild Forecasting the 2016 Election

Another example of bias in polling can be found in Obama’s approval rating. Although his ratings were extremely stable, they have been shooting up in the past few months. Why? Because he does extremely well when people are asked about his replacements. So, is this an artifact of polling? And do we care? Schoen made the same error in his polling for Bloomberg – before people answered his questions, he reminded them of all of the Mayor’s accomplishments to make sure they knew who he was.

The entire industry of polling is built on this idea of probability polling, but because we don’t have comparable for outcomes most things (like Obama’s true approval rating or whether Mayor Bloomberg would have won the election), we need a good procedure for getting ideas so that we can trust that the outcome is meaningful.

When you average multiple polls, the impact of all of these errors is mitigated. In an aggregate poll, the predicted answer is never more than 2% away from the actual answer. So for David, this raised the question of whether you can take simple, cheap and fast samples and make the data useful.

Using an online poll from the MSN homepage, he and his team managed to transform a data set heavily skewed towards white, older males and make it representative of the entire population. In traditional polls, the respondents of the poll will have demographics of gender, age, race, geography and more attached to them. These respondents are given a value between 0 and 10 to weight them – if there are too few women, all women are given a weighting greater than one, and if there are too many men, they are given a rating of less than one. This method is flawed, however, since it can place too much weight on any one person who happens to be part of an underrepresented population. 

What David did instead was look at a much larger set of demographics – state, age, race, sex, latent party identification, education level, and basically any other variable that distinguishes one voter from another. Then, he can create hundreds of thousands of unique combinations of these variables, to partition the population to very small, cleanly represented groups. Then, he takes the survey data and makes a model describing how any one of these groups would have responded to the poll. Finally, these predicted results are compared to the responses of any given respondent. An older, white male voter is then described by the trends of the entire male population, the entire older population, the entire white population, etc. This allows for a more accurate polling result that factors in every piece of historical data available to the pollsters.

David Rothschild Forecasting the 2016 Election

David continues to use this methodology to predict this year’s election, collect data on the greatest NBA player of all time and more – you can see the results on MSN.com or on his blog, PredictWise.com.

 

PENN WHARTON PPI
RESOURCE SPOTLIGHT:

  • <h3>The World Bank Data (U.S.)</h3><p><img width="130" height="118" alt="" src="/live/image/gid/4/width/130/height/118/484_world-bank-logo.rev.1407788945.jpg" class="lw_image lw_image484 lw_align_left" srcset="/live/image/scale/2x/gid/4/width/130/height/118/484_world-bank-logo.rev.1407788945.jpg 2x, /live/image/scale/3x/gid/4/width/130/height/118/484_world-bank-logo.rev.1407788945.jpg 3x" data-max-w="1406" data-max-h="1275"/>The <strong>World Bank</strong> provides World Development Indicators, Surveys, and data on Finances and Climate Change.</p><p> Quick link: <a href="http://data.worldbank.org/country/united-states" target="_blank">http://data.worldbank.org/country/united-states</a></p><p>See all <a href="/data-resources/">data and resources</a> »</p>
  • <h3>National Center for Education Statistics</h3><p><strong><img width="400" height="80" alt="" src="/live/image/gid/4/width/400/height/80/479_nces.rev.1407787656.jpg" class="lw_image lw_image479 lw_align_right" data-max-w="400" data-max-h="80"/>The National Center for Education Statistics (NCES) is the primary federal entity for collecting and analyzing data related to education in the U.S. and other nations.</strong> NCES is located within the U.S. Department of Education and the Institute of Education Sciences. NCES has an extensive Statistical Standards Program that consults and advises on methodological and statistical aspects involved in the design, collection, and analysis of data collections in the Center. To learn more about the NCES, <a href="http://nces.ed.gov/about/" target="_blank">click here</a>.</p><p> Quick link to NCES Data Tools: <a href="http://nces.ed.gov/datatools/index.asp?DataToolSectionID=4" target="_blank">http://nces.ed.gov/datatools/index.asp?DataToolSectionID=4</a></p><p> Quick link to Quick Tables and Figures: <a href="http://nces.ed.gov/quicktables/" target="_blank">http://nces.ed.gov/quicktables/</a></p><p> Quick link to NCES Fast Facts (Note: The primary purpose of the Fast Facts website is to provide users with concise information on a range of educational issues, from early childhood to adult learning.): <a href="http://nces.ed.gov/fastfacts/" target="_blank">http://nces.ed.gov/fastfacts/#</a></p><p>See all <a href="/data-resources/">data and resources</a> »</p>
  • <h3>Federal Aviation Administration: Accident & Incident Data</h3><p><img width="100" height="100" alt="" src="/live/image/gid/4/width/100/height/100/80_faa-logo.rev.1402681347.jpg" class="lw_image lw_image80 lw_align_left" srcset="/live/image/scale/2x/gid/4/width/100/height/100/80_faa-logo.rev.1402681347.jpg 2x, /live/image/scale/3x/gid/4/width/100/height/100/80_faa-logo.rev.1402681347.jpg 3x" data-max-w="550" data-max-h="550"/>The NTSB issues an accident report following each investigation. These reports are available online for reports issued since 1996, with older reports coming online soon. The reports listing is sortable by the event date, report date, city, and state.</p><p> Quick link: <a href="http://www.faa.gov/data_research/accident_incident/" target="_blank">http://www.faa.gov/data_research/accident_incident/</a></p><p>See all <a href="/data-resources/">data and resources</a> »</p>
  • <h3>USDA Nutrition Assistance Data</h3><p><img width="180" height="124" alt="" src="/live/image/gid/4/width/180/height/124/485_usda_logo.rev.1407789238.jpg" class="lw_image lw_image485 lw_align_right" srcset="/live/image/scale/2x/gid/4/width/180/height/124/485_usda_logo.rev.1407789238.jpg 2x, /live/image/scale/3x/gid/4/width/180/height/124/485_usda_logo.rev.1407789238.jpg 3x" data-max-w="1233" data-max-h="850"/>Data and research regarding the following <strong>USDA Nutrition Assistance</strong> programs are available through this site:</p><ul><li>Supplemental Nutrition Assistance Program (SNAP) </li><li>Food Distribution Programs </li><li>School Meals </li><li>Women, Infants and Children </li></ul><p> Quick link: <a href="http://www.fns.usda.gov/data-and-statistics" target="_blank">http://www.fns.usda.gov/data-and-statistics</a></p><p>See all <a href="/data-resources/">data and resources</a> »</p>
  • <h3>Internal Revenue Service: Tax Statistics</h3><p><img width="155" height="200" alt="" src="/live/image/gid/4/width/155/height/200/486_irs_logo.rev.1407789424.jpg" class="lw_image lw_image486 lw_align_left" srcset="/live/image/scale/2x/gid/4/width/155/height/200/486_irs_logo.rev.1407789424.jpg 2x" data-max-w="463" data-max-h="596"/>Find statistics on business tax, individual tax, charitable and exempt organizations, IRS operations and budget, and income (SOI), as well as statistics by form, products, publications, papers, and other IRS data.</p><p> Quick link to <strong>Tax Statistics, where you will find a wide range of tables, articles, and data</strong> that describe and measure elements of the U.S. tax system: <a href="http://www.irs.gov/uac/Tax-Stats-2" target="_blank">http://www.irs.gov/uac/Tax-Stats-2</a></p><p>See all <a href="/data-resources/">data and resources</a> »</p>
  • <h3>National Bureau of Economic Research (Public Use Data Archive)</h3><p><img width="180" height="43" alt="" src="/live/image/gid/4/width/180/height/43/478_nber.rev.1407530465.jpg" class="lw_image lw_image478 lw_align_right" data-max-w="329" data-max-h="79"/>Founded in 1920, the <strong>National Bureau of Economic Research</strong> is a private, nonprofit, nonpartisan research organization dedicated to promoting a greater understanding of how the economy works. The NBER is committed to undertaking and disseminating unbiased economic research among public policymakers, business professionals, and the academic community.</p><p> Quick Link to <strong>Public Use Data Archive</strong>: <a href="http://www.nber.org/data/" target="_blank">http://www.nber.org/data/</a></p><p>See all <a href="/data-resources/">data and resources</a> »</p>
  • <h3>MapStats</h3><p> A feature of FedStats, MapStats allows users to search for <strong>state, county, city, congressional district, or Federal judicial district data</strong> (demographic, economic, and geographic).</p><p> Quick link: <a href="http://www.fedstats.gov/mapstats/" target="_blank">http://www.fedstats.gov/mapstats/</a></p><p>See all <a href="/data-resources/">data and resources</a> »</p>
  • <h3>The Penn World Table</h3><p> The Penn World Table provides purchasing power parity and national income accounts converted to international prices for 189 countries/territories for some or all of the years 1950-2010.</p><p><a href="https://pwt.sas.upenn.edu/php_site/pwt71/pwt71_form.php" target="_blank">Quick link.</a> </p><p>See all <a href="/data-resources/">data and resources</a> »</p>
  • <h3>HUD State of the Cities Data Systems</h3><p><strong><img width="200" height="200" alt="" src="/live/image/gid/4/width/200/height/200/482_hud_logo.rev.1407788472.jpg" class="lw_image lw_image482 lw_align_left" srcset="/live/image/scale/2x/gid/4/width/200/height/200/482_hud_logo.rev.1407788472.jpg 2x, /live/image/scale/3x/gid/4/width/200/height/200/482_hud_logo.rev.1407788472.jpg 3x" data-max-w="612" data-max-h="613"/>The SOCDS provides data for individual Metropolitan Areas, Central Cities, and Suburbs.</strong> It is a portal for non-national data made available through a number of outside institutions (e.g. Census, BLS, FBI and others).</p><p> Quick link: <a href="http://www.huduser.org/portal/datasets/socds.html" target="_blank">http://www.huduser.org/portal/datasets/socds.html</a></p><p>See all <a href="/data-resources/">data and resources</a> »</p>
  • <h3>NOAA National Climatic Data Center</h3><p><img width="200" height="198" alt="" src="/live/image/gid/4/width/200/height/198/483_noaa_logo.rev.1407788692.jpg" class="lw_image lw_image483 lw_align_left" srcset="/live/image/scale/2x/gid/4/width/200/height/198/483_noaa_logo.rev.1407788692.jpg 2x, /live/image/scale/3x/gid/4/width/200/height/198/483_noaa_logo.rev.1407788692.jpg 3x" data-max-w="954" data-max-h="945"/>NOAA’s National Climatic Data Center (NCDC) is responsible for preserving, monitoring, assessing, and providing public access to the Nation’s treasure of <strong>climate and historical weather data and information</strong>.</p><p> Quick link to home page: <a href="http://www.ncdc.noaa.gov/" target="_blank">http://www.ncdc.noaa.gov/</a></p><p> Quick link to NCDC’s climate and weather datasets, products, and various web pages and resources: <a href="http://www.ncdc.noaa.gov/data-access/quick-links" target="_blank">http://www.ncdc.noaa.gov/data-access/quick-links</a></p><p> Quick link to Text & Map Search: <a href="http://www.ncdc.noaa.gov/cdo-web/" target="_blank">http://www.ncdc.noaa.gov/cdo-web/</a></p><p>See all <a href="/data-resources/">data and resources</a> »</p>
  • <h3>Congressional Budget Office</h3><p><img width="180" height="180" alt="" src="/live/image/gid/4/width/180/height/180/380_cbo-logo.rev.1406822035.jpg" class="lw_image lw_image380 lw_align_right" data-max-w="180" data-max-h="180"/>Since its founding in 1974, the Congressional Budget Office (CBO) has produced independent analyses of budgetary and economic issues to support the Congressional budget process.</p><p> The agency is strictly nonpartisan and conducts objective, impartial analysis, which is evident in each of the dozens of reports and hundreds of cost estimates that its economists and policy analysts produce each year. CBO does not make policy recommendations, and each report and cost estimate discloses the agency’s assumptions and methodologies. <strong>CBO provides budgetary and economic information in a variety of ways and at various points in the legislative process.</strong> Products include baseline budget projections and economic forecasts, analysis of the President’s budget, cost estimates, analysis of federal mandates, working papers, and more.</p><p> Quick link to Products page: <a href="http://www.cbo.gov/about/our-products" target="_blank">http://www.cbo.gov/about/our-products</a></p><p> Quick link to Topics: <a href="http://www.cbo.gov/topics" target="_blank">http://www.cbo.gov/topics</a></p><p>See all <a href="/data-resources/">data and resources</a> »</p>
  • <h3>Federal Reserve Economic Data (FRED®)</h3><p><strong><img width="180" height="79" alt="" src="/live/image/gid/4/width/180/height/79/481_fred-logo.rev.1407788243.jpg" class="lw_image lw_image481 lw_align_right" data-max-w="222" data-max-h="97"/>An online database consisting of more than 72,000 economic data time series from 54 national, international, public, and private sources.</strong> FRED®, created and maintained by Research Department at the Federal Reserve Bank of St. Louis, goes far beyond simply providing data: It combines data with a powerful mix of tools that help the user understand, interact with, display, and disseminate the data.</p><p> Quick link to data page: <a href="http://research.stlouisfed.org/fred2/tags/series" target="_blank">http://research.stlouisfed.org/fred2/tags/series</a></p><p>See all <a href="/data-resources/">data and resources</a> »</p>