• <div class="header-image" style="background-image: url(/live/image/gid/4/2897_V6N9_Header.rev.1540219621.jpg);">​</div><div class="header-background-color"/>

Does Big Data Really Make Economic Policy More Effective?

October 30, 2016
“Big Data” has transcended its usual domain of Silicon Valley and Wall Street to become a major buzzword in the policymaking world. Nowhere is this more visible than in the recent influential work of Stanford economist Raj Chetty and colleagues, who have made use of millions of IRS tax records—confidential information previously unavailable to researchers—to study important topics ranging from teacher effectiveness [1] to the determinants of upward mobility among low-income households in America [2].

By Ben Hyman 

Beyond its influence in academia and coverage in well-known media outlets [3], this kind of work is so novel and powerful that even presidential nominee Hillary Clinton purportedly met with Chetty for a 2-hour discussion about his findings and their implications for policy [4]. By using IRS records that link parental federal income filings to their children’s future tax returns, Chetty’s research team was able to track children’s economic outcomes years after being exposed to high or low quality teachers, and what happened to the future income of children of equal socioeconomic backgrounds when they moved to different local economies, cities, and neighborhoods.

What makes this kind of data so appealing is not only that millions of observations lead to great precision in estimates—generating policy-friendly statements such as “a one tax dollar increase in X results in a Y increase in Z”—but also that Big Data make results widely generalizable. This stands in contrast to another more experimental movement [5] in economic policy evaluation toward randomized controlled trials (RCTs)—social experiments mimicking clinical trials to test the effects of various economic interventions. RCTs tend to score highly on getting the causal relationship of an intervention right, but poorly in their generalizability to the larger economy.

Despite progress in the “answers” to important debates, there are many questions in economic policy that we think we already know the answers to. This forces us to ask ourselves whether a “lack of Big Data” has really been the great constraint to unlocking prosperity-conducive policies in this country. Two topics economists generally reach consensus on are the broad effects of trade liberalization on jobs (such as the recent debate over the Trans-Pacific Partnership trade deal), and rent control policies on housing affordability. Economists—famous for “two-handed” responses that present two sides of any story—uncharacteristically agree on these issues, as indicated by two recent polls of economists on the effects of trade deals [6] and rent control [7].

Political scientists studying voting behavior have competing theories for why voters may not support issues that are likely in their self-interest. As Gregory Mankiw notes, “…people are more altruistic than economists usually think they are. But it’s also possible that it is just hard for people to determine with much precision how national policies affect their personal circumstances…” [8] Mankiw goes on to cite political scientists Edward Mansfield and Diana Mutz at the University of Pennsylvania who suggest that “…preferences are driven less by economic considerations and more by an individual’s psychological worldview” (New York Times, July 2016).

Despite this healthy skepticism, researchers move forward on the premise that large and more tangible studies such as Chetty’s can help bridge information gaps and better highlight the winners and losers from different policies. While not all of us are future Nobel Laureates with the connections and entrepreneurialism required to attain confidential IRS tax data [9], a venue does exist for researchers that are willing to advance data collection efforts at the U.S. Census Bureau to make use of big data.

While in Washington D.C., I am working as a “Special Sworn Status” researcher at one of the U.S. Census Bureau’s 23 federal statistics Research Data Centers (RDCs). At the Census, I am working on a project to analyze the effects of local tax credit policies on the behavior of thousands of large employers in California, to help determine whether these incentives are effective at keeping good jobs in the state, or whether and in which context they may detrimental to the economic welfare of the state. Increasing access to big data such as those discussed here continue to expand our understanding of important questions. Whether they help bridge the gap between voters and issues however, remains to be seen.

References

  [1] Chetty, Raj, John N. Friedman, and Jonah E. Rockoff. “Measuring the impacts of teachers I and II: Evaluating bias in teacher value-added estimates.” The American Economic Review 104.9 (2014): 2593-2632 and 2633-2679, and http://www.rajchetty.com/chettyfiles/value_added.htm.

  [2] Chetty, R., Hendren, N., Kline, P., & Saez, E. (2014). Where is the land of Opportunity? The Geography of Intergenerational Mobility in the United States. The Quarterly Journal of Economics, 129(4), 1553-1623, and http://www.equality-of-opportunity.org/

  [3] David Leonhardt. “In Climbing Income Ladder Location Matters.”  New York Times, July 22, 2013.  http://www.nytimes.com/2013/07/22/business/in-climbing-income-ladder-location-matters.html?pagewanted=all

  [4] Bob Davis. “Economist Raj Chetty’s Proposals on Inequality Draw Interest on Both Sides of the Political Aisle.”  Wall Street Journal, October 20, 2015.   http://www.wsj.com/articles/economist-raj-chettys-proposals-on-inequality-draw-interest-on-both-sides-of-the-political-aisle-1445383469

  [5] Ian Parker. “The Poverty Lab. Transforming development economics, one experiment at a time.”  The New Yorker, May 17, 2010.    http://www.newyorker.com/magazine/2010/05/17/the-poverty-lab

  [6] IGM Experts Panel, Unviersity of Chicago, Booth School of Business, November 11, 2014. http://www.igmchicago.org/igm-economic-experts-panel/poll-results?SurveyID=SV_d68906VNWqVmiGN

  [7] IGM Experts Panel, Unviersity of Chicago, Booth School of Business, Febuary 7, 2011.  http://www.igmchicago.org/igm-economic-experts-panel/poll-results?SurveyID=SV_6upyzeUpI73V5k0

  [8] N. Gregory Mankiw. “Why Voters Don’t Buy It When Economists Say Global Trade is Good.” The New York Times, The Upshot, July 29, 2016. http://www.nytimes.com/2016/07/31/upshot/why-voters-dont-buy-it-when-economists-say-global-trade-is-good.html

  [9] Jeffrey Mervis. “How Two Economists Got Direct Access to IRS Tax Records.” Science Magazine. May 22, 2014. http://www.sciencemag.org/news/2014/05/how-two-economists-got-direct-access-irs-tax-records

Student Blog Disclaimer
  • The views expressed on the Student Blog are the author’s opinions and don’t necessarily represent the Penn Wharton Public Policy Initiative’s strategies, recommendations, or opinions.

PENN WHARTON PPI
RESOURCE SPOTLIGHT:

  • <h3>Internal Revenue Service: Tax Statistics</h3><p><img width="155" height="200" alt="" src="/live/image/gid/4/width/155/height/200/486_irs_logo.rev.1407789424.jpg" class="lw_image lw_image486 lw_align_left" srcset="/live/image/scale/2x/gid/4/width/155/height/200/486_irs_logo.rev.1407789424.jpg 2x" data-max-w="463" data-max-h="596"/>Find statistics on business tax, individual tax, charitable and exempt organizations, IRS operations and budget, and income (SOI), as well as statistics by form, products, publications, papers, and other IRS data.</p><p> Quick link to <strong>Tax Statistics, where you will find a wide range of tables, articles, and data</strong> that describe and measure elements of the U.S. tax system: <a href="http://www.irs.gov/uac/Tax-Stats-2" target="_blank">http://www.irs.gov/uac/Tax-Stats-2</a></p><p>See all <a href="/data-resources/">data and resources</a> »</p>
  • <h3>National Bureau of Economic Research (Public Use Data Archive)</h3><p><img width="180" height="43" alt="" src="/live/image/gid/4/width/180/height/43/478_nber.rev.1407530465.jpg" class="lw_image lw_image478 lw_align_right" data-max-w="329" data-max-h="79"/>Founded in 1920, the <strong>National Bureau of Economic Research</strong> is a private, nonprofit, nonpartisan research organization dedicated to promoting a greater understanding of how the economy works. The NBER is committed to undertaking and disseminating unbiased economic research among public policymakers, business professionals, and the academic community.</p><p> Quick Link to <strong>Public Use Data Archive</strong>: <a href="http://www.nber.org/data/" target="_blank">http://www.nber.org/data/</a></p><p>See all <a href="/data-resources/">data and resources</a> »</p>
  • <h3>MapStats</h3><p> A feature of FedStats, MapStats allows users to search for <strong>state, county, city, congressional district, or Federal judicial district data</strong> (demographic, economic, and geographic).</p><p> Quick link: <a href="http://www.fedstats.gov/mapstats/" target="_blank">http://www.fedstats.gov/mapstats/</a></p><p>See all <a href="/data-resources/">data and resources</a> »</p>
  • <h3>National Center for Education Statistics</h3><p><strong><img width="400" height="80" alt="" src="/live/image/gid/4/width/400/height/80/479_nces.rev.1407787656.jpg" class="lw_image lw_image479 lw_align_right" data-max-w="400" data-max-h="80"/>The National Center for Education Statistics (NCES) is the primary federal entity for collecting and analyzing data related to education in the U.S. and other nations.</strong> NCES is located within the U.S. Department of Education and the Institute of Education Sciences. NCES has an extensive Statistical Standards Program that consults and advises on methodological and statistical aspects involved in the design, collection, and analysis of data collections in the Center. To learn more about the NCES, <a href="http://nces.ed.gov/about/" target="_blank">click here</a>.</p><p> Quick link to NCES Data Tools: <a href="http://nces.ed.gov/datatools/index.asp?DataToolSectionID=4" target="_blank">http://nces.ed.gov/datatools/index.asp?DataToolSectionID=4</a></p><p> Quick link to Quick Tables and Figures: <a href="http://nces.ed.gov/quicktables/" target="_blank">http://nces.ed.gov/quicktables/</a></p><p> Quick link to NCES Fast Facts (Note: The primary purpose of the Fast Facts website is to provide users with concise information on a range of educational issues, from early childhood to adult learning.): <a href="http://nces.ed.gov/fastfacts/" target="_blank">http://nces.ed.gov/fastfacts/#</a></p><p>See all <a href="/data-resources/">data and resources</a> »</p>
  • <h3>Federal Reserve Economic Data (FRED®)</h3><p><strong><img width="180" height="79" alt="" src="/live/image/gid/4/width/180/height/79/481_fred-logo.rev.1407788243.jpg" class="lw_image lw_image481 lw_align_right" data-max-w="222" data-max-h="97"/>An online database consisting of more than 72,000 economic data time series from 54 national, international, public, and private sources.</strong> FRED®, created and maintained by Research Department at the Federal Reserve Bank of St. Louis, goes far beyond simply providing data: It combines data with a powerful mix of tools that help the user understand, interact with, display, and disseminate the data.</p><p> Quick link to data page: <a href="http://research.stlouisfed.org/fred2/tags/series" target="_blank">http://research.stlouisfed.org/fred2/tags/series</a></p><p>See all <a href="/data-resources/">data and resources</a> »</p>
  • <h3>NOAA National Climatic Data Center</h3><p><img width="200" height="198" alt="" src="/live/image/gid/4/width/200/height/198/483_noaa_logo.rev.1407788692.jpg" class="lw_image lw_image483 lw_align_left" srcset="/live/image/scale/2x/gid/4/width/200/height/198/483_noaa_logo.rev.1407788692.jpg 2x, /live/image/scale/3x/gid/4/width/200/height/198/483_noaa_logo.rev.1407788692.jpg 3x" data-max-w="954" data-max-h="945"/>NOAA’s National Climatic Data Center (NCDC) is responsible for preserving, monitoring, assessing, and providing public access to the Nation’s treasure of <strong>climate and historical weather data and information</strong>.</p><p> Quick link to home page: <a href="http://www.ncdc.noaa.gov/" target="_blank">http://www.ncdc.noaa.gov/</a></p><p> Quick link to NCDC’s climate and weather datasets, products, and various web pages and resources: <a href="http://www.ncdc.noaa.gov/data-access/quick-links" target="_blank">http://www.ncdc.noaa.gov/data-access/quick-links</a></p><p> Quick link to Text & Map Search: <a href="http://www.ncdc.noaa.gov/cdo-web/" target="_blank">http://www.ncdc.noaa.gov/cdo-web/</a></p><p>See all <a href="/data-resources/">data and resources</a> »</p>
  • <h3>USDA Nutrition Assistance Data</h3><p><img width="180" height="124" alt="" src="/live/image/gid/4/width/180/height/124/485_usda_logo.rev.1407789238.jpg" class="lw_image lw_image485 lw_align_right" srcset="/live/image/scale/2x/gid/4/width/180/height/124/485_usda_logo.rev.1407789238.jpg 2x, /live/image/scale/3x/gid/4/width/180/height/124/485_usda_logo.rev.1407789238.jpg 3x" data-max-w="1233" data-max-h="850"/>Data and research regarding the following <strong>USDA Nutrition Assistance</strong> programs are available through this site:</p><ul><li>Supplemental Nutrition Assistance Program (SNAP) </li><li>Food Distribution Programs </li><li>School Meals </li><li>Women, Infants and Children </li></ul><p> Quick link: <a href="http://www.fns.usda.gov/data-and-statistics" target="_blank">http://www.fns.usda.gov/data-and-statistics</a></p><p>See all <a href="/data-resources/">data and resources</a> »</p>
  • <h3>Congressional Budget Office</h3><p><img width="180" height="180" alt="" src="/live/image/gid/4/width/180/height/180/380_cbo-logo.rev.1406822035.jpg" class="lw_image lw_image380 lw_align_right" data-max-w="180" data-max-h="180"/>Since its founding in 1974, the Congressional Budget Office (CBO) has produced independent analyses of budgetary and economic issues to support the Congressional budget process.</p><p> The agency is strictly nonpartisan and conducts objective, impartial analysis, which is evident in each of the dozens of reports and hundreds of cost estimates that its economists and policy analysts produce each year. CBO does not make policy recommendations, and each report and cost estimate discloses the agency’s assumptions and methodologies. <strong>CBO provides budgetary and economic information in a variety of ways and at various points in the legislative process.</strong> Products include baseline budget projections and economic forecasts, analysis of the President’s budget, cost estimates, analysis of federal mandates, working papers, and more.</p><p> Quick link to Products page: <a href="http://www.cbo.gov/about/our-products" target="_blank">http://www.cbo.gov/about/our-products</a></p><p> Quick link to Topics: <a href="http://www.cbo.gov/topics" target="_blank">http://www.cbo.gov/topics</a></p><p>See all <a href="/data-resources/">data and resources</a> »</p>
  • <h3>Federal Aviation Administration: Accident & Incident Data</h3><p><img width="100" height="100" alt="" src="/live/image/gid/4/width/100/height/100/80_faa-logo.rev.1402681347.jpg" class="lw_image lw_image80 lw_align_left" srcset="/live/image/scale/2x/gid/4/width/100/height/100/80_faa-logo.rev.1402681347.jpg 2x, /live/image/scale/3x/gid/4/width/100/height/100/80_faa-logo.rev.1402681347.jpg 3x" data-max-w="550" data-max-h="550"/>The NTSB issues an accident report following each investigation. These reports are available online for reports issued since 1996, with older reports coming online soon. The reports listing is sortable by the event date, report date, city, and state.</p><p> Quick link: <a href="http://www.faa.gov/data_research/accident_incident/" target="_blank">http://www.faa.gov/data_research/accident_incident/</a></p><p>See all <a href="/data-resources/">data and resources</a> »</p>
  • <h3>HUD State of the Cities Data Systems</h3><p><strong><img width="200" height="200" alt="" src="/live/image/gid/4/width/200/height/200/482_hud_logo.rev.1407788472.jpg" class="lw_image lw_image482 lw_align_left" srcset="/live/image/scale/2x/gid/4/width/200/height/200/482_hud_logo.rev.1407788472.jpg 2x, /live/image/scale/3x/gid/4/width/200/height/200/482_hud_logo.rev.1407788472.jpg 3x" data-max-w="612" data-max-h="613"/>The SOCDS provides data for individual Metropolitan Areas, Central Cities, and Suburbs.</strong> It is a portal for non-national data made available through a number of outside institutions (e.g. Census, BLS, FBI and others).</p><p> Quick link: <a href="http://www.huduser.org/portal/datasets/socds.html" target="_blank">http://www.huduser.org/portal/datasets/socds.html</a></p><p>See all <a href="/data-resources/">data and resources</a> »</p>
  • <h3>The Penn World Table</h3><p> The Penn World Table provides purchasing power parity and national income accounts converted to international prices for 189 countries/territories for some or all of the years 1950-2010.</p><p><a href="https://pwt.sas.upenn.edu/php_site/pwt71/pwt71_form.php" target="_blank">Quick link.</a> </p><p>See all <a href="/data-resources/">data and resources</a> »</p>
  • <h3>The World Bank Data (U.S.)</h3><p><img width="130" height="118" alt="" src="/live/image/gid/4/width/130/height/118/484_world-bank-logo.rev.1407788945.jpg" class="lw_image lw_image484 lw_align_left" srcset="/live/image/scale/2x/gid/4/width/130/height/118/484_world-bank-logo.rev.1407788945.jpg 2x, /live/image/scale/3x/gid/4/width/130/height/118/484_world-bank-logo.rev.1407788945.jpg 3x" data-max-w="1406" data-max-h="1275"/>The <strong>World Bank</strong> provides World Development Indicators, Surveys, and data on Finances and Climate Change.</p><p> Quick link: <a href="http://data.worldbank.org/country/united-states" target="_blank">http://data.worldbank.org/country/united-states</a></p><p>See all <a href="/data-resources/">data and resources</a> »</p>