• <div class="header-image" style="background-image: url(/live/image/gid/4/2611_Header_V6N2_web_4.rev.1518551584.jpg);">​</div><div class="header-background-color"/>

Evidence-Based Policy: 2002 No Child Left Behind Act

April 17, 2017
In 1983, against the backdrop of accelerating globalization and capitalism throughout the world, the National Commission on Excellence in Education released a landmark report “Nation at Risk”, highlighting the gross inadequacies of the United States’ education system. This sparked a wave of neoliberalism in the education policy sphere and ultimately led to the passage of the No Child Left Behind (NCLB) Act of 2002. It introduced mandatory standardized testing, stricter state accountability systems, and higher teacher qualification requirements. Today, the debate on the NCLB Act remains as polarized as ever.

President Trump’s Secretary of Education, Betsy DeVos, continues the neoliberal ideology in education policy into the 115th Congress, arguing for more school choice, voucher programs, and government deregulation. This moment is therefore a critical and opportune one for us to reexamine the effectiveness of the NCLB Act, not in terms of ideological arguments and rhetoric, but based on research data and analyses. Based on numerous studies by education policy researchers, it is clear that the NCLB Act, through the use of standardized tests, incentives and sanctions, improved education outcomes for students across the nation. Crucially, this played a significant role in tackling the “rising tide of mediocrity” in the American education system.

Passed with bipartisan support in 2011, the NCLB Act was based on the premise that higher standards, stronger accountability, and greater school choice would improve education outcomes. Specifically, it introduced the following initiatives. First, states must administer standardized reading, math and science examinations at various grade levels, and schools must report the aggregate results for both the student population and specific disadvantaged groups. Second, schools must achieve “adequate yearly progress” (AYP) as defined by the states or face sanctions of increasing severity. Sanctions faced by these schools include being mandated to offer students the option to transfer into a higher-performing school, being required to implement supplemental education services, and even being forced to shut down, among many others. Third, teachers must be “highly qualified”, which largely refers to a bachelor’s degree in their teaching subject and state certification.

America’s Mediocre Education System

Compared to international standards, America’s education system lags behind. In the 2015 PISA assessment, the United States ranked 19th in science, 20th in reading and 31st in mathematics out of 35 OECD countries, decisively lower than average, in spite of having a higher income per capita and spending almost twice the average amount on education per pupil than other nations [1]. In fact, according to a McKinsey study, this persistent gap in academic achievement between children in the United States and their international counterparts deprived the economy of as much as $2.3 trillion in economic output in 2008 [2].

Fortunately, academic achievement in the United States has been steadily improving, in part due to the efforts of education policymakers and legislators, such as the NCLB Act. The standard measure of the achievement levels is the National Assessment of Educational Progress (NAEP). In math, the average fourth-grade scores improved from 235 (out of 500 points) in 2003 to 240 in 2009 and the average eighth-grade scores improved from 278 in 2003 to 283 in 2009 [3]. In reading, the average fourth-grade scores improved from 218 in 2003 to 221 in 2009 and the average eighth-grade scores improved marginally from 263 in 2003 to 264 in 2009 [3].

Nonetheless, these test scores do not reflect the full picture, and the debate for and against the NCLB Act remains as contentious as ever.

Image: Trend in Main NAEP Scores Since 1990<br/>Source: The Brookings InstitutionImage: Trend in Main NAEP Scores Since 1990
Source: The Brookings Institution

Debate over No Child Left Behind

Proponents of the NCLB Act argue that standardized testing holds both teachers and schools accountable, allowing students and parents to compare the performance of these teachers and schools with others. This underscores the Schumpeterian concept of creative destruction, in which organizations with operational slack either have to improve or be replaced by more competent and innovative ones. For example, in 2003, about 30 percent of all public schools did not meet AYP for at least one year, with nearly seven percent not meeting AYP for two or more years [4]. Corresponding sanctions, such as being mandated to offer students the option to transfer into a higher performing school, were imposed on these schools. As a result, almost 70,000 students exercised the transfer option in that year, resulting in not only a better education for these students, but also an increased pressure on these schools to improve [5]. Finally, apart from raising accountability, the NCLB Act also improves teaching standards through standardized testing, which provides teachers the guidance to determine what and when to teach students. This was in stark contrast to the past in which schools determined their own curriculum, irrespective of the latest developments in pedagogy, international education benchmarks, and industry requirements.

On the other hand, detractors of the NCLB Act contend that standardized testing causes teachers to “teach to the test”, stifling their teaching abilities and creativity. Moreover, skills and values, such as creativity, empathy, and resilience, that are not covered in the tests may be neglected. This focus on rote learning and repetitive test taking harms the social and emotional well-being of students. Furthermore, these high-stakes tests are not reflective of solving real-world issues, and some students do not cope well with such rigid forms of assessments. In particular, the excessive use of multiple choice questions in standardized tests hinders an accurate assessment of a student’s understanding the topic at hand. Special education and Limited English Proficient students also take the same standardized exams as the general student population, thereby neglecting their unique learning difficulties. Additionally, in pursuit of achieving AYP goals, states have progressively lowered their assessment standards, resulting in worse education outcomes [6]. Lastly, critics of the NCLB Act argue that since the sanctions are determined based on improvements in the percentage of students who achieve proficiency standards, this has resulted in an “education triage”, in which the lowest and highest performing students are neglected because they do not contribute to the school’s ability to meet AYP. Schools reportedly only focus on “bubble students”, who refer to students below but near the proficiency standards.

Evidentiary Analysis

The vast majority of education policy researchers believe that the effectiveness of the NCLB Act in increasing academic achievement is more nuanced that traditional politicians and activists portray. On balance, the NCLB Act resulted in moderate improvements in academic achievement when compared to what schools would have achieved without such accountability systems in place.

First and foremost, one of the most important findings by policy researchers is that standardized tests are both valid and reliable, accurately predicting numerous life outcomes such as wages, body mass index, number of arrests and life satisfaction [7]. Furthermore, given that these examinations incentivize the learning of essential employment skills in mathematics, science and reading literacy, standardized tests can play an important role in raising workforce productivity. In fact, Hanushek and Woessmann conclude decisively in their study the extremely robust and consistent positive relationship between test scores and growth rates. From 1971 to 2003, using data from 12 OECD countries, the test-score trend of individual countries accounted for 64 percent of the variation in their growth trends [8]. Indeed, skills will play an increasingly important role in the American economy. According to a study by Rothwell from the Brookings Institute, only a third of adults possess a bachelor’s degree or more, despite the fact that almost half of all newly available jobs requires one [9]. In the light of an increasingly knowledge-based economy, coupled with the proliferation of robots, automation and artificial intelligence, the lackluster performance by the United States in raising its numeracy and literacy levels will be increasingly detrimental to its economy, workforce and wages.

Against this backdrop, it is unsurprising that education policymakers have turned to better standardized tests, stronger incentives and harsher penalties to spur state schools to improve. For example, Dee and Jacob found that stricter accountability systems imposed by the NCLB Act improved the math performance of fourth-grade students by almost two-third of a year’s worth of growth, after controlling for external factors like socioeconomic status (2009) [10]. Among these students, the effect was the largest among the bottom percentile and among disadvantaged subgroups like African Americans, Hispanics and low-income families. However, the study showed a statistically insignificant improvement in the math performance of eighth-grade students and no evidence that the NCLB improved fourth-grade reading performance. Separately, Hanushek and Raymond (2005) [11], Springer (2008) [12], Wong, Cook, and Steiner (2009) [13], and Neal and Schanzenbach (2010) [14] corroborate the general findings that the NCLB Act improved academic achievement among students, although to varying degrees. Overall, this seems to suggest that the NCLB Act was most effective at raising basic skills easily tested by standardized exams, such as a fourth-grade math exam, and in particular among the most disadvantaged subgroups.

Despite these positive effects, academic policy researchers generally conclude that “triage” effects do exist as a result of the AYP system. For example, Neal and Schanzenbach showed that the variation in proficiency standards for different grades between the Chicago Public School accountability system and that of the NCLB Act resulted in no change in academic performance for students who did not have a realistic chance of becoming proficient in the near term and improvements in students near the proficiency standard (2010) [14]. These results are corroborated by Ladd and Lauen (2010) [15] and Krieg (2008) [16].

Moreover, there is a general consensus among researchers that the AYP standards incentivized states to lower their proficiency standards, leading to counterproductive results. For instance, Cronin et. al. present evidence that seven out of 12 states lowered their proficiency standards between 2003 and 2006, with the declines being the largest in states with the highest initial proficiency standards (2005) [17]. Indeed, by 2011, Usher showed that nearly half of all schools in the country were considered failing (2015) [18].

On the other hand, researchers, whilst finding some evidence of “teaching to the test”, generally argue for improvements to standardized tests, rather than abandoning these tests altogether. This is particularly important to academics because standardized testing produces high-quality quantitative results that empower the analysis of education policies. For example, comparing results in 1994 and 1998, Klein et al presented evidence that in math, the improvements among fourth-grade students in the high-stakes Texas Assessment of Academic Skills (TAAS) were greater than that in the lower-stakes NAEP test (2000) [19]. Furthermore, according to the TAAS, the difference between the scores of whites and students of color is small and decreasing greatly, whereas according to the NAEP, it was large and increasing slightly. This hints at some evidence of students being coached to the unique styles of these tests, the worsening of skills outside of what is taught in narrower standardized curriculum and the presence of racial biases in some standardized tests. Despite these criticisms, Klein et al. emphasize that they nonetheless believe that a large-scale assessment is essential to monitoring student progress. This is reiterated by Bambrick-Santoyo’s Driven by Data (2010), along with many other researchers [20].

Image: Inflation-adjusted Federal Spending Per Pupil since 1970<br/>Source: The Cato InstituteImage: Inflation-adjusted Federal Spending Per Pupil since 1970
Source: The Cato Institute

Finally, the NCLB Act has proven to be relatively cost-effective at improving students’ academic achievement levels. For example, by 2008, the policy increased annual spending per pupil by a significant $600 in states that did not have school accountability programs prior to the NCLB Act [21]. This increased expenditure was directed to student instruction and educational support services and represented a reasonable return on investment in terms of its corresponding test score increases. Furthermore, at the federal level, total federal education funding increased by about 30 percent, from $42 billion in 2001 to $56 billion in 2004, thereby providing state schools with significant federal support [22]. With education spending representing one of the most cost-effective means to raise social mobility and economic productivity, the shift in federal funding towards education signifies tremendous progress towards investing in the future of our country.

Conclusion

America’s education system has fallen behind. While the NCLB Act has indeed helped improve education outcomes moderately in a cost-efficient manner, it has also led to notable drawbacks: “triage” effects, artificially low proficiency standards and poorly designed standardized tests. Nonetheless, progress has been made. Through the passage of the Every Student Succeeds Act in 2015, the federal government remedied much of the major weaknesses of the NCLB Act, whilst demonstrating its continual commitment to standardized testing and state accountability systems. As schools across the United States enter into a new era of increased accountability, competition and school choice under the new Trump administration, it will be more important than ever to rely on objective analyses and research to strengthen our education system, our workforce and our country.

References

  [1] http://www.oecd.org/pisa/pisa-2015-United-States.pdf
  [2] http://www.mckinsey.com/industries/social-sector/our-insights/the-economic-cost-of-the-us-education-gap
  [3] https://www.brookings.edu/wp-content/uploads/2016/06/0317_education_loveless.pdf
  [4] https://greatlakescenter.org/docs/Policy_Briefs/GLC_AYP_Mathis_FINAL.pdf
  [5] http://all4ed.org/wp-content/uploads/2004/06/Volume4No10.pdf
  [6] https://cdn.americanprogress.org/wp-content/uploads/kf/STANDARDS-BASED_FRAMEWORK.PDF
  [7] http://www.pnas.org/content/113/47/13354.full#sec-4
  [8] http://hanushek.stanford.edu/sites/default/files/publications/Hanushek+Woessmann 2012 JEconGrowth 17(4).pdf
  [9] https://www.brookings.edu/research/education-job-openings-and-unemployment-in-metropolitan-america/
  [10] http://www.jstor.org/stable/pdf/40802083.pdf
  [11] http://hanushek.stanford.edu/sites/default/files/publications/hanushek+raymond.2005 jpam 24-2.pdf
  [12] https://my.vanderbilt.edu/matthewspringer/files/2015/02/Springer-2008.pdf
  [13] http://www.ipr.northwestern.edu/publications/docs/workingpapers/2009/IPR-WP-09-11.pdf
  [14] http://www.mitpressjournals.org/doi/pdf/10.1162/rest.2010.12318
  [15] http://www.jstor.org/stable/pdf/40802083.pdf
  [16] http://www.mitpressjournals.org/doi/pdf/10.1162/edfp.2008.3.2.250
  [17] http://files.eric.ed.gov/fulltext/ED491250.pdf
  [18] http://www.cep-dc.org/cfcontent_file.cfm?Attachment=Usher_Report_AYP2010-2011_121511.pdf
[19] http://epaa.asu.edu/ojs/article/download/440/56
  [20] https://books.google.com/books/about/Driven_by_Data.html?id=wJ0IS6At4osC
  [21] http://www.jstor.org/stable/pdf/41012846.pdf
  [22] https://www2.ed.gov/about/overview/budget/budget09/summary/edlite-section1.html

Student Blog Disclaimer
  • The views expressed on the Student Blog are the author’s opinions and don’t necessarily represent the Penn Wharton Public Policy Initiative’s strategies, recommendations, or opinions.

PENN WHARTON PPI
RESOURCE SPOTLIGHT:

  • <h3>The Penn World Table</h3><p> The Penn World Table provides purchasing power parity and national income accounts converted to international prices for 189 countries/territories for some or all of the years 1950-2010.</p><p><a href="https://pwt.sas.upenn.edu/php_site/pwt71/pwt71_form.php" target="_blank">Quick link.</a> </p><p>See all <a href="/data-resources/">data and resources</a> »</p>
  • <h3>National Center for Education Statistics</h3><p><strong><img width="400" height="80" alt="" src="/live/image/gid/4/width/400/height/80/479_nces.rev.1407787656.jpg" class="lw_image lw_image479 lw_align_right" data-max-w="400" data-max-h="80"/>The National Center for Education Statistics (NCES) is the primary federal entity for collecting and analyzing data related to education in the U.S. and other nations.</strong> NCES is located within the U.S. Department of Education and the Institute of Education Sciences. NCES has an extensive Statistical Standards Program that consults and advises on methodological and statistical aspects involved in the design, collection, and analysis of data collections in the Center. To learn more about the NCES, <a href="http://nces.ed.gov/about/" target="_blank">click here</a>.</p><p> Quick link to NCES Data Tools: <a href="http://nces.ed.gov/datatools/index.asp?DataToolSectionID=4" target="_blank">http://nces.ed.gov/datatools/index.asp?DataToolSectionID=4</a></p><p> Quick link to Quick Tables and Figures: <a href="http://nces.ed.gov/quicktables/" target="_blank">http://nces.ed.gov/quicktables/</a></p><p> Quick link to NCES Fast Facts (Note: The primary purpose of the Fast Facts website is to provide users with concise information on a range of educational issues, from early childhood to adult learning.): <a href="http://nces.ed.gov/fastfacts/" target="_blank">http://nces.ed.gov/fastfacts/#</a></p><p>See all <a href="/data-resources/">data and resources</a> »</p>
  • <h3>NOAA National Climatic Data Center</h3><p><img width="200" height="198" alt="" src="/live/image/gid/4/width/200/height/198/483_noaa_logo.rev.1407788692.jpg" class="lw_image lw_image483 lw_align_left" srcset="/live/image/scale/2x/gid/4/width/200/height/198/483_noaa_logo.rev.1407788692.jpg 2x, /live/image/scale/3x/gid/4/width/200/height/198/483_noaa_logo.rev.1407788692.jpg 3x" data-max-w="954" data-max-h="945"/>NOAA’s National Climatic Data Center (NCDC) is responsible for preserving, monitoring, assessing, and providing public access to the Nation’s treasure of <strong>climate and historical weather data and information</strong>.</p><p> Quick link to home page: <a href="http://www.ncdc.noaa.gov/" target="_blank">http://www.ncdc.noaa.gov/</a></p><p> Quick link to NCDC’s climate and weather datasets, products, and various web pages and resources: <a href="http://www.ncdc.noaa.gov/data-access/quick-links" target="_blank">http://www.ncdc.noaa.gov/data-access/quick-links</a></p><p> Quick link to Text & Map Search: <a href="http://www.ncdc.noaa.gov/cdo-web/" target="_blank">http://www.ncdc.noaa.gov/cdo-web/</a></p><p>See all <a href="/data-resources/">data and resources</a> »</p>
  • <h3>Internal Revenue Service: Tax Statistics</h3><p><img width="155" height="200" alt="" src="/live/image/gid/4/width/155/height/200/486_irs_logo.rev.1407789424.jpg" class="lw_image lw_image486 lw_align_left" srcset="/live/image/scale/2x/gid/4/width/155/height/200/486_irs_logo.rev.1407789424.jpg 2x" data-max-w="463" data-max-h="596"/>Find statistics on business tax, individual tax, charitable and exempt organizations, IRS operations and budget, and income (SOI), as well as statistics by form, products, publications, papers, and other IRS data.</p><p> Quick link to <strong>Tax Statistics, where you will find a wide range of tables, articles, and data</strong> that describe and measure elements of the U.S. tax system: <a href="http://www.irs.gov/uac/Tax-Stats-2" target="_blank">http://www.irs.gov/uac/Tax-Stats-2</a></p><p>See all <a href="/data-resources/">data and resources</a> »</p>
  • <h3>The World Bank Data (U.S.)</h3><p><img width="130" height="118" alt="" src="/live/image/gid/4/width/130/height/118/484_world-bank-logo.rev.1407788945.jpg" class="lw_image lw_image484 lw_align_left" srcset="/live/image/scale/2x/gid/4/width/130/height/118/484_world-bank-logo.rev.1407788945.jpg 2x, /live/image/scale/3x/gid/4/width/130/height/118/484_world-bank-logo.rev.1407788945.jpg 3x" data-max-w="1406" data-max-h="1275"/>The <strong>World Bank</strong> provides World Development Indicators, Surveys, and data on Finances and Climate Change.</p><p> Quick link: <a href="http://data.worldbank.org/country/united-states" target="_blank">http://data.worldbank.org/country/united-states</a></p><p>See all <a href="/data-resources/">data and resources</a> »</p>
  • <h3>USDA Nutrition Assistance Data</h3><p><img width="180" height="124" alt="" src="/live/image/gid/4/width/180/height/124/485_usda_logo.rev.1407789238.jpg" class="lw_image lw_image485 lw_align_right" srcset="/live/image/scale/2x/gid/4/width/180/height/124/485_usda_logo.rev.1407789238.jpg 2x, /live/image/scale/3x/gid/4/width/180/height/124/485_usda_logo.rev.1407789238.jpg 3x" data-max-w="1233" data-max-h="850"/>Data and research regarding the following <strong>USDA Nutrition Assistance</strong> programs are available through this site:</p><ul><li>Supplemental Nutrition Assistance Program (SNAP) </li><li>Food Distribution Programs </li><li>School Meals </li><li>Women, Infants and Children </li></ul><p> Quick link: <a href="http://www.fns.usda.gov/data-and-statistics" target="_blank">http://www.fns.usda.gov/data-and-statistics</a></p><p>See all <a href="/data-resources/">data and resources</a> »</p>
  • <h3>Federal Reserve Economic Data (FRED®)</h3><p><strong><img width="180" height="79" alt="" src="/live/image/gid/4/width/180/height/79/481_fred-logo.rev.1407788243.jpg" class="lw_image lw_image481 lw_align_right" data-max-w="222" data-max-h="97"/>An online database consisting of more than 72,000 economic data time series from 54 national, international, public, and private sources.</strong> FRED®, created and maintained by Research Department at the Federal Reserve Bank of St. Louis, goes far beyond simply providing data: It combines data with a powerful mix of tools that help the user understand, interact with, display, and disseminate the data.</p><p> Quick link to data page: <a href="http://research.stlouisfed.org/fred2/tags/series" target="_blank">http://research.stlouisfed.org/fred2/tags/series</a></p><p>See all <a href="/data-resources/">data and resources</a> »</p>
  • <h3>Congressional Budget Office</h3><p><img width="180" height="180" alt="" src="/live/image/gid/4/width/180/height/180/380_cbo-logo.rev.1406822035.jpg" class="lw_image lw_image380 lw_align_right" data-max-w="180" data-max-h="180"/>Since its founding in 1974, the Congressional Budget Office (CBO) has produced independent analyses of budgetary and economic issues to support the Congressional budget process.</p><p> The agency is strictly nonpartisan and conducts objective, impartial analysis, which is evident in each of the dozens of reports and hundreds of cost estimates that its economists and policy analysts produce each year. CBO does not make policy recommendations, and each report and cost estimate discloses the agency’s assumptions and methodologies. <strong>CBO provides budgetary and economic information in a variety of ways and at various points in the legislative process.</strong> Products include baseline budget projections and economic forecasts, analysis of the President’s budget, cost estimates, analysis of federal mandates, working papers, and more.</p><p> Quick link to Products page: <a href="http://www.cbo.gov/about/our-products" target="_blank">http://www.cbo.gov/about/our-products</a></p><p> Quick link to Topics: <a href="http://www.cbo.gov/topics" target="_blank">http://www.cbo.gov/topics</a></p><p>See all <a href="/data-resources/">data and resources</a> »</p>
  • <h3>National Bureau of Economic Research (Public Use Data Archive)</h3><p><img width="180" height="43" alt="" src="/live/image/gid/4/width/180/height/43/478_nber.rev.1407530465.jpg" class="lw_image lw_image478 lw_align_right" data-max-w="329" data-max-h="79"/>Founded in 1920, the <strong>National Bureau of Economic Research</strong> is a private, nonprofit, nonpartisan research organization dedicated to promoting a greater understanding of how the economy works. The NBER is committed to undertaking and disseminating unbiased economic research among public policymakers, business professionals, and the academic community.</p><p> Quick Link to <strong>Public Use Data Archive</strong>: <a href="http://www.nber.org/data/" target="_blank">http://www.nber.org/data/</a></p><p>See all <a href="/data-resources/">data and resources</a> »</p>
  • <h3>Federal Aviation Administration: Accident & Incident Data</h3><p><img width="100" height="100" alt="" src="/live/image/gid/4/width/100/height/100/80_faa-logo.rev.1402681347.jpg" class="lw_image lw_image80 lw_align_left" srcset="/live/image/scale/2x/gid/4/width/100/height/100/80_faa-logo.rev.1402681347.jpg 2x, /live/image/scale/3x/gid/4/width/100/height/100/80_faa-logo.rev.1402681347.jpg 3x" data-max-w="550" data-max-h="550"/>The NTSB issues an accident report following each investigation. These reports are available online for reports issued since 1996, with older reports coming online soon. The reports listing is sortable by the event date, report date, city, and state.</p><p> Quick link: <a href="http://www.faa.gov/data_research/accident_incident/" target="_blank">http://www.faa.gov/data_research/accident_incident/</a></p><p>See all <a href="/data-resources/">data and resources</a> »</p>
  • <h3>HUD State of the Cities Data Systems</h3><p><strong><img width="200" height="200" alt="" src="/live/image/gid/4/width/200/height/200/482_hud_logo.rev.1407788472.jpg" class="lw_image lw_image482 lw_align_left" srcset="/live/image/scale/2x/gid/4/width/200/height/200/482_hud_logo.rev.1407788472.jpg 2x, /live/image/scale/3x/gid/4/width/200/height/200/482_hud_logo.rev.1407788472.jpg 3x" data-max-w="612" data-max-h="613"/>The SOCDS provides data for individual Metropolitan Areas, Central Cities, and Suburbs.</strong> It is a portal for non-national data made available through a number of outside institutions (e.g. Census, BLS, FBI and others).</p><p> Quick link: <a href="http://www.huduser.org/portal/datasets/socds.html" target="_blank">http://www.huduser.org/portal/datasets/socds.html</a></p><p>See all <a href="/data-resources/">data and resources</a> »</p>
  • <h3>MapStats</h3><p> A feature of FedStats, MapStats allows users to search for <strong>state, county, city, congressional district, or Federal judicial district data</strong> (demographic, economic, and geographic).</p><p> Quick link: <a href="http://www.fedstats.gov/mapstats/" target="_blank">http://www.fedstats.gov/mapstats/</a></p><p>See all <a href="/data-resources/">data and resources</a> »</p>