Harvard publishes first multi-state study of textbook efficacy in the Common Core era
Cambridge, MA (Friday, March 1, 2019)—In recent years, education leaders have hailed curriculum choice as a low-cost way to improve student success. But in the first multi-state effort to measure textbook efficacy since the implementation of the Common Core, researchers at the Center for Education Policy Research (CEPR) at Harvard University saw no difference in the average fourth- and fifth-grade math achievement gains of schools using different elementary math textbooks. At current levels of curriculum usage and professional development, textbook choice alone does not seem to improve student achievement.
To estimate school-level differences in student achievement growth, the research team—led by Thomas J. Kane (Harvard University) and David Blazar (University of Maryland)—used data on student achievement and math textbook adoptions in fourth- and fifth-grade classrooms in six states (California, Louisiana, Maryland, New Jersey, New Mexico, and Washington) from three school years (2014–15 through 2016–17). They also surveyed teachers on their use of textbooks and supplemental materials, as well as the availability and use of professional development related to math instruction and curriculum. The study sample included almost 6,000 schools and over 1,200 teachers across the six states.
Among the key findings:
● Contrary to prior studies, this analysis finds little evidence of differences in average math achievement growth in schools using different elementary math curricula. Although the researchers saw substantial variation in achievement growth among the schools using each particular curriculum, there were no differences in average student achievement growth between curricula. The findings were similar for specific subgroups of students, such as English language learners, students eligible for free or reduced-price lunch, and students with high- or low-baseline achievement. Nor was the variation in textbook effectiveness any different in the subset of schools in which teachers reported the highest average levels of textbook usage, or in schools that had been using the text for two or more years.
● The vast majority of teachers used their school’s official curriculum in more than half their lessons, but few used it exclusively. Ninety-three percent of teachers reported using the official curriculum in more than half of their lessons for some purpose, such as creating tasks or activities for class, selecting examples to present, or assigning problems for independent practice or homework. Just 25% of teachers use the textbook in nearly all their lessons for all essential activities, including in-class exercises, practice problems, and homework problems. Only 7% of teachers reported that they used their textbook exclusively.
● The elementary math textbook market is concentrated, but market share for specific texts varies by state. Roughly 70% of elementary schools in the six states used one of seven texts, and 90% used one of 15 texts (out of 38 textbooks observed in this study). For instance, in New Mexico, the market was nearly evenly split among three textbook series—enVision, My Math, and Stepping Stones—all of which were written for or adapted to the Common Core. Comparatively, in Louisiana almost 60% of schools used Engage NY/Eureka, an open source curriculum also written for the Common Core.
● Time spent in professional development varied by curriculum, but was generally low. Teachers using Engage NY/Eureka reported receiving the most professional development tailored to the curriculum, but it was still modest: 1.6 days, on average, compared to 0.8 to 1.4 days, on average, for teachers using other textbooks. Moreover, 62% of teachers using Engage NY/Eureka reported working with a math coach, compared to 38% of teachers using other textbooks.
● The research on textbook efficacy is sparse. Of the 38 textbooks observed in the study sample, only five have been evaluated in a manner meeting the highest evidence standards of the federal What Works Clearinghouse, while only three of these are among the top fifteen most commonly used textbooks in our sample.
“Some leaders may see the adoption of a new curriculum as an ‘easy, inexpensive and quick’ alternative to more controversial, expensive, or time-consuming policies such as teacher evaluation or classroom coaching,” says CEPR faculty director and study lead Tom Kane. “It may be a mistake to think of curriculum choice and teaching reforms as alternative ways of improving student outcomes. Rather, to gain the benefits of either, districts may need to do both.”
To arrange an interview with the researchers, please contact Jackie Kerstetter at jacqueline_kerstetter@gse.harvard.edu.
About the Researchers: The research team was composed of experts from a number of universities across the country. Thomas J. Kane is an economist and Walter H. Gale Professor of Education at the Harvard Graduate School of Education, as well as faculty director of CEPR. David Blazar is an assistant professor of education policy and economics at the University of Maryland. Blake Heller is a doctoral student at the Harvard Kennedy School and a Partnering in Education Research (PIER) fellow at the Center for Education Policy Research (CEPR) at Harvard University. Morgan Polikoff is an associate professor of education at the University of Southern California, Rossier School of Education. Douglas Staiger is the John French Professor in Economics at Dartmouth College. Scott Carrell is an associate professor of economics at the University of California, Davis. Dan Goldhaber is the Director of the Center for Analysis of Longitudinal Data in Education Research (CALDER) at the American Institutes for Research (AIR), where he is also a vice president. Douglas Harris is a professor of economics and the Schleider Foundation Chair in Public Education at Tulane University. Michal Kurlaender is a professor of education policy and the Chancellor Fellow at the University of California, Davis. Rachel Hitch was the project director at CEPR during this research. Kristian Holden is a researcher for CALDER at AIR.
About the Center for Education Policy Research at Harvard University: The Center for Education Policy Research at Harvard University seeks to transform education through quality research and evidence. CEPR and its partners believe all students will learn and thrive when education leaders make decisions using evidence, rather than hunches and untested assumptions. Learn more at cepr.harvard.edu.
###