Resources

Resources by Type

Research Report

Chin, M., Kane, T., Kozakowski, W., Schueler, B., & Staiger, D. (Working Paper). School District Reform in Newark: Within- and Between- School Changes in Achievement Growth. NBER Working Paper 23922 . Publisher's VersionAbstract
In 2011-12, Newark launched a set of educational reforms supported by $20 million gift. Using data from 2009 through 2016, we evaluate the change in Newark students’ achievement growth relative to similar students and schools elsewhere in New Jersey. We measure achievement growth using a “value-added” model, controlling for prior achievement, demographics and peer characteristics. By the fifth year of reform, Newark saw statistically significant gains in English and no significant change in math achievement growth. Perhaps due to the disruptive nature of the reforms, growth declined initially before rebounding in recent years. Aided by the closure of low value-added schools, much of the improvement was due to shifting enrollment from lower-to higher-growth district and charter schools. Shifting enrollment accounted for 62 percent of the improvement in English. In math, such shifts offset what would have been a decline in achievement growth.
Kelcey, B., McGinn, D., Hill, H. C., & Charalambous, C. (Working Paper). The Generalizability of Item Parameters Across Lessons.Abstract

The purpose of this study is to investigate three aspects of construct validity for the Mathematical Quality of Instruction classroom observation instrument: (1) the dimensionality of scores, (2) the generalizability of these scores across districts, and (3) the predictive validity of these scores in terms of student achievement.

McGinn, D., Kelcey, B., Hill, H., & Chin, M. (Working Paper). Using Item Response Theory to Learn about Observational Instruments.Abstract

As many states are slated to soon use scores derived from classroom observation instruments in high-stakes decisions, developers must cultivate methods for improving the functioning of these instruments. We show how multidimensional, multilevel item response theory models can yield information critical for improving the performance of observational instruments.

Bacher-Hicks, A., Chin, M., Hill, H., & Staiger, D. (Working Paper). Explaining Teacher Effects on Achievement Using Measures from Multiple Research Traditions.Abstract

Researchers have identified many characteristics of teachers and teaching that contribute to student outcomes. However, most studies investigate only a small number of these characteristics, likely underestimating the overall contribution. In this paper, we use a set of 28 teacher-level predictors drawn from multiple research traditions to explain teacher-level variation in student outcomes. These predictors collectively explain 28% of teacher-level variability in state standardized math test scores and 40% in a predictor-aligned math test. In addition, each individual predictor explains only a small, relatively unique portion of the total teacher-level variability. This first finding highlights the importance of choosing predictors and outcomes that are well aligned, and the second suggests that the phenomena underlying teacher effects is multidimensional. 

Blazar, D., Litke, E., Barmore, J., & Gogolen, C. (Working Paper). What Does It Mean to be Ranked a "High" or "Low" Value-Added Teacher? Observing Differences in Instructional Quality Across Districts.Abstract

Education agencies are evaluating teachers using student achievement data. However, very little is known about the comparability of test-based or "value-added" metrics across districts and the extent to which they capture variability in classroom practices. Drawing on data from four urban districts, we find that teachers are categorized differently when compared within versus across districts. In addition, analyses of scores from two observation instruments, as well qualitative viewing of lesson videos identify stark differences in instructional practices across districts among teachers who receive similar within-district value-added rankings. Exploratory analyses suggest that these patterns are not explained by observable background characteristics of teachers and that factors beyond labor market sorting likely play a key role. 

Callen, I., Carbonari, M. V., DeArmond, M., Dewey, D., Dizon-Ross, E., Goldhaber, D., Isaacs, J., et al. (2023). Summer School As a Learning Loss Recovery Strategy After COVID-19: Evidence from Summer 2022.Abstract
To make up for pandemic-related learning losses, many U.S. public school districts have increased enrollment in their summer school programs. We assess summer school as a strategy for COVID-19 learning recovery by tracking the academic progress of students who attended summer school in 2022 across eight districts serving 400,000 students. Based on students’ spring to fall progress, we find a positive impact for summer school on math test achievement (0.03 standard deviation, SD), but not on reading tests. These effects are predominantly driven by students in upper elementary grades. To put the results into perspective, if we assume that these districts have losses similar to those present at the end of the 2022–23 school year (i.e., approximately -0.2 SD), we estimate summer programming closed approximately 2% to 3% of the districts’ total learning losses in math, but none in reading.
Fahle, E., Kane, T. J., Patterson, T., Reardon, S., Staiger, D., & Stuart, E. A. (2023). School District and Community Factors Associated With Learning Loss During the COVID-19 Pandemic.Abstract

We analyze data from approximately 7,800 school districts to describe variation in pandemic-related learning losses among communities and student subgroups. We attempt to understand mechanisms that led to learning losses, as well as explore how historical data from those districts can inform our expectations for how quickly districts will rebound from such losses. We show that learning losses during the pandemic were large and highly variable among communities. Similar to previous research, we find that losses were larger in lower-income and minority districts and in districts which remained remote or hybrid for longer periods during the 2020-21 school year. Among districts, the math learning loss per week of remote/hybrid instruction was larger in high-minority and high-poverty districts. Within districts, however, White students and non-economically disadvantaged students lost about the same amount of ground as Black, Hispanic and economically disadvantaged students. This suggests that the mechanisms driving losses operated at the district or community level, rather than household level. Several community-level characteristics were related to learning losses: broadband access, disruptions to social and economic activity, and trust in government institutions. However, no individual predictor provided strong explanatory power. Relative to historical years, losses during the pandemic were substantial, and an exploratory analysis of historical shocks to achievement suggests that the effects of the pandemic are likely to persist without continued concerted investments in student learning.

Kane, T. J., Doty, E., Patterson, T., & Staiger, D. O. (2022). What Do Changes in State Test Scores Imply for Later Life Outcomes?. Publisher's VersionAbstract
In the three decades before the pandemic, mean achievement of U.S. 8th graders in math rose by more than half a standard deviation on the National Assessment of Educational Progress (NAEP). Between 2019 and 2022, U.S. students had forfeited 40 percent of that rise. To anticipate the consequences of the recent decline, we investigate the past relationship between NAEP scores and students’ later life outcomes by year and state of birth. We find that a standard deviation improvement in a birth cohort’s 8th grade math achievement was associated with an 8 percent rise in income, as well as improved educational attainment and declines in teen motherhood, incarceration and arrest rates. If allowed to become permanent, our findings imply that the recent losses would represent a 1.6 percent decline in present value of lifetime earnings for the average K-12 student (or $19,400), totaling $900 billion for the 48 million students enrolled in public schools during the 2020-21 school year.
  •  
  • 1 of 5
  • »

SDP Partner Diagnostic

Practitioner Resource

Technical Resource

Case Study

Newsletter

Presentation

Kane, T. J. (2016). Askwith Forum – Teaching Higher: Educators’ Perspectives on Common Core Implementation.Abstract

With the debate over the federal role in education at rest with the passage of the Every Student Succeeds Act (ESSA), it is time to refocus attention on how to help the states move forward and succeed using the Common Core State Standards (CCSS). In this Askwith Forum, Professor Thomas Kane will share findings about CCSS implementation strategies from the Center for Education Policy Research at Harvard University. This will be followed by a panel of educators, who will share their experiences, pain points, and successes with the CCSS over this past year.

Learn more about Teaching Higher.

Hill, H. (2014). Lessons Learned from Instruction | Results from a Study of Upper-Elementary Mathematics Classrooms. Beyond the Numbers Convening.Abstract

While research has generated substantial information regarding the characteristics of effective mathematics teachers and classrooms, scholars have rarely tested multiple aspects of teachers or teaching within a single study. Without testing multiple variables simultaneously, it is difficult to identify specific aspects of mathematics teachers and teaching that may be particularly impactful on student learning, and to understand the degree to which these characteristics are related to one another. This plenary draws on data from a three-year study measuring multiple components of teacher and teaching quality to investigate these issues.

Resources By Focus Area

Teacher Effectiveness

Postsecondary Access & Success

School Improvement & Redesign