Publications

Working Paper
Bacher-Hicks, A., Chin, M., Hill, H., & Staiger, D. (Working Paper). Explaining Teacher Effects on Achievement Using Measures from Multiple Research Traditions.Abstract

Researchers have identified many characteristics of teachers and teaching that contribute to student outcomes. However, most studies investigate only a small number of these characteristics, likely underestimating the overall contribution. In this paper, we use a set of 28 teacher-level predictors drawn from multiple research traditions to explain teacher-level variation in student outcomes. These predictors collectively explain 28% of teacher-level variability in state standardized math test scores and 40% in a predictor-aligned math test. In addition, each individual predictor explains only a small, relatively unique portion of the total teacher-level variability. This first finding highlights the importance of choosing predictors and outcomes that are well aligned, and the second suggests that the phenomena underlying teacher effects is multidimensional. 

Download the working paper
Kelcey, B., McGinn, D., Hill, H. C., & Charalambous, C. (Working Paper). The Generalizability of Item Parameters Across Lessons.Abstract

The purpose of this study is to investigate three aspects of construct validity for the Mathematical Quality of Instruction classroom observation instrument: (1) the dimensionality of scores, (2) the generalizability of these scores across districts, and (3) the predictive validity of these scores in terms of student achievement.

Read the paper summary
McGinn, D., Kelcey, B., Hill, H., & Chin, M. (Working Paper). Using Item Response Theory to Learn about Observational Instruments.Abstract

As many states are slated to soon use scores derived from classroom observation instruments in high-stakes decisions, developers must cultivate methods for improving the functioning of these instruments. We show how multidimensional, multilevel item response theory models can yield information critical for improving the performance of observational instruments.

Read the paper summary
Blazar, D., Litke, E., Barmore, J., & Gogolen, C. (Working Paper). What Does It Mean to be Ranked a "High" or "Low" Value-Added Teacher? Observing Differences in Instructional Quality Across Districts.Abstract

Education agencies are evaluating teachers using student achievement data. However, very little is known about the comparability of test-based or "value-added" metrics across districts and the extent to which they capture variability in classroom practices. Drawing on data from four urban districts, we find that teachers are categorized differently when compared within versus across districts. In addition, analyses of scores from two observation instruments, as well qualitative viewing of lesson videos identify stark differences in instructional practices across districts among teachers who receive similar within-district value-added rankings. Exploratory analyses suggest that these patterns are not explained by observable background characteristics of teachers and that factors beyond labor market sorting likely play a key role. 

Read the working paper
Chin, M., & Goldhaber, D. (Working Paper). Exploring Explanations for the "Weak" Relationship Between Value Added and Observation-Based Measures of Teacher Performance. sree2015_simulation_working_paper.pdf
2016
West, M. R., Morton, B. A., & Herlihy, C. M. (2016). Achievement Network’s Investing in Innovation Expansion: Impacts on Educator Practice and Student Achievement.Abstract

Achievement Network (ANet) was founded in 2005 as a school-level intervention to support the use of academic content standards and assessments to improve teaching and learning. Initially developed within the Boston charter school sector, it has expanded to serve over 500 schools in nine geographic networks across the United States. The program is based on the belief that if teachers are provided with timely data on student performance from interim assessments tied to state standards, if school leaders provide support and create structures that help them use that data to identify student weaknesses, and if teachers have knowledge of how to improve the performance of students who are falling behind, then they will become more effective at identifying and addressing gaps in student learning. This will, in turn, improve student performance, particularly for high-need students.

In 2010, ANet received a development grant from the U.S. Department of Education’s Investing in Innovation (i3) Program. The grant funded both the expansion of the program to serve up to 60 additional schools in five school districts, as well as an external evaluation of the expansion. The Center for Education Policy Research (CEPR) at Harvard University partnered with ANet to design a matched-pair, school-randomized evaluation of their program’s impact on educator practice and student achievement in schools participating in its i3-funded expansion.

Achievement Network - Full Research Report
Kane, T. J., Owens, A. M., Marinell, W. H., Thal, D. R. C., & Staiger, D. O. (2016). Teaching Higher: Educators' Perspectives on Common Core Implementation. Teaching Higher - Full Report Teaching Higher - Report Abstract
Hurwitz, M., Mbekeani, P. P., Nipson, M., & Page, L. C. (2016). Surprising Ripple Effects: How Changing the SAT Score-Sending Policy for Low-Income Students Impacts College Access and Success. Social Science Research Network. Publisher's VersionAbstract

A growing economics literature reveals that small and subtle policy adjustments can induce relatively large “ripple effects.” We contribute to this literature by evaluating a College Board initiative, launched in the fall of 2007, which increased the number of free official SAT score reports afforded to low-income students and changed the time horizon over which these free score sends could be used. By resetting the default number of free SAT score reports from four to eight for SAT fee-waiver recipients, the College Board hoped to increase the number of college applications submitted by these students and to improve their college match. Using a difference-in-differences analytic strategy, we show that low-income students took advantage of this policy and were roughly 10 percentage points more likely to send eight or more score reports. We find that this policy achieved its intended goal of increasing college access and that it also favorably impacted college completion rates. Specifically, we estimate that inducing a low-income student to send one more score report, on average, increased on-time college attendance by nearly 5 percentage points and five-year bachelor’s completion by slightly more than 3 percentage points. The policy impact was driven entirely by students who, based on SAT scores, were competitive candidates for admission to four-year colleges.

surprising_ripple_effects.pdf
Kane, T. J. (2016). Askwith Forum – Teaching Higher: Educators’ Perspectives on Common Core Implementation.Abstract

With the debate over the federal role in education at rest with the passage of the Every Student Succeeds Act (ESSA), it is time to refocus attention on how to help the states move forward and succeed using the Common Core State Standards (CCSS). In this Askwith Forum, Professor Thomas Kane will share findings about CCSS implementation strategies from the Center for Education Policy Research at Harvard University. This will be followed by a panel of educators, who will share their experiences, pain points, and successes with the CCSS over this past year.

Learn more about Teaching Higher.

2015
Greenberg, M. (2015). Best Foot Forward Project: Research Findings from Year 1. bff-year1-research-findings.pdf
(2015). SDP College-Going Diagnostic for Wake County Public School System . Strategic Data Project.Abstract

In 2011, the Strategic Data Project (SDP) began a partnership with the Wake County Public School System (WCPSS). As part of this partnership, SDP collaborated with WCPSS to analyze patterns of high school students’ on-track status, graduation, college enrollment, and college persistence. This set of high-leverage, policy-relevant analyses constitutes the SDP College-Going Diagnostic.

Download the Report
Kane, T. J., Gehlbach, H., Greenberg, M., Quinn, D., & Thal, D. (2015). The Best Foot Forward Project: Substituting Teacher-Collected Video for In-Person Classroom Observations. l4a_the_best_foot_forward_project.pdf
(2015). Best Foot Forward: Video Observation Toolkit. Download the TookitAbstract

This toolkit provides practical guidance for education practitioners on using video observations to help teachers accelerate their development. Inside you will find four sections to help you start video observations in your school community. Each section includes a discussion of important lessons from the Best Foot Forward project, a study of digital video in classroom observations, and adaptable tools for implementation.

Quinn, D. M., Kane, T. J., Greenberg, M., & Thal, D. (2015). Effects of a Video-Based Teacher Observation Program on the De-privatization of Instruction: Evidence from a Randomized Experiment. l2a_de-privatization-of-instruction.pdf
(2015). SDP College-Going Diagnostic for Kentucky . Strategic Data Project, Center for Education Policy Research at Harvard University. Publisher's VersionAbstract

The Strategic Data Project (SDP) and the Kentucky Department of Education (KDE) collaborated on the SDP College-Going Diagnostic—a set of policy-relevant analyses that track Kentucky public school students’ graduation from high school through enrollment and persistence in college. This interactive report highlights the key findings from this research collaboration and is designed to facilitate exploration across important student characteristics.

(2015). Research Project Overview: The Study of Pre-College Math Remediation Programs in Tennessee . Center for Education Policy Research at Harvard University.Abstract

In an effort to promote college enrollment and degree completion, the state of Tennessee has invested a student-centric, technology-based blended learning model of high school mathematics instruction, The Seamless Alignment and Integrated Learning Support (SAILS).

The SAILS program provides high school seniors likely to require math remediation in college with coursework equivalent to college-level developmental education classes. Eligible students who complete the program are able to satisfy math require-ments for high school graduation and, upon postsecondary matriculation, to enroll directly in credit-bearing coursework toward a college degree.

Researchers at the Center for Education Policy Research at Harvard University and Vanderbilt Peabody College of Education are partnering with the SAILS Program and Tennessee state leadership to conduct an evaluation of SAILS. Using a range of quantitative and qualitative research methods, the study will examine the impact of participation in SAILS on students’ short- and long-term outcomes and investigate the mechanisms by which the program may promote students’ postsecondary success.

sails-research-project-overview.pdf
(2015). Changing the Culture of Data Use in Delaware:How State Leaders Used Analytics to Create Education Policies That Matter . Strategic Data Project.Abstract

This case illustrates how the work of leaders and analysts in the Delaware Department of Education (DDOE) and the agency’s partnership with the Strategic Data Project (SDP), a program of the Center for Education Policy Research at Harvard University, created momentum for statewide policy change.  By exploring Delaware leaders’ use of data and analytics to challenge assumptions and inform the development of better policies and practices, the case illustrates the importance of leadership, analytic and technical competency, and strategic partnerships when leading education reform.  The case specifically highlights the power of human capital analytics to diagnose the current status of Delaware’s educator pipeline, from preparation through development and retention, and how effectively communicating with these analyses built coalitions of support and drove a culture of data use at both the state and district level.
Download the case study [SDP website]

(2015). SDP Educator Diagnostic for Delaware Department of Education . Strategic Data Project.Abstract

The Strategic Data Project (SDP) collaborated with the state of Delaware to illuminate patterns related to three critical areas of policy focus for the state: the recruitment, placement, and success of new and early career teachers; teacher impact on student learning; and teacher retention and the stability of the state’s teacher workforce.

Download full report
(2015). SDP College-Going Diagnostic for Cleveland Metropolitan School District . Strategic Data Project.Abstract

The Strategic Data Project (SDP) partnered with the Cleveland Metropolitan School District (CMSD) to expand the use of data to inform policy and management decisions within the district. As part of this partnership, SDP collaborated with CMSD to analyze the high school graduation and college-going outcomes of CMSD students.

Download full report
(2015). SDP College-Going Diagnostic for Tennessee . Strategic Data Project.Abstract

SDP launched a College-Going Diagnostic research collaboration with the Tennessee Department of Education (TDOE) as part of a larger partnership between the two organizations. In defining the scope of work for this project, TDOE policymakers were particularly interested in investigating how students’ transition from high school to postsecondary education differed across schools, regions, and student subgroups. The SDP College-Going Diagnostic examines the extent to which Tennessee high school students faced specific barriers to postsecondary enrollment, such as inadequate academic preparation for college-level coursework and limited college access.

Download full report