Publications

Working Paper
Chin, M., & Goldhaber, D. (Working Paper). Exploring Explanations for the "Weak" Relationship Between Value Added and Observation-Based Measures of Teacher Performance. Read the working paper
Bacher-Hicks, A., Chin, M., Hill, H., & Staiger, D. (Working Paper). Explaining Teacher Effects on Achievement Using Measures from Multiple Research Traditions.Abstract

Researchers have identified many characteristics of teachers and teaching that contribute to student outcomes. However, most studies investigate only a small number of these characteristics, likely underestimating the overall contribution. In this paper, we use a set of 28 teacher-level predictors drawn from multiple research traditions to explain teacher-level variation in student outcomes. These predictors collectively explain 28% of teacher-level variability in state standardized math test scores and 40% in a predictor-aligned math test. In addition, each individual predictor explains only a small, relatively unique portion of the total teacher-level variability. This first finding highlights the importance of choosing predictors and outcomes that are well aligned, and the second suggests that the phenomena underlying teacher effects is multidimensional. 

Read the working paper
Kelcey, B., McGinn, D., Hill, H. C., & Charalambous, C. (Working Paper). The Generalizability of Item Parameters Across Lessons.Abstract

The purpose of this study is to investigate three aspects of construct validity for the Mathematical Quality of Instruction classroom observation instrument: (1) the dimensionality of scores, (2) the generalizability of these scores across districts, and (3) the predictive validity of these scores in terms of student achievement.

Read the paper summary
McGinn, D., Kelcey, B., Hill, H., & Chin, M. (Working Paper). Using Item Response Theory to Learn about Observational Instruments.Abstract

As many states are slated to soon use scores derived from classroom observation instruments in high-stakes decisions, developers must cultivate methods for improving the functioning of these instruments. We show how multidimensional, multilevel item response theory models can yield information critical for improving the performance of observational instruments.

Read the paper summary
Blazar, D., Litke, E., Barmore, J., & Gogolen, C. (Working Paper). What Does It Mean to be Ranked a "High" or "Low" Value-Added Teacher? Observing Differences in Instructional Quality Across Districts.Abstract

Education agencies are evaluating teachers using student achievement data. However, very little is known about the comparability of test-based or "value-added" metrics across districts and the extent to which they capture variability in classroom practices. Drawing on data from four urban districts, we find that teachers are categorized differently when compared within versus across districts. In addition, analyses of scores from two observation instruments, as well qualitative viewing of lesson videos identify stark differences in instructional practices across districts among teachers who receive similar within-district value-added rankings. Exploratory analyses suggest that these patterns are not explained by observable background characteristics of teachers and that factors beyond labor market sorting likely play a key role. 

Read the working paper
2016
Hill, H. C., Kraft, M. A., & Herlihy, C. (2016). Developing Common Core Classrooms Through Rubric-Based Coaching . Center for Education Policy Research at Harvard University.Abstract

The project team is still awaiting student test data to complete the evaluation, but this brief provides a short update on survey results. Students of MQI-coached teachers report that their teachers ask more substantive questions, and require more use of mathematical vocabulary as compared to students of control teachers. Students in MQI-coached classrooms also reported more student talk in class. Teachers who received MQI Coaching tended to find their professional development significantly more useful than control teachers, and were also more likely to report that their mathematics instruction improved over the course of the year.

Read the early research findings report.
(2016). The Track Record of Charter Schools in Massachusetts.Abstract

Against the backdrop of a contentious ballot question, charter schools in Massachusetts have faced scrutiny across multiple dimensions. This event brings together several of the preeminent researchers on the topic to share their findings, followed by a period of directed questions, and audience Q&A.

 

Macke Raymond Presentation Thomas Kane Presentation Joshua Angrist Presentation Paul Schlichtman Presentation
Kane, T. J. (2016). Let the Numbers Have Their Say: Evidence on Massachusetts' Charter Schools . Center for Education Policy Research at Harvard University.Abstract

In Massachusetts, the charter school debate has centered on four concerns:

  • that the achievement of the high-scoring charter schools is due to selective admission and retention policies and not the education that the charter schools provide,
  • that charter schools are underserving English language learners and special education students,
  • that charter schools are disciplining students at higher rates in order to drive troublesome students back to traditional schools, and
  • that charter schools are undermining traditional public schools financially.

This report summarizes the evidence pertaining to these four concerns.

Read the report
(2016). DreamBox Learning Achievement Growth in the Howard County Public School System and Rocketship Education . Center for Education Policy at Harvard University. Read the key findings report Read the technical appendix
(2016). Findings from a National Study on Research Use Among School and District Leaders (Technical Report No. 1) . National Center for Research in Policy and Practice. Read the report (NCRPP Website)
West, M. R., Morton, B. A., & Herlihy, C. M. (2016). Achievement Network’s Investing in Innovation Expansion: Impacts on Educator Practice and Student Achievement.Abstract

Achievement Network (ANet) was founded in 2005 as a school-level intervention to support the use of academic content standards and assessments to improve teaching and learning. Initially developed within the Boston charter school sector, it has expanded to serve over 500 schools in nine geographic networks across the United States. The program is based on the belief that if teachers are provided with timely data on student performance from interim assessments tied to state standards, if school leaders provide support and create structures that help them use that data to identify student weaknesses, and if teachers have knowledge of how to improve the performance of students who are falling behind, then they will become more effective at identifying and addressing gaps in student learning. This will, in turn, improve student performance, particularly for high-need students.

In 2010, ANet received a development grant from the U.S. Department of Education’s Investing in Innovation (i3) Program. The grant funded both the expansion of the program to serve up to 60 additional schools in five school districts, as well as an external evaluation of the expansion. The Center for Education Policy Research (CEPR) at Harvard University partnered with ANet to design a matched-pair, school-randomized evaluation of their program’s impact on educator practice and student achievement in schools participating in its i3-funded expansion.

Read the report
Kane, T. J. (2016). Askwith Forum – Teaching Higher: Educators’ Perspectives on Common Core Implementation.Abstract

With the debate over the federal role in education at rest with the passage of the Every Student Succeeds Act (ESSA), it is time to refocus attention on how to help the states move forward and succeed using the Common Core State Standards (CCSS). In this Askwith Forum, Professor Thomas Kane will share findings about CCSS implementation strategies from the Center for Education Policy Research at Harvard University. This will be followed by a panel of educators, who will share their experiences, pain points, and successes with the CCSS over this past year.

Learn more about Teaching Higher.

Kane, T. J., Owens, A. M., Marinell, W. H., Thal, D. R. C., & Staiger, D. O. (2016). Teaching Higher: Educators' Perspectives on Common Core Implementation. Read the report Read the report abstract
Hurwitz, M., Mbekeani, P. P., Nipson, M., & Page, L. C. (2016). Surprising Ripple Effects: How Changing the SAT Score-Sending Policy for Low-Income Students Impacts College Access and Success. Education Evaluation and Policy Analysis. Publisher's VersionAbstract

Subtle policy adjustments can induce relatively large “ripple effects.” We evaluate a College Board initiative that increased the number of free SAT score reports available to low-income students and changed the time horizon for using these score reports. Using a difference-in-differences analytic strategy, we estimate that targeted students were roughly 10 percentage points more likely to send eight or more reports. The policy improved on-time college attendance and 6-year bachelor’s completion by about 2 percentage points. Impacts were realized primarily by students who were competitive candidates for 4-year college admission. The bachelor’s completion impacts are larger than would be expected based on the number of students driven by the policy change to enroll in college and to shift into more selective colleges. The unexplained portion of the completion effects may result from improvements in nonacademic fit between students and the postsecondary institutions in which they enroll.

2015
(2015). SDP College-Going Diagnostic for Wake County Public School System . Strategic Data Project.Abstract

In 2011, the Strategic Data Project (SDP) began a partnership with the Wake County Public School System (WCPSS). As part of this partnership, SDP collaborated with WCPSS to analyze patterns of high school students’ on-track status, graduation, college enrollment, and college persistence. This set of high-leverage, policy-relevant analyses constitutes the SDP College-Going Diagnostic.

Read the report
Kane, T. J., Gehlbach, H., Greenberg, M., Quinn, D., & Thal, D. (2015). The Best Foot Forward Project: Substituting Teacher-Collected Video for In-Person Classroom Observations. Read the report
(2015). Best Foot Forward: Video Observation Toolkit. Download the TookitAbstract

This toolkit provides practical guidance for education practitioners on using video observations to help teachers accelerate their development. Inside you will find four sections to help you start video observations in your school community. Each section includes a discussion of important lessons from the Best Foot Forward project, a study of digital video in classroom observations, and adaptable tools for implementation.

Quinn, D. M., Kane, T. J., Greenberg, M., & Thal, D. (2015). Effects of a Video-Based Teacher Observation Program on the De-privatization of Instruction: Evidence from a Randomized Experiment. l2a_de-privatization-of-instruction.pdf
(2015). SDP College-Going Diagnostic for Kentucky . Strategic Data Project, Center for Education Policy Research at Harvard University. Publisher's VersionAbstract

The Strategic Data Project (SDP) and the Kentucky Department of Education (KDE) collaborated on the SDP College-Going Diagnostic—a set of policy-relevant analyses that track Kentucky public school students’ graduation from high school through enrollment and persistence in college. This interactive report highlights the key findings from this research collaboration and is designed to facilitate exploration across important student characteristics.

Blazar, D. (2015). Effective teaching in elementary mathematics: Identifying classroom practices that support student achievement. Economics of Education Review , 48, 16-29. Publisher's VersionAbstract

Recent investigations into the education production function have moved beyond traditional teacher inputs, such as education, certification, and salary, focusing instead on observational measures of teaching practice. However, challenges to identification mean that this work has yet to coalesce around specific instructional dimensions that increase student achievement. I build on this discussion by exploiting within-school, between-grade, and cross-cohort variation in scores from two observation instruments; further, I condition on a uniquely rich set of teacher characteristics, practices, and skills. Findings indicate that inquiry-oriented instruction positively predicts student achievement. Content errors and imprecisions are negatively related, though these estimates are sensitive to the set of covariates included in the model. Two other dimensions of instruction, classroom emotional support and classroom organization, are not related to this outcome. Findings can inform recruitment and development efforts aimed at improving the quality of the teacher workforce. 

blazar_2015_effective_teaching_in_elementary_mathematics_eer.pdf

Pages