Publications

2014
Kraft, M. A., & Papay, J. P. (2014). Can Professional Environments in Schools Promote Teacher Development? Explaining Heterogeneity in Returns to Teaching Experience. Educational Evaluation and Policy Analysis , 36 (4), 476-500. Publisher's VersionAbstract

Although wide variation in teacher effectiveness is well established, much less is known about differences in teacher improvement over time. We document that average returns to teaching experience mask large variation across individual teachers, and across groups of teachers working in different schools. We examine the role of school context in explaining these differences using a measure of the professional environment constructed from teachers’ responses to state-wide surveys. Our analyses show that teachers working in more supportive professional environments improve their effectiveness more over time than teachers working in less supportive contexts. On average, teachers working in schools at the 75th percentile of professional environment ratings improved 38% more than teachers in schools at the 25th percentile after ten years.

Download full report
Blazar, D., Gogolen, C., Hill, H. C., Humez, A., & Lynch, K. (2014). Predictors of Teachers' Instructional Practices.Abstract

We extend this line of research by investigating teacher career and background characteristics, personal resources, and school and district resources that predict an array of instructional practices identified on a mathematics-specific observational instrument, MQI, and a general instrument, CLASS. To understand these relationships, we use correlation and regression analyses. For a subset of teachers for whom we have data from multiple school years, we exploit within-teacher, cross-year variation to examine the relationship between class composition and instructional quality that is not confounded with the sorting of "better" students to "better" teachers. We conclude that multiple teacher- and school-level characteristics--rather than a single factor--are related to teachers' classroom practices.

Read the working paper
West, M. R., Kraft, M. A., Finn, A. S., Duckworth, A. L., Gabrieli, C. F. O., & Gabrieli, J. D. E. (2014). Promise and Paradox: Measuring Students' Non-cognitive Skills and the Impact of Schooling.Abstract

The authors used self-report surveys to gather information on a broad set of non-cognitive skills from 1,368 eighth-grade students attending Boston Public Schools and linked this information to administrative data on their demographics and test scores. At the student level, scales measuring conscientiousness, self-control, grit, and growth mindset are positively correlated with attendance, behavior, and test-score gains between fourth- and eighth-grade. Conscientiousness, self-control, and grit are unrelated to test-score gains at the school level, however, and students attending over-subscribed charter schools with higher average test-score gains score lower on these scales than do students attending district schools. Exploiting charter school admissions lotteries, the authors replicate previous findings indicating positive impacts of charter school attendance on math achievement, but find negative impacts on these non-cognitive skills. The authors provide suggestive evidence that these paradoxical results are driven by reference bias, or the tendency for survey responses to be influenced by social context. The results therefore highlight the importance of improved measurement of non-cognitive skills in order to capitalize on their promise as a tool to inform education practice and policy.

Download full report
Braslow, D., & Humez, A. (2014). Using Surveys as Proxies for Observations in Measuring Mathematics Instruction.Abstract

Using data from elementary mathematics teachers, we examine the correspondence between self-reports and observational measures of two instructional dimensions--reform-orientation and classroom climate--and the relative ability of these measures to predict teachers' contributions to student learning.

Read the working paper
Hill, H. (2014). Lessons Learned from Instruction | Results from a Study of Upper-Elementary Mathematics Classrooms. Beyond the Numbers Convening.Abstract

While research has generated substantial information regarding the characteristics of effective mathematics teachers and classrooms, scholars have rarely tested multiple aspects of teachers or teaching within a single study. Without testing multiple variables simultaneously, it is difficult to identify specific aspects of mathematics teachers and teaching that may be particularly impactful on student learning, and to understand the degree to which these characteristics are related to one another. This plenary draws on data from a three-year study measuring multiple components of teacher and teaching quality to investigate these issues.

Lessons Learned from Instruction: Summary Lessons Learned from Instruction: PowerPoint
2013
(2013). SDP Toolkit for Effective Data Use . Strategic Data Project. Learn more and download [SDP website]Abstract

The SDP Toolkit for Effective Data Use is a resource guide for education agency analysts who collect and analyze data on student achievement. Completing the toolkit produces a set of basic, yet essential, human capital and college-going analyses that every education agency should have as a foundation to inform strategic management and policy decisions.

Castleman, B. L., Page, L. C., & Snowdon, A. L. (2013). SDP Summer Melt Handbook: A Guide to Investigating and Responding to Summer Melt . Strategic Data Project.Abstract

The SDP Summer Melt Handbook is a resource for education leaders interested in examining whether summer melt is occurring in their agency. The handbook not only serves to diagnose the phenomenon, but also helps leaders understand what they can do to address it.
Learn more about the Summer Melt Handbook on the SDP Website

(2013). SDP College-Going Diagnostic for Boston Public Schools . Strategic Data Project.Abstract

Boston Public Schools collaborated with SDP to produce the SDP College-Going Diagnostic for its district. The diagnostic is designed to identify potential areas for action to increase students’ levels of academic achievement, preparedness for college, and postsecondary attainment. It is also intended to demonstrate how districts can capitalize on existing data to understand its current performance, set future goals, and strategically plan responses.

Download full report
(2013). SDP College-Going Diagnostic for The School District of Philadelphia . Strategic Data Project.Abstract

The School District of Philadelphia partnered with SDP to produce the SDP College-Going Diagnostic. The diagnostic analyses summarized in this report focus on 1) student performance in the district during high school and into college, 2) critical junctures along the way that affect student success, and 3) student characteristics and other factors that are most strongly related to college enrollment and persistence.

Download full report
(2013). SDP College-Readiness Diagnostic for Los Angeles Unified School District . Strategic Data Project.Abstract

Los Angeles Unified School District (LAUSD) partnered with SDP to produce the SDP College-Readiness Diagnostic for its district. The diagnostic analyses focus on 1) how students across the district progress toward high school graduation, 2) whether and how students who fall off track for graduation recover and go on to graduate, and 3) the progress of students toward the completion of A-G requirements.

sdp-lausd-cg.pdf
(2013). SDP Human Capital Diagnostic for Boston Public Schools . Strategic Data Project.Abstract

Boston Public Schools collaborated with SDP to produce the SDP Human Capital Diagnostic for its district. The diagnostic is designed to identify patterns of teacher effectiveness and areas for policy change that could leverage teacher effectiveness to improve student achievement. It is also intended to demonstrate how districts can capitalize on existing data to understand its current performance, set future goals, and strategically plan responses.

Download full report
Hill, H. C., Gogolen, C., Litke, E., Humez, A., Blazar, D., Corey, D., Barmore, J., et al. (2013). Examining High and Low Value-Added Mathematics: Can Expert Observers Tell the Difference? In Association for Public Policy Analysis & Management Fall Research Conference . Washington, DC.Abstract

In this study, we use value-added scores and video data in order to mount an exploratory study of high- and low-VAM teachers' instruction. Specifically, we seek to answer two research questions: First, can expert observers of mathematics instruction distinguish between high- and low-VAM teachers solely by observing their instruction? Second, what instructional practices, if any, consistently characterize high but not low-VAM teacher classrooms? To answer these questions, we use data generated by 250 fourth- and fifth-grade math teachers and their students in four large public school districts.Preliminary analyses indicate that a teacher's value-added rank was often not obvious to this team of expert observers.

Download the conference paper
(2013). Graduate Exit Survey Toolkit . Strategic Data Project.Abstract

This toolkit provides useful resources for designing and rolling out a high school graduate exit survey, as well as effectively analyzing survey results in a school district. Anyone who is interested in implementing a high school exit survey, reworking a current exit survey, or effectively analyzing survey results in a school district can leverage this resource.

Learn more about the [Graduate Exit Survey].

Lynch, K., Chin, M., & Blazar, D. (2013). How Well Do Teacher Observations Predict Value-Added? Exploring Variability Across Districts. In Association for Public Policy Analysis & Management Fall Research Conference . Washington, DC.Abstract

In this study we ask: Do observational instruments predict teachers' value-added equally well across different state tests and district/state contexts? And, to what extent are differences in these correlations a function of the match between the observation instrument and tested content? We use data from the Gates Foundation-funded Measures of Effective Teaching (MET) Project(N=1,333) study of elementary and middle school teachers from six large public school districts,and from a smaller (N=250) study of fourth- and fifth-grade math teachers from four large public school districts. Early results indicate that estimates of the relationship between teachers' value-added scores and their observed classroom instructional quality differ considerably by district.

Download the conference paper
Hill, H. C., & Grossman, P. (2013). Learning from Teacher Observations: Challenges and Opportunities Posed by New Teacher Evaluation Systems. Harvard Educational Review.Abstract

In this article, Heather Hill and Pam Grossman discuss the current focus on using teacher observation instruments as part of new teacher evaluation systems being considered and implemented by states and districts. They argue that if these teacher observation instruments are to achieve the goal of supporting teachers in improving instructional practice, they must be subject-specific, involve content experts in the process of observation, and provide information that is both accurate and useful for teachers. They discuss the instruments themselves, raters and system design, and timing of and feedback from the observations. They conclude by outlining the challenges that policy makers face in designing observation systems that will work to improve instructional practice at scale.

Download full report
Chin, M., Hill, H., McGinn, D., Staiger, D., & Buckley, K. (2013). Using Validity Criteria to Enable Model Selection: An Exploratory Analysis. Association for Public Policy Analysis and Management Fall Research Conference.Abstract

In this paper, the authors propose that an important determinant of value-added model choice should be alignment with alternative indicators of teacher and teaching quality. Such alignment makes sense from a theoretical perspective because better alignment is thought to indicate more valid systems. To provide initial evidence on this issue, they first calculated value-added scores for all fourth and fifth grade teachers within four districts, then extracted scores for 160 intensively studied teachers.Initial analyses using a subset of alternative indicators suggest that alignment between value-added scores and alternative indicators differ by model, though not significantly.

Download working paper
2012
Hickman, J. J., Fu, J., & Hill, H. C. (2012). Technical Report: Creation and Dissemination of Upper-Elementary Mathematics Assessment Modules.Abstract

This document provides information on the grades 4 and 5 mathematics test modules developed as a joint Harvard/ETS venture from 2009 to 2012. Hundreds of items that were initially developed were then culled by a series of reviews, revisions, pilot tests, cognitive interviews, and field tests. 

ncte-assessments-tech-report.pdf
(2012). SDP College-Going Diagnostic for Gwinnett County Public Schools . Strategic Data Project.Abstract

Gwinnett County Public Schools worked with SDP to create the SDP College-Going Diagnostic for its district. The diagnostic is designed to identify potential areas for action to increase students’ levels of academic achievement, preparedness for college, and postsecondary attainment. It is also intended to demonstrate how districts can capitalize on existing data to understand its current performance, set future goals, and strategically plan responses.

Download full report
(2012). SDP Human Capital Diagnostic for Gwinnett County Public Schools . Strategic Data Project.Abstract

Gwinnett County Public Schools worked with SDP to produce the SDP Human Capital Diagnostic for its district. The diagnostic is designed to identify patterns of teacher effectiveness and areas for policy change that could leverage teacher effectiveness to improve student achievement. It is also intended to demonstrate how districts can capitalize on existing data to understand its current performance, set future goals, and strategically plan responses.

Download full report
(2012). SDP Human Capital Diagnostic for Los Angeles Unified School District . Strategic Data Project.Abstract

Los Angeles Unified School District (LAUSD) partnered with SDP to produce the SDP Human Capital Diagnostic for its district. The diagnostic is designed to identify patterns of teacher effectiveness and areas for policy change that could leverage teacher effectiveness to improve student achievement. It is also intended to demonstrate how districts can capitalize on existing data to understand its current performance, set future goals, and strategically plan responses.

Download full report

Pages