Resources

The effect of evaluation on employee performance is traditionally studied in the context of the principal-agent problem. Evaluation can, however, also be characterized as an investment in the evaluated employee’s human capital. We study a sample of mid-career public school teachers where we can consider these two types of evaluation effect separately. Employee evaluation is a particularly salient topic in public schools where teacher effectiveness varies substantially and where teacher evaluation itself is increasingly a focus of public policy proposals. We find evidence that a quality classroom-observation-based evaluation and performance measures can improve mid-career teacher performance both during the period of evaluation, consistent with the traditional predictions; and in subsequent years, consistent with human capital investment. However the estimated improvements during evaluation are less precise. Additionally, the effects sizes represent a substantial gain in welfare given the program’s costs.

The authors administered an in-depth survey to new math teachers in New York City and collected information on a number of non-traditional predictors of effectiveness: teaching specific content knowledge, cognitive ability, personality traits, feelings of self-efficacy, and scores on a commercially available teacher selection instrument. They find that a number of these predictors have statistically and economically significant relationships with student and teacher outcomes. The authors conclude that, while there may be no single factor that can predict success in teaching, using a broad set of measures can help schools improve the quality of their teachers.

Researchers from the Harvard Graduate School of Education, MIT, and the University of Michigan have released the results of a new study that suggests that urban charter schools in Massachusetts have large positive effects on student achievement at both the middle and high school levels. Results for nonurban charter schools were less clear; some analyses indicated positive effects on student achievement at the high school level, while results for middle school students were much less encouraging.

View the Press Release

View the PowerPoint Presentation

Whether using the randomized lotteries or statistical controls for measured background characteristics, we generally find large positive effects for Charter Schools, at both the middle school and high school levels. For each year of attendance in middle school, we estimate that Charter Schools raise student achievement .09 to .17 standard deviations in English Language Arts and .18 to .54 standard deviations in math relative to those attending traditional schools in the Boston Public Schools. The estimated impact on math achievement for Charter middle schools is extraordinarily large. Increasing performance by .5 standard deviations is the same as moving from the 50th to the 69th percentile in student performance. This is roughly half the size of the blackwhite achievement gap. In high school, the estimated gains are somewhat smaller than in middle school: .16 to .19 standard deviations in English Language Arts; .16 to .19 in mathematics; .2 to .28 in writing topic development; and .13 to .17 in writing composition with the lottery-based results. The estimated impacts of middle schools and high school Charters are similar in both the “observational” and “lottery-based” results.

The authors used a random-assignment experiment in Los Angeles Unified School District to evaluate various non-experimental methods for estimating teacher effects on student test scores. Having estimated teacher effects during a pre-experimental period, they used these estimates to predict student achievement following random assignment of teachers to classrooms. While all of the teacher effect estimates considered were significant predictors of student achievement under random assignment, those that controlled for prior student test scores yielded unbiased predictions and those that further controlled for mean classroom characteristics yielded the best prediction accuracy. In both the experimental and non-experimental data, the authors found that teacher effects faded out by roughly 50 percent per year in the two years following teacher assignment.

The National Board for Professional Teaching Standards (NBPTS) assesses teaching practice based on videos and essays submitted by teachers. For this study, the authors compared the performance of classrooms of elementary students in Los Angeles randomly assigned to NBPTS applicants and to comparison teachers. The authors conclude that students assigned to highly-rated applicants outperformed those in the comparison classrooms by more than those assigned to poorly-rated teachers. Moreover, the estimates with and without random assignment were similar.

In this newsletter we highlight a new NCTE study, Developing Common Core Classrooms Through Rubric-Based Coaching, and interview one of the Co-Principal Investigators, Dr. Matthew Kraft. 

This newsletter is part of a series originally intended for NCTE study participants. 

Listen to the full interview here: