Publications

2015
(2015). Research Project Overview: The Study of Pre-College Math Remediation Programs in Tennessee . Center for Education Policy Research at Harvard University.Abstract

In an effort to promote college enrollment and degree completion, the state of Tennessee has invested a student-centric, technology-based blended learning model of high school mathematics instruction, The Seamless Alignment and Integrated Learning Support (SAILS).

The SAILS program provides high school seniors likely to require math remediation in college with coursework equivalent to college-level developmental education classes. Eligible students who complete the program are able to satisfy math require-ments for high school graduation and, upon postsecondary matriculation, to enroll directly in credit-bearing coursework toward a college degree.

Researchers at the Center for Education Policy Research at Harvard University and Vanderbilt Peabody College of Education are partnering with the SAILS Program and Tennessee state leadership to conduct an evaluation of SAILS. Using a range of quantitative and qualitative research methods, the study will examine the impact of participation in SAILS on students’ short- and long-term outcomes and investigate the mechanisms by which the program may promote students’ postsecondary success.

sails-research-project-overview.pdf
Greenberg, M. (2015). Best Foot Forward Project: Research Findings from Year 1. Read the report
(2015). Changing the Culture of Data Use in Delaware:How State Leaders Used Analytics to Create Education Policies That Matter . Strategic Data Project.Abstract

This case illustrates how the work of leaders and analysts in the Delaware Department of Education (DDOE) and the agency’s partnership with the Strategic Data Project (SDP), a program of the Center for Education Policy Research at Harvard University, created momentum for statewide policy change.  By exploring Delaware leaders’ use of data and analytics to challenge assumptions and inform the development of better policies and practices, the case illustrates the importance of leadership, analytic and technical competency, and strategic partnerships when leading education reform.  The case specifically highlights the power of human capital analytics to diagnose the current status of Delaware’s educator pipeline, from preparation through development and retention, and how effectively communicating with these analyses built coalitions of support and drove a culture of data use at both the state and district level.
Download the case study [SDP website]

Hill, H. C., Charalambous, C. Y., & Chin, M. (2015). Teacher Characteristics and Student Learning: Toward a More Comprehensive Examination of the Association. Read the working paper Teacher Characteristics Table
Hill, H. C., Chin, M., & Blazar, D. (2015). Teachers' Knowledge of Students: Defining a Domain. Read the working paper
Blazar, D., Braslow, D., Charalambous, C., & Hill, H. C. (2015). Attending to General and Content-Specific Dimensions of Teaching: Exploring Factors Across Two Observation Instruments.Abstract

New observation instruments used in research and evaluation settings assess teachers along multiple domains of teaching practice, both general and content-specific. However, this work infrequently explores the relationship between these domains. In this study, we use exploratory and confirmatory factor analyses of two observation instruments - the Classroom Assessment Scoring System (CLASS) and the Mathematical Quality of Instruction (MQI) - to explore the extent to which we might integrate both general and content-specific view of teaching. Importantly, bi-factor analyses that account for instrument-specific variation enable more robust conclusions than in existing literature. Findings indicate that there is some overlap between instruments, but that the best factor structures include both general and content-specific practices. This suggests new approaches to measuring mathematics instruction for the purposes of evaluation and professional development. 

Read the working paper
(2015). SDP College-Going Diagnostic for the New York State Education Department . Strategic Data Project.Abstract

The Strategic Data Project (SDP) designed the College-Going Diagnostic to inform state and district leaders about high school graduation, college enrollment, and persistence rates; and to identify potential areas for action to increase student achievement in high school, preparedness for college, and postsecondary attainment. In 2012, the New York State Education Department (NYSED) and SDP launched an initiative to assess student attainment in New York State public schools, and identified a set of questions to understand how first-time ninth-grade students in New York State public schools progress through high school and into college. The interactive graphics supporting the findings in this brief are designed to promote deeper engagement with the analysis through data exploration.
View the interactive report on the SDP website

(2015). SDP College-Going Diagnostic for Cleveland Metropolitan School District . Strategic Data Project.Abstract

The Strategic Data Project (SDP) partnered with the Cleveland Metropolitan School District (CMSD) to expand the use of data to inform policy and management decisions within the district. As part of this partnership, SDP collaborated with CMSD to analyze the high school graduation and college-going outcomes of CMSD students.

Download full report
(2015). SDP College-Going Diagnostic for Tennessee . Strategic Data Project.Abstract

SDP launched a College-Going Diagnostic research collaboration with the Tennessee Department of Education (TDOE) as part of a larger partnership between the two organizations. In defining the scope of work for this project, TDOE policymakers were particularly interested in investigating how students’ transition from high school to postsecondary education differed across schools, regions, and student subgroups. The SDP College-Going Diagnostic examines the extent to which Tennessee high school students faced specific barriers to postsecondary enrollment, such as inadequate academic preparation for college-level coursework and limited college access.

Download full report
(2015). SDP Educator Diagnostic for Delaware Department of Education . Strategic Data Project.Abstract

The Strategic Data Project (SDP) collaborated with the state of Delaware to illuminate patterns related to three critical areas of policy focus for the state: the recruitment, placement, and success of new and early career teachers; teacher impact on student learning; and teacher retention and the stability of the state’s teacher workforce.

Download full report
(2015). SDP Human Capital Diagnostic for Colorado . Strategic Data Project.Abstract

The Strategic Data Project (SDP) collaborated with the Colorado Department of Education (CDE) and the Colorado Education Initiative (CEI) to conduct SDP’s Human Capital Diagnostic—a series of high leverage, policy-relevant analyses related to the state’s educator workforce. SDP’s Human Capital Diagnostic investigates questions on five critical topics related to teachers and teacher effectiveness: recruitment, placement, development, evaluation, and retention.

Download full report
(2015). SDP Key Findings Report for Colorado: Mathematics Teacher Placement Patterns . Strategic Data Project.Abstract

The Strategic Data Project (SDP) partnered with the Colorado Department of Education (CDE) and the Colorado Education Initiative (CEI) to investigate whether Colorado public school students who are academically behind their peers are disproportionately placed with novice teachers.

Download full report
(2015). SDP Key Findings Report for Colorado: Students' Progression from Ninth Grade to College, by Prior Academic Achievement . Strategic Data Project.Abstract

The Strategic Data Project (SDP) partnered with the Colorado Department of Education(CDE) and the Colorado Education Initiative (CEI) to investigate statewide trends in students’ high school graduation and their enrollment and persistence in college. This brief summarizes several of the key findings from this research collaboration.

Download full report
2014
(2014). SDP Strategic Use of Data Rubric . Cambridge, MA, Strategic Data Project.Abstract

The SDP Strategic Use of Data Rubric is a resource to provide direction and support to education organizations in their efforts to transform their use of data. The rubric establishes a common language and framework to more clearly illustrate what effective data use at the system level can look like.
Learn more and download the rubric [SDP website]

Kelcey, B., Hill, H. C., & McGinn, D. (2014). Approximate measurement invariance in cross-classified rater-mediated assessments. Frontiers in Psychology , 5 (1469). Publisher's VersionAbstract

An important assumption underlying meaningful comparisons of scores in rater-mediated assessments is that measurement is commensurate across raters. When raters differentially apply the standards established by an instrument, scores from different raters are on fundamentally different scales and no longer preserve a common meaning and basis for comparison. In this study, we developed a method to accommodate measurement noninvariance across raters when measurements are cross-classified within two distinct hierarchical units. We conceptualized random item effects cross-classified graded response models and used random discrimination and threshold effects to test, calibrate, and account for measurement noninvariance among raters. By leveraging empirical estimates of rater-specific deviations in the discrimination and threshold parameters, the proposed method allows us to identify noninvariant items and empirically estimate and directly adjust for this noninvariance within a cross-classified framework. Within the context of teaching evaluations, the results of a case study suggested substantial noninvariance across raters and that establishing an approximately invariant scale through random item effects improves model fit and predictive validity.

Download the full article
Hill, H. C., & Chin, M. (2014). Year-to-Year Stability in Measures of Teachers and Teaching. Read the working paper
(2014). NCTE Newsletter - Fall 2014 [AUDIO] . National Center for Teacher Effectiveness.Abstract

In this newsletter we highlight a new NCTE study, Developing Common Core Classrooms Through Rubric-Based Coaching, and interview one of the Co-Principal Investigators, Dr. Matthew Kraft. 

This newsletter is part of a series originally intended for NCTE study participants. 

Listen to the full interview here:

NCTE Newsletter Fall 2014 Read the interview transcript
(2014). SDP College-Going Diagnostic for Albuquerque Public Schools . Strategic Data Project.Abstract

This report examines Albuquerque Public Schools (APS) students’ high school performance, college enrollment, and college persistence patterns, and compares these patterns across a variety of student characteristics and academic experiences. To conduct the analyses, researchers connected APS administrative student data (including demographics and test scores) to college enrollment records and to student surveys conducted by SDP Data Fellows at several APS high schools. These data sources allowed the diagnostic to track students’ progress through high school to graduation, and to examine their college-going aspirations and actual college outcomes.

Download full report
(2014). SDP Course-Grades Diagnostic for Denver Public Schools . Strategic Data Project.Abstract

In the fall of the 2012–13 school year, the Denver Public Schools and SDP engaged in a research collaboration aimed at providing the district with preliminary evidence that could inform some of these questions, potentially help refine existing reform efforts, and motivate support for future initiatives, such as establishing consistent grading policies rooted in new Common Core standards. This report presents findings from our research diagnostic, illuminating the extent to which average GPAs, standardized assessment scores, and college remediation rates vary across the district and within DPS high schools; how GPAs differ for students from different racial/ethnic backgrounds; and whether knowing a student’s middle school helps predicts her subsequent success.

Download full report
(2014). SDP Human Capital Diagnostic for Albuquerque Public Schools . Strategic Data Project.Abstract

The Human Capital Diagnostic is the result of a partnership between SDP and APS designed to bring data to bear on policy and management decisions to improve student outcomes. Researchers connected student demographics and test scores to teacher human resource data and calculated objective measures of teacher effectiveness that are linked to teacher characteristics. The diagnostic analyses leverage these measures of effectiveness to explore their relationship with characteristics of teachers, schools, and students. They are not intended to draw conclusions about the contribution made by any individual teacher but rather to understand overall district trends.

sdp-diagnostic-hc-albuquerque.pdf

Pages