Harvard center releases evaluation of Tennessee SAILS math remediation program

October 29, 2018

SAILS eliminated students’ delay in entering college-level courses; did not increase math achievement

More than a third of community college entrants nationally are required to take remedial courses once in college, thereby delaying student progress toward completing a degree. Researchers at Harvard University’s Center for Education Policy Research (CEPR) and Vanderbilt University’s Peabody College have released an evaluation of Tennessee’s innovative solution to remediation’s costly delay: the Seamless Alignment and Integrated Learning Support (SAILS) program, an online remedial math course which students complete during their senior year of high school to avoid having to take math remediation in college. The study finds that although the SAILS program successfully eliminated students’ delay in entering college-level courses, the program did not improve students’ math achievement nor boost their likelihood of passing college math. The program had a positive impact on credit accumulation by the end of the second year in college.

Read the policy brief | Read the research report

Researchers Thomas J. Kane (Harvard), Angela Boatman (Vanderbilt), Whitney Kozakowski (Harvard), Chris Bennett (Vanderbilt), Rachel Hitch (Harvard), and Dana Weisenfeld (Harvard) examined the program’s impact under two distinct policy environments: during 2013–14 when Tennessee required pre-requisite remediation, and during 2014–15 and 2015–16 when the state changed to a co-requisite remediation model, in which students enrolled in remedial math simultaneously with college-level math. To account for other policy changes in Tennessee—such as the introduction of free community tuition in 2014—the researchers took advantage of the staggered timing of the SAILS implementation across the state and compared same-year changes for SAILS-participating and non-participating high schools. They also compared the outcomes of students immediately above and below the ACT Math cutoff score used to identify which students would require remediation.

Among the key findings:

  1. Under the Tennessee’s prerequisite policy, the SAILS program allowed students to enroll in college-level math at higher rates and accumulate additional credits. During the first year in community college, SAILS participants were 29 percentage points more likely to enroll in college math. Roughly half of those students passed the course, amounting to a 13-percentage point increase in the percentage of community college students statewide who passed college math by the end of their first year. However, the impacts on college math enrollment and completion were smaller by the second year, as students from non-participating high schools completed their pre-requisite remediation and caught up. By the end of their second year in college, SAILS participants also completed additional college credits—4.5 credits or 5 courses—compared to their counterparts from non-SAILS high schools.
  2. After the co-requisite policy was introduced, SAILS no longer had an impact on students’ taking or passing college math during their first year of community college, nor on the total number of credits completed at the end of their second year. In lifting the barrier to entry into college-level courses, the state’s co-requisite policy largely supplanted the SAILS program by allowing students to do their remediation alongside college-level courses. Even though SAILS allowed students to avoid taking remedial classes concurrently with their college classes, there is no evidence that SAILS impacted the number of college credits students completed under the co-requisite policy.
  3. The SAILS program did not improve students’ math achievement nor boost their likelihood of passing college math upon taking the course. SAILS participation did not improve performance on a test administered to students immediately above and below the remediation ACT Math cutoff score. About half of the new students who enrolled in college math as a result of SAILS passed the course during their first year in college. The passing rates for SAILS graduates were not higher than for students with similar ACT scores who had not taken SAILS.
  4. SAILS improved students’ perceptions of the usefulness and enjoyment of math. Students just below the ACT Math cutoff score of 19 (out of 36) were 6.5 percentage points more likely than those just above the cutoff score—who did not participate in SAILS—to perceive that their math course content would be useful in their careers, 10 percentage points more likely to indicate they were better prepared for college math, and 6 percentage points more likely to say that they were interested in math than those immediately above the remediation threshold, who took a different, non-SAILS high school math course. The impacts were particularly large for Black students.

“In Tennessee, our results show that remediation is neither the cause nor the solution for low college completion,” Kane says. “In order to move the needle on degree attainment, colleges will need to identify and address other barriers that are hampering student progress.”

About the Authors: Thomas J. Kane is an economist and Walter H. Gale Professor of Education and Economics at the Harvard Graduate School of Education, as well as faculty director of the Center for Education Policy Research (CEPR). Angela Boatman is an assistant professor of public policy and higher education in the Department of Leadership, Policy, and Organizations at Vanderbilt University’s Peabody College. Whitney Kozakowski is a Partnering in Education Research (PIER) Fellow and doctoral student at the Harvard Graduate School of Education. Chris Bennett is a doctoral student in the Department of Leadership, Policy, and Organizations at Vanderbilt University’s Peabody College. Rachel Hitch is the project director overseeing CEPR’s research evaluation projects. Dana Weisenfeld is a research analyst at CEPR.

About the Center for Education Policy Research: The Center for Education Policy Research seeks to transform education through the power of quality research and evidence. CEPR and its partners believe all students will learn and thrive when education leaders make vital decisions using facts and findings, rather than trends and untested assumptions. Learn more at cepr.harvard.edu.

###