The Philadelphia School District’s decision in 2012 to end suspensions for minor infractions provided fertile ground for the research team of Matthew Steinberg at the University of Pennsylvania and Johanna Lacoe at Mathematica Policy Research. Their findings have proved a Rorschach test, with both sides of the fraught debate on school discipline drawing conclusions from their work. FutureEd chatted with Steinberg about the academic impact on both suspended students and their peers.

Critics and supporters of discipline reform has drawn on your research to make their own arguments. What do you think is the most important lesson we should take from your work?

First, suspending students for non-violent classroom misconduct does not benefit either the suspended students or their peers. Second, we should not expect changes in student behavior simply by removing consequences for student misconduct, such as limiting the use of suspensions. Indeed, district-level policy reforms designed to reduce the use of suspensions should be coupled with intensive school-level supports for schools struggling with the most with student misconduct.

Doing so will help address the underlying behavior that produces suspensions, providing necessary resources for school leaders and teachers to help improve the behavioral climate in these schools.

Did you find evidence linking student suspensions to weaker academic performance?

Our study finds that out-of-school suspensions have a negative effect on student achievement for students suspended for any infraction, including for non-violent or what we call classroom disorder infractions. These included using profanity or general classroom disruption. We also find that the effect on achievement is concentrated in the academic year of the suspension. We find no evidence that a suspension in the prior school year has any adverse consequence for the academic achievement of suspended students in the current year.

How do you measure that impact?

We looked at two outcomes. We looked at achievement, and then we looked at student absences or attendance. What we find specifically is suspensions for any reason are tied to lower scores in math and English language arts tests and that the negative effect increases with each additional day of suspension. The impact varies depending on the age of the student.

What about attendance?

We find some evidence that suspension has a negative effect on school attendance, particularly for younger students, grades 3 through 5. But we’re not talking about many missed days. We also find that students suspended for classroom disorder infractions are significantly more likely to receive a suspension in the subsequent school year.

You also look at how the suspensions impact peers in the classroom.

What we find is that for students in grades where there are more serious misconduct coupled with a higher rate of suspension, student achievement is lower. What we also find is that there is no strong evidence that exposure to low-level, classroom-disorder misconduct has any association with peer achievement. This again suggests that a policy where we suspend, at high rates, kids for low-level misconduct is not defensible.

How is your study different from past research?

The literature goes back 15, 20 years already, where researchers have looked at the association between receiving an out-of-school suspension and student achievement. That research has compared students in the same school who did and did not receive suspensions, and the achievement consequences. What we were able to do was compare students’ achievement to their records in other years when they weren’t as likely to be suspended.

While we find significant adverse effects on student achievement, the magnitude of this effect is substantially smaller than what has been found in prior research and suggests that student-level factors that are typically unobserved by researchers—such as student motivation, ability, etc.—are correlated with both the likelihood a student receives a suspension and a student’s academic achievement.

How can you tell that the suspensions are causing weaker performance versus the weaker performers are acting out and being suspended?

The direction of causality as you’ve identified it is always an important issue that researchers must address. What we’re doing is relying on a comparison of achievement changes within the same students. We’re basically asking, for the same student who received a suspension in one year and not in another year, how did his/her achievement differ?

You could imagine that, on average, students who receive suspensions are much lower-performing (which is what we find in Philadelphia and has been found elsewhere) than students who do not receive suspensions. But the question is, have we accounted for all the factors that may be related to both receiving a suspension and a student’s achievement trajectory so that we can isolate the effect of just receiving the suspension? And we believe we have.

We also then rely on the change in Philadelphia’s district-wide discipline policy reform beginning of the 2012-13 school year.

Let’s imagine a student misbehaved in the 2011-12 school year. That was the year before the reform. Then the student misbehaved in the same way, let’s say, using profanity in the classroom, in the 2012-13 year. The only thing that would change is that in the 12-13 year, the probability that this student would get suspended for profanity should go down.

In Philadelphia, did the discipline reform stop schools from suspending students for classroom disorder infractions?

In the 2011-12 school year, 25 percent of all suspension in Philadelphia were for either classroom disruption or the use of profanity, In the 2012-13 year, after the reform, the percentage of suspensions due to these infractions went down to 15 percent, and then 2013-14, they went down to 12 percent. So while some schools eliminated suspensions for these two forms of low-level, non-violent conduct, many schools in Philadelphia either did not comply or, in some cases, actually increased the use of these suspensions.

Why was that happening?

The schools that were not complying were, as one might expect, the schools that served the most academically and the most economically disadvantaged kids in Philadelphia.

These schools were probably overwhelmed. Just saying you’re not going to suspend these kids doesn’t make the conduct go away. You need to have an alternative. But the district did not provide any support or resources to address the underlying student behavioral issue that were leading to the high rates of suspension.

That seem like an important part of this work.

That is the part of the policy conversation that is typically ignored. Instead of focusing on whether or not we should institute a consequence for student misbehavior, we also need to address the underlying issues. Johanna and I argue that in light of the fact that suspension caused negative consequences for student achievement, we need to also think about what we can do to improve students’ behavior before it reaches the point of schools having to suspend them.

How would you suggest doing that?

One approach that comes to mind is called Positive Behavioral Interventions and Support. It’s typically a school-wide intervention, and there’s experimental evidence that shows that those types of interventions do, in fact, improve student behavior. There are fewer suspensions. And as a result, schools tend to be safer.

Did this reform reduce the racial disparity we see in suspensions in Philadelphia and elsewhere?

Unfortunately, we find some worsening in the racial suspension or discipline gaps following the introduction of Philadelphia’s 2012 reform. The evidence suggests that while there was a modest decline in the use of suspensions for classroom disorder infractions among Black and Hispanic students by the end of the first year of the policy compared to White students, we actually find a commensurate increase in suspension days for more serious offenses.

As a result, on balance, we find a slight increase of about 0.08 days per student, or about 8 additional days of suspension for every 100, for Black students relative to White students.

That’s discouraging. How did that happen?

This evidence suggests the possibility—though, again, we don’t have direct evidence—that some schools may have re-labeled student misconduct among Black and Hispanic students from classroom disorder infractions to more serious infractions so that they could actually use suspensions as a behavioral consequence.

Is the school district aware of this? Are officials there doing anything about it?

We’ve presented this evidence to the district. And the district has initiated a number of interventions at the school level, and a local foundation is working with them on school climate and safety issues.

Ultimately, in Philadephia and elsewhere, the policy conversation should focus both on efforts to reduce the use of exclusionary discipline practices, particularly for the least severe forms of student misconduct, and to couple these efforts with the provision of intensive resources and training at the school level to assist school leaders and teachers to both implement district reforms and to improve the school climate by addressing student misconduct.

— Phyllis Jordan

Phyllis W. Jordan is FutureEd’s editorial director.

This piece originally appeared on the FutureEd website. FutureEd is an independent, solution-oriented think tank at Georgetown’s McCourt School of Public Policy. Follow on Twitter at @futureedGU