Bullhorn in hand, University of Oklahoma President David Boren had some words for the Sigma Alpha Epsilon students allegedly caught on video chanting racist slurs last weekend.“I have a message for those who have misused their freedom of speech in this way,” Boren said. “My message to them is: You’re disgraceful. You have violated every principle that this university stands for.”

Violating university principles may be worthy of condemnation, but should “misusing” your free speech be punished with expulsion? Boren did just that Tuesday when he announced that two students who allegedly played a “leadership role” in the SAE chant would be expelled from the university. He said he hopes that “students involved in this incident will learn from this experience and realize that it is wrong to use words to hurt, threaten, and exclude other people,” and promised “appropriate disciplinary action” against other culprits once they're identified.

But as UCLA School of Law professor Eugene Volokh noted shortly before Boren’s announcement, a public university student has a right to express himself without being expelled—even if that expression is a virulent, racist chant. “First, racist speech is constitutionally protected, just as is expression of other contemptible ideas,” Volokh wrote. “And universities may not discipline students based on their speech.”

Public universities, that is. In 1972, the Supreme Court made clear that “state colleges and universities are not enclaves immune from the sweep of the First Amendment,” and a year later the court reaffirmed that “the mere dissemination of ideas—no matter how offensive to good taste—on a state university campus may not be shut off in the name alone of conventions of decency.” That case, Papish v. University of Missouri Curators, involved a set of facts relevant to the Oklahoma case: Expulsion of a graduate journalism student who was sanctioned for handing out a newspaper “containing indecent forms of speech.” Noting that a public university can reasonably regulate “the time, place, and manner of speech and its dissemination,” the court ultimately ruled that a student cannot be “expelled because of the disapproved content of the newspaper,” and ordered the student reinstated.

Of course, there are exceptions to this rule: when the offensive speech represents a “true threat” to the listener, or if the words had somehow been used to pick a fight or to incite others to violence. None of that seems to be the case here, as the chant appears to have been sung while on a school trip and not directed to a particular audience. The chant was shameful, no doubt, but the expelled students have a strong case that the University of Oklahoma violated their free-speech rights.

The school, for its part, seems to have anchored the expulsion on the notion that the students created a “hostile educational environment” for everyone else with their “racist and exclusionary chant.” But where does such hostility and exclusion begin and end? The reason that broad First Amendment protections should trump such undefined and malleable concerns is more pragmatic than anything else. Consider the recent Charlie Hebdo massacre and France’s radical prosecutorial stance in the face of comments it found distasteful. Or New York’s own mini-panic over Facebook emoji after the shooting death of two officers in Brooklyn. Do we punish the offensive few at the expense of greater freedom to express a wider, better range of speech?

In the case of colleges and universities, which the Supreme Court has called the “marketplace of ideas,” the freedom to express a variety of viewpoints holds a special premium. The sheer range of potentially “hostile” or “exclusionary” subjects covered in classrooms and student-run organizations is legion. Take the subject of police brutality, where students could reasonably disagree about whether racism occurs on a case-by-case basis or whether it’s the product of deeper institutional forces and biases. Yes, such debates could get heated, but so it goes with discussions around gender, sexual orientation, discrimination, religion, Middle East relations, war, and other topics over which no single narrative or perspective controls.

Plus, as Volokh noted, it’s very likely the expelled students have already paid dearly for their remarks—with personal embarrassment, the closing of the fraternity chapter, and diminished educational and professional prospects. When you add to that punishment the gauntlet of the state, there’s no telling what other forms of speech suppression will be allowed in the future. So let’s condemn the racist speech. But let’s also condemn any official efforts to penalize it.