Lionel Shriver, an American novelist with a new book out (The Mandibles), opted to make her keynote address at the Brisbane Writers Festival in Australia last week an extended rant against the concept of “cultural appropriation,” and to do so while wearing a sombrero—a reference, in part, to a controversy at Maine’s Bowdoin College where non-Mexican students got in trouble for wearing miniature sombreros at a tequila-themed party. “But what does this have to do with writing fiction?” she asked. “The moral of the sombrero scandals is clear: You’re not supposed to try on other people’s hats.”

Shriver’s speech touched on several other scandals, such as Katy Perry dressing as a geisha and Oberlin protesters objecting to Asian-inspired dining hall food, and concluded that political correctness had indeed gotten out of hand. She dismissed “identity politics” outright: “Being Asian is not an identity. Being gay is not an identity. Being deaf, blind, or wheelchair-bound is not an identity, nor is being economically deprived.” And she defended having created, in The Mandibles, a black female character who’s led around on a leash by white people.

She also made some sensible points about fiction and creativity—which were promptly drowned out by the controversy she had courted.

While the crowd audibly enjoyed the speech, according to The New York Times, festival organizers “censored her on the festival website and publicly disavowed her remarks,” and “scheduled the rebuttal opposite a session Saturday afternoon in which Ms. Shriver was promoting her new novel.” Among the invited opponents: Yassmin Abdel-Magied, a writer who’d walked out during Shriver’s talk. “The stench of privilege hung heavy in the air, and I was reminded of my ‘place’ in the world,” she wrote in The Guardian. Abdel-Magied rightly accuses Shriver of insensitivity, but also sets a restrictive, overly political vision for what literature should be:

It’s not always OK if a white guy writes the story of a Nigerian woman because the actual Nigerian woman can’t get published or reviewed to begin with. It’s not always OK if a straight white woman writes the story of a queer Indigenous man, because when was the last time you heard a queer Indigenous man tell his own story? How is it that said straight white woman will profit from an experience that is not hers, and those with the actual experience never be provided the opportunity?

Cultural appropriation is both a real phenomenon—responsible for rock-n-roll and the Washington Redskins alike—and a ripe target for criticism and mockery, since the concept renders nearly every garment or foodstuff fraught. (I type this while wearing a fast-fashion “peasant” shirt whose design is surely a cultural appropriation, though of which culture I can’t say.) And while the trend Shriver describes—“the left’s embrace of gotcha hypersensitivity”—certainly isn’t in her imagination, it’s also true that the right has embraced knee-jerk insensitivity. (Nor are these left-right divisions so neat when it comes to the P.C. debates.)

So which is it? Is Shriver a refreshing voice of reason, calling out the anti-free-speech excesses of the literary left? Or is she just the latest novelist-provocateur whose attack on political correctness is a thinly veiled defense of her privilege?

In Brisbane, Shriver cited one Armenian-American reader who took issue with a character in her seventh novel. She also cited a San Francisco Chronicle review of Chris Cleave’s Little Bee that asked, “When a white male author writes as a young Nigerian girl, is it an act of empathy, or identity theft?” But these examples hardly suggest a chilling effect worth pontificating about. It’s never clear from Shriver’s speech, or indeed in the entire debate over supposed P.C. silencing, is the scale of the problem—to the extent that it exists at all.

We may never know, as something so nebulous is impossible to quantify. But Abdel-Magied bolstered Shriver’s argument by leaping from a case that there should be more opportunities for writers of color (true) to one that white writers should not create nonwhite characters (which is absurd). By suggesting that a white guy can’t write a story about a Nigerian woman, Abdel-Magied isn’t just condemning offensive representation but any representation—and she’s doing it based solely on the identities of the writer and character. To see the flaw here, consider the TV version of this question: Do we want white show creators to put only white characters onscreen? Given that most show creators are white, that would hardly bring the kind of progress that Abdel-Magied demands. (As I’ve argued in regards to women in literature, change needs to come from an overhaul of the profession itself, not by placing standards on fictional characters.)

I don’t want to exaggerate any individual critic’s influence on what other writers—let alone famous ones—are allowed to write. And in this particular case, it seems likely that the publicity surrounding Shriver’s new book (including, unavoidably, this article) will function as promotion. But there is, if not a silencing effect, a stifling one whenever critics with a platform offer such broad-stroke prescriptions.

In an essay for Literary Hub shortly before the Shriver controversy, Brandon Taylor covers similar ground as Abdel-Magied—both argue against the literature of white obliviousness—but they wind up with opposite arguments. Where Abdel-Magied wants white writers to stick with white characters, Taylor argues:

There are classes, panels, workshops, blog posts, editors, anthologies, writing series, reading series, and seminars about how to write the other, which is often taken to mean people of color. I admit that I am perplexed by the absence of classes meant to instruct us in writing white middle-class ennui. No special roundtables to discuss how best to write about straight people or cis people or able-bodied people. There is no special secret to writing about people who do not look like you. There is no technique that you need that is different from writing about self. If you can write self, you can write other.

Taylor condemns throwaway minor characters that are mere stereotypes, but also nonwhite characters who are only background characters, claiming that a writer who fails to show “a Latina cleaning woman” in her own home “is performing a violence.” Taylor’s concluding line is perhaps the most frustrating: “The solution to problematic stories, both at the level of craft and at the level of human experience, is empathy.” The notion that we’re looking for a “solution to problematic stories” is stifling, too. A story—that is, a fictional story—isn’t allowed to be “problematic”?

Shriver is right that literature depends on writers’ willingness and ability to create characters different from themselves; otherwise the fiction landscape would look even more homogenous, and the writing itself would be significantly less inspired. There is also a genuine danger in shifting literature away from authorial liberty and toward a hyper-politicized idea of what a story can be. The power of fiction—its ability to make imagined people seem more true-to-life than real ones—is lost when all novels are treated like political statements.

It doesn’t follow, however, that anyone offended by a stereotypical literary depiction is a hypersensitive millennial crybaby who fails to appreciate art. In fairly criticizing the hyper-politicization of fiction, Shriver blundered in assuming she can or should control readers’ responses.

Especially for writers from traditionally privileged demographics, the message seems to be that it’s a whole lot safer just to make all your characters from that same demographic, so you can be as hard on them as you care to be, and do with them what you like. Availing yourself of a diverse cast, you are not free; you have inadvertently invited a host of regulations upon your head, as if just having joined the EU. Use different races, ethnicities, and minority gender identities, and you are being watched.

Yes, you are being watched—but that is all. Free speech means that writers can write whatever they want, and those who’ve read their work can say whatever they want to say about it. So who is trying to silence whom, exactly?