Content Moderation Case Study: Dealing With Controversial & Sexual Fan Fiction (May 2007)
Summary: Sexual content can be challenging for content moderation on a number of different levels -- especially when it involves fictional content about taboo, controversial, or even illegal activities. Literary fiction around these topics has been controversial throughout history, including books like Vladimir Nabokov's Lolita, which focuses on a story told (somewhat unreliably) by a middle-aged male English professor who becomes obsessed with a 12 year-old girl.But while there have been widespread public debates about whether or not such written works have artistic merit or are obscene, the debate becomes different when such content is hosted on social media platforms, and raises questions about whether or not it complies with terms of service.LiveJournal, the very popular blogging platform in the mid-2000s, faced that question in 2007. A religious group called Warriors for Innocence, that was ostensibly set up to track down child abuse online, launched a public campaign accusing LiveJournal (at the time owned by another blogging company, SixApart) of harboring people promoting child sexual abuse. In response, LiveJournal suspended approximately 500 accounts. Many of the suspended accounts, however, hosted fictional writings, including fan fiction about the Harry Potter universe, as well as a (Spanish-language) LiveJournal that hosted a discussion about Nabokov's Lolita.Many of the LiveJournal users were upset about this, and argued that even if they were writing about taboo sexual content, fiction about criminal behavior is quite different than supporting or engaging in the same criminal behavior.
However given that all the material in question is fiction and artwork it seems preposterous to censor these communities. If works of fiction that address illegal or immoral activities are going to be subject to this treatment surely crime thrillers and murder mysteries should be censored just as heavily as erotic material. Part of the reason I use livejournal is because of the freedom it allowed for writers such as myself who deal with difficult and unconventional subject matter. If this purge continues I will be forced to leave livejournal find another outlet for my writing and I am sure I am not the only lj user who feels this way.Decisions to be made by SixApart:
Questions and policy implications to consider:
- How do you distinguish fictional stories that describe criminal activities from supporting or encouraging such activities?
- How responsive should a website be to public campaigns that seek to condemn certain content online?
- Should the company be judging the literary merit of written works?
Resolution: SixApart's CEO later apologized, saying that they screwed up the removal process.
- Where, if anywhere, is the line to be drawn between fictional works depicting abuse, and policies against abuse?
- Different people may view these works through very different lenses. How do you handle concerns raised by some, as compared to the creative outlet it provides others?
For reasons we are still trying to figure out what was supposed to be a well planned attempt to clean up a few journals that were violating LiveJournal's policies that protect minors turned into a total mess. I can only say I'm sorry, explain what we did wrong and what we are doing to correct these problems and explain what we were trying to do but messed up so completely.Many of the suspended journals were put back online, after each was manually reviewed by the company. He admitted that they struggled with some content that used a thin veneer of fictional or academic interest to actually promote that activity, and the company sought to shut down those accounts.
Another issue we needed to deal with was journals that used a thin veneer of fictional or academic interest in events and storylines that include child rape, pedophilia, and similar themes in order to actually promote these activities. While there are stories, essays, and discussions that include discussion of these issues in an effort to understand and prevent them, others use a pretext to promote these activities. It's often very hard to tell the difference.It is also worth noting that approximately six months after this incident, SixApart sold LiveJournal.Originally posted on the Trust & Safety Foundation website.
Read more here
posted at: 12:00am on 07-Jan-2021
path: /Policy | permalink | edit (requires password)
Pennsylvania School District Asks Supreme Court To Allow It To Continue To Violate Students' First Amendment Rights
The Mahanoy Area School District of Pennsylvania, which currently serves 11,000 residents and 1,100 students -- wants to keep wasting local and federal tax dollars defending its decision to infringe on the free speech rights of students.In 2019, the district was sued by "B.L.," a student whose f-bomb laden Snapchat message was reported to school officials. Using the vernacular of high school students, B.L. expressed her displeasure with things that often displeased students.
fuck school fuck softball fuck cheer fuck everythingAs a result of a snitch within the cheer team's midst, B.L. was booted from the cheerleading roster and suspended by the school. But the Snapchat selfie with the f-words was taken while B.L. was off campus and after school hours. It made no difference to the school, which felt it had the right to punish B.L. for any expression regarding school activities. In support, it pointed to the "agreement" B.L. had signed when she signed up for cheerleading, forbidding the creation of any "negative information" about the school while participating in extracurricular activities.The district court said the school went too far, extending its control of students' speech past the confines of the school campus. If the speech had proven to be disruptive to other students or the business of schooling, it may have been within its rights to punish B.L. But the facts showed the only people "disrupted" were the person reporting the Snapchat, and the cheerleading coach who decided to be offended by it. Oh, and B.L., who was (temporarily) no longer a student or a cheerleader.The district continued to insist it was right. It took its case to the Third Circuit Court of Appeals, which pretty much said the same thing the lower court did. Then it went further, pointing out the school's inability to stay within the confines of its own extracurricular activities policies.
B.L.’s snap contained foul language and disrespected her school and team. But the rule’s language suggests it applies only “at games, fundraisers, and other events,” a suggestion echoed by its invocation of “[g]ood sportsmanship.” Id. That would not cover a weekend post to Snapchat unconnected with any game or school event and before the cheerleading season had even begun. And common sense supports this reading: It is hard to believe a reasonable student would understand that by agreeing to the Respect Rule, she was waiving all rights to malign the school once safely off campus and in the world at large. Indeed, one of the cheerleading coaches recognized that the rule “doesn’t say anything about not being able to use foul language or inappropriate gestures . . . away from school.” J.A. 90. So this rule is of no help to the School District.This punishment of B.L. violated her rights. This is indisputable, said the Third Circuit. Any other interpretation would allow schools to punish students for exercising their First Amendment rights, no matter how, when, or where they exercised them.
As arms of the state, public schools have an interest in teaching civility by example, persuasion, and encouragement, but they may not leverage the coercive power with which they have been entrusted to do so. Otherwise, we give school administrators the power to quash student expression deemed crude or offensive—which far too easily metastasizes into the power to censor valuable speech and legitimate criticism. Instead, by enforcing the Constitution’s limits and upholding free speech rights, we teach a deeper and more enduring version of respect for civility and the “hazardous freedom” that is our national treasure and “the basis of our national strength.”The school district is unwilling to accept this. It apparently strongly believes it should be able to punish speech in whatever dislikable form it encounters. The school district is asking the Supreme Court to reverse its two federal court losses.Its Supreme Court petition [PDF] is pretty terrible. On the upside, it's short. But it opens with a disturbing assertion that makes it clear far too many schools have far too little respect for the rights of their students. Referring to its two consecutive defeats in court, the school says this:
Commentators called the decision a “bombshell”; respondents’ counsel dubbed it a “landmark.” For amici representing thousands of school districts and 1.7 million teachers, the decision is a disaster.A "disaster." That's what the district calls its inability to punish students for stating they dislike school/school activities while speaking to friends at off-campus locations. According to its brief, the school thinks its two federal courts losses will somehow prevent schools from addressing serious harms caused by off-campus speech.
Relying on Tinker, several States within the Third Circuit passed laws requiring schools to address off-campus cyberbullying or harassment with a close nexus to the school. Absent this Court’s intervention, schools within the Third Circuit cannot protect student welfare without risking damages suits.All I can say to this is "good." This will require schools to take more care when addressing cyberbullying and harassment. And if it is occurring off-campus, it's probably nothing the school should be expected to take responsibility for, even if it's nominally charged with ensuring student safety. There are holes in the coverage, to be sure. But not everything that affects students is actually a school's problem.And the opening paragraph further undercuts its case. The district believes there's a circuit split that must be resolved by the nation's top court. But there's a key element missing from this case -- an element the school states but can't actually show has any bearing on the case at hand.
In five circuits and under the law of the Pennsylvania Supreme Court, schools can discipline substantially disruptive off-campus speech that is closely linked to the school environment. But the decision below disempowered 5,800 schools within the Third Circuit from addressing this same speech.There was nothing "substantially disruptive" about B.L.'s speech. The biggest disruption was felt by B.L., who was kicked off the cheer team and suspended. Otherwise, school operations and the district's student body remained unaffected.The district also mischaracterizes the Third Circuit's ruling to advance its own narrative: that the speech was disrupted, despite all evidence to the contrary:
Respondents seek to erode this mountain into a molehill by contending (at i, 14-17) that B.L.’s speech was not substantially disruptive and thus not subject to discipline even had she spoken out at school. But the Third Circuit rejected that argument, reserving the substantial-disruption question after acknowledging that B.L.’s posts undisputedly upended the cheerleading program.The Court didn't do anything of the sort. The school only asserted that the Snapchat in question was "likely" to "disrupt" only the cheerleading program. There's a lot of ground between disrupting the regular education process and "upending" one extracurricular activity. Without more to deal with, the Third Circuit decided there wasn't enough on the record to justify setting precedent on this issue.
First, our choice to sidestep the issue in J.S. adhered to the maxim that, where possible, we should avoid difficult constitutional questions in favor of simpler resolutions. There, it was sensible to avoid the issue because we could resolve the case by applying well-settled precedent addressing the substantial disruption standard in the context of the school environment. But that is not the case here. The School District’s defense of its decision to punish B.L. focuses not on disruption of the school environment at large, but on disruption in the extracurricular context—specifically, the cheerleading program B.L. decried in her snap.The only part of the Third Circuit's decision that acknowledged any sort of "substantial disruption" was the concurring opinion, which said finding in B.L.'s favor might pose disciplinary difficulties in other situations. What the concurrence didn't say it that it found B.L.'s Snapchat "substantially disruptive."For good measure, the district conjures COVID to justify further limiting the Constitutional rights of its ~1,100 students… but with the Supreme Court, which could limit the rights of millions of students all at once.
[T]he question presented recurs constantly and has become even more urgent as COVID-19 has forced schools to operate online. Only this Court can resolve this threshold First Amendment question bedeviling the Nation’s nearly 100,000 public schools.Chances are, this court won't. Most of the nation's 100,000 schools aren't "bedeviled" by this case emanating from one small district in Pennsylvania -- one that deals with harsh punishment being handed down over speech that occurred off-campus and after school hours. There's nothing in this very brief Brief that justifies the nation's top court disturbing a narrow ruling that found a school harmed the Constitutional rights of one student by overreacting to some online swearing. Unfortunately, this means Pennsylvania residents are still paying the district to argue against respecting their rights. Hopefully, this will be dumped as soon as SCOTUS has a chance to lay its eyes on it.
Read more here
posted at: 12:00am on 07-Jan-2021
path: /Policy | permalink | edit (requires password)