Post No Evil: Content Moderation Decisions Are Always Trickier Than You Think
Furnished content.
Two years ago, we told anyone who wanted to understand the impossibility of content moderation to listen to an episode of the podcast/radio show Radiolab. Obviously, content moderation questions are back in the news again, and Radiolab recently re-released the episode with some updated content. Most of it is the same, but there's some more at the end to relate it to the latest news with the various attacks on social media coming from the president, the DOJ, and Congress.Once again, I cannot recommend anything more than listening to this entire discussion. It's full of great examples of the impossible nature of content moderation. And, it especially highlights why the various proposals brought forth by Congress that say that social media companies need to have explicit rules for what is and what is not allowed is simply not practical in the real world, where there are so many edge cases, and so many times where the policy needs to be adapted due to a new edge case. Here's just a little bit of the transcript to whet your appetite for the whole damn thing. It's an early example in the story, highlighting the difficulty of dealing with breast feeding pictures under its policy:
[NEWS CLIP: A social networking website is under fire for its policy on photos of women breastfeeding their children.]SIMON: Big time.STEPHANIE MUIR: 12,000 members participated, and the media requests started pouring in.[NEWS CLIP: The Facebook group called, "Hey Facebook: Breastfeeding is Not Obscene.]STEPHANIE MUIR: I did hundreds of interviews for print. Chicago Tribune, Miami Herald, Time Magazine, New York Times, Washington Post ...[ARCHIVE CLIP, Dr. Phil: You know, the internet is an interesting phenomenon.]STEPHANIE MUIR: ... Dr. Phil. It was a media storm. And eventually, perhaps as a result of our group and our efforts, Facebook was forced to get much more specific about their rules.SIMON: So for example, by then nudity was already not allowed on the site. But they had no definition for nudity. They just said no nudity. And so the Site Integrity Team, those 12 people at the time, they realized they had to start spelling out exactly what they meant.KATE KLONICK: Precisely. All of these people at Facebook were in charge of trying to define nudity.FACEBOOK EMPLOYEE: So I mean yeah, the first cut at it was visible male and female genitalia. And then visible female breasts. And then the question is well, okay, how much of a breast needs to be showing before it's nude? And the thing that we landed on was, if you could see essentially the nipple and areola, then that's nudity.SIMON: And it would have to be taken down. Which theoretically at least, would appease these protesters because, you know, now when a picture would pop up of a mother breastfeeding, as long as the child was blocking the view of the nipple and the areola, they could say, "Cool, no problem."KATE KLONICK: Then you start getting pictures that are women with just their babies on their chest with their breasts bare. Like, for example, maybe baby was sleeping on the chest of a bare-breasted woman and not actively breastfeeding.FACEBOOK EMPLOYEE: Okay, now what? Like, is this actually breastfeeding? No, it's actually not breastfeeding. The woman is just holding the baby and she has her top off.JAD: Yeah, but she was clearly just breastfeeding the baby.ROBERT: Well, maybe just before.SIMON: Well, I would say it's sort of like kicking a soccer ball. Like, a photo of someone who has just kicked a soccer ball, you can tell the ball is in the air, but there is no contact between the foot and the ball in that moment potentially. So although it is a photo of someone kicking a soccer ball, they are not, in fact, kicking the soccer ball in that photo.ROBERT: [laughs]JAD: [laughs] That's a good example.SIMON: And this became the procedure or the protocol or the approach for all of these things, was we have to base it purely on what we can see in the image.KATE KLONICK: And so they didn't allow that to stay up under the rules, because it could be too easily exploited for other types of content, like nudity or pornography.FACEBOOK EMPLOYEE: We got to the only way you could objectively say that the baby and the mother were engaged in breastfeeding is if the baby's lips were touching the woman's nipple.SIMON: So they included what you could call, like, an attachment clause. But as soon as they got that rule in place ...FACEBOOK EMPLOYEE: Like, you would see, you know, a 25-year-old woman and a teenage-looking boy, right? And, like, what the hell is going on there?KATE KLONICK: Oh, yeah. It gets really weird if you, like, start entering into, like, child age. And I wasn't even gonna bring that up because it's kind of gross.FACEBOOK EMPLOYEE: It's like breastfeeding porn.JAD: Is that a thing?ROBERT: Are there sites like that?SIMON: Apparently. And so this team, they realized they needed to have a nudity rule that allowed for breastfeeding but also had some kind of an age cap.FACEBOOK EMPLOYEE: So -- so then we were saying, "Okay. Once you've progressed past infancy, then we believe that it's inappropriate."SIMON: But then pictures would start popping up on their screen and they'd be like, "Wait. Is that an infant?" Like, where's the line between infant and toddler?FACEBOOK EMPLOYEE: And so the thing that we landed on was, if it looked like the child could walk on his or her own, then too old.SIMON: Big enough to walk? Too big to breastfeed.ROBERT: Oh, that could be 18 months.JAD: Yeah, that's like a year old in some cases.SIMON: Yeah. And, like, the World Health Organization recommends breastfeeding until, you know, like, 18 months or two years, which meant there were a lot of photos still being taken down.KATE KLONICK: Within days, we were continuing to hear reports from people that their photographs were still being targeted.SIMON: But ...[NEWS CLIP: Facebook did offer a statement saying ...]FACEBOOK EMPLOYEE: You know, that's where we're going to draw the line.Suffice it to say, that is not, in fact, where Facebook drew the line. Indeed, just last year we wrote about the company still having issues with drawing the line around this particular issue.So if you want to talk intelligently about these issues, you should first listen to the Radiolab broadcast. It's only a little over an hour, and well worth your time:
edit: Policy/auto___post_no_evil__content_moderation_decisions_are_always_trickier_than_you_think.wikieditish...