Facebook AI Moderation Continues To Suck Because Moderation At Scale Is Impossible

Furnished content.


For several years now, we've been beating the idea that content at moderation is impossible to get right, otherwise known as Masnick's Impossibility Theorem. The idea there is not that platforms shouldn't do any form of moderation, or that they shouldn't continue to try to improve the method for moderation. Instead, this is all about expectations setting, partially for a public that simply wants better content to show up on their various devices, but even more so for political leaders that often see a problem happening on the internet and assume that the answer is simply "moar tech!".Being an internet behemoth, Facebook catches a lot of heat for when its moderation practices suck. Several years ago, Mark Zuckerberg announced that Facebook had developed an AI-driven moderation program, alongside the claim that this program would capture "the vast majority" of objectionable content. Anyone who has spent 10 minutes on Facebook in the years since realizes how badly Facebook failed towards that goal. And, as it turns out, failed in both directions.By that I mean that, while much of our own commentary on all this has focused on how often Facebook's moderation ends up blocking non-offending content, a recent Ars Technica post on just how much hate speech makes its way onto the platform has some specific notes about how some of the most objectionable content is misclassified by the AI moderation platform.

Facebook’s internal documents reveal just how far its AI moderation tools are from identifying what human moderators were easily catching. Cockfights, for example, were mistakenly flagged by the AI as a car crash. “These are clearly cockfighting videos,” the report said. In another instance, videos livestreamed by perpetrators of mass shootings were labeled by AI tools as paintball games or a trip through a carwash.

Still, Facebook’s leadership has been more concerned with taking down too many posts, company insiders told WSJ. As a result, they said, engineers are now more likely to train models that avoid false positives, letting more hate speech slip through undetected.

0 comments, click here to add the first

Content Moderation Case Studies: Snapchat Disables GIPHY Integration After Racist 'Sticker' Is Discovered (2018)

Furnished content.


Summary: Snapchat debuted to immediate success a decade ago, drawing in millions of users with its playful take on instant messaging that combined photos and short videos with a large selection of filters and "stickers." Stickers are graphics that can be applied to messages, allowing users to punch up their presentations (so to speak).Snapchat’s innovations in the messaging space proved incredibly popular, moving Snapchat from upstart to major player in a few short years. It also created more headaches for moderators as sent messages soared past millions per day to billions.Continuing its expansion of user options, Snapchat announced its integration with Giphy, a large online repository of GIFs, in February 2018. This gave users access to Giphy's library of images to use as stickers in messages.But the addition of thousands of images to billions of messages quickly resulted in an unforeseen problem. In early March of 2018, Snapchat users reported a search of the GIPHY image database for the word "crime" surfaced a racist sticker, as reported by Josh Constine for TechCrunch:

“We first reported Instagram was building a GIPHY integration back in January before it launched a week later, with Snapchat adding a similar feature in February. But it wasn’t long before things went wrong. First spotted by a user in the U.K. around March 8th, the GIF included a racial slur.” — Josh Constine, TechCrunch

• What measures can be put in place to prevent moderation problems from moving from one platform to another during cross-platform integration?
• What steps should be taken prior to launch to integrate moderation efforts between platforms? 
• What can "upline" content providers do to ensure content moving from their platforms to others meets the content standards of the "downline" platforms? 

• What procedures aid in facilitating cross-platform moderation? 
• Which party should have final say on moderation efforts, the content provider or the content user?

“We’ve been in close contact with GIPHY throughout this process and we’re confident that they have put measures in place to ensure that Instagram users have a good experience” an Instagram spokesperson told TechCrunch.

After investigation of the incident, this sticker was available due to a bug in our content moderation filters specifically affecting GIF stickers.

We have fixed the bug and have re-moderated all of the GIF stickers in our library.

The GIPHY staff is also further reviewing every GIF sticker by hand and should be finished shortly.

0 comments, click here to add the first