Sat, 12 Dec 2020

Content Moderation Case Study: Facebook's AI Continues To Struggle With Identifying Nudity (2020)
Furnished content.


Summary: Since its inception, Facebook has attempted to be more "family-friendly" than other social media services. Its hardline stance on nudity, however, has often proved problematic, as its AI (and its human moderators) have flagged accounts for harmless images and/or failed to consider context when removing images or locking accounts.The latest example of Facebook's AI failing to properly moderate nudity involves garden vegetables. A seed business in Newfoundland, Canada was notified its image of onions had been removed for violating the terms of service. Its picture of onions apparently set off the auto-moderation, which flagged the image for containing "products with overtly sexual positioning." A follow-up message noted the picture of a handful of onions in a wicker basket was "sexually suggestive."

• Should more automated nudity/sexual content decisions be backstopped by human moderators?
• Is the possibility of over-blocking worth the reduction in labor costs?
• Is over-blocking preferable to under-blocking when it comes to moderating content?
• Is Facebook large enough to comfortably absorb any damage to its reputation or user goodwill when its moderation decisions affect content that doesn't actually violate its policies?
• Is it even possible for a platform of Facebook's size to accurately moderate content and/or provide better options for challenging content removals?

• Is the handling of nudity in accordance with the United States' more historically Puritianical views really the best way to moderate content submitted by users all over the world?
• Would it be more useful to users is content were hidden -- but not deleted -- when it appears to violate Facebook's terms of service, allowing posters and readers to access the content if they choose to after being notified of its potential violation?
• Would a more transparent appeals process allow for quicker reversals of incorrect moderation decisions?

"We use automated technology to keep nudity off our apps," wrote Meg Sinclair, Facebook Canada's head of communications. "But sometimes it doesn't know a walla walla onion from a, well, you know. We restored the ad and are sorry for the business' trouble."

edit: Policy/auto___content_moderation_case_study__facebook_s_ai_continues_to_struggle_with_identifying_nudity__2020_.wikieditish...