Content Moderation Case Study: Removing Nigerian Police Protest Content Due To Confusion With COVID Misinfo Rules (2020)
Furnished content.
Summary: With the beginning of the COVID-19 pandemic, most of the large social media companies very quickly put in place policies to try to handle the flood of disinformation about the disease, responses, and treatments. How successful those new policies have been is subject to debate, but in at least one case, the effort to fact check and moderate COVID information ran into a conflict with people reporting on violent protests (totally unrelated to COVID) in Nigeria.
Yesterday our systems were incorrectly flagging content in support of #EndSARS, and marking posts as false. We are deeply sorry for this. The issue has now been resolved, and we apologize for letting our community down in such a time of need.Facebook's head of communications for sub-Saharan Africa, Kezia Anim-Addo, gave Tomiwa Ilori, writing for Slate, some more details on the combination of errors that resulted in this unfortunate situation:
In our efforts to address misinformation, once a post is marked false by a third party face checker, we can use technology to fan out and find duplicates of that post so if someone sees an exact match of the debunked post, there will also be a warning label on it that it's been marked as false.In this situation, there was a post with a doctored image about the SARS virus that was debunked by a Third-Party Fact Checking partnerThe original false image was matched as debunked, and then our systems began fanning out to auto-match to other imagesA technical system error occurred where the doctored images was connected to another different image, which then also incorrectly started to be matched as debunked. This created a chain of fan outs pulling in more images and continuing to match them as debunked.This is why the system error accidentally matched some of the #EndSARS posts as misinformation.Thus, it seems like a combination of factors was at work here, including a technical error and the similarities in the SARS name.Originally posted to the Trust & Safety Foundation website.
edit: Policy/auto___content_moderation_case_study__removing_nigerian_police_protest_content_due_to_confusion_with_covid_misinfo_rules__2020_.wikieditish...