Content Moderation Case Study: Discord Adds AI Moderation To Help Fight Abusive Content (2021)
Furnished content.
Summary: In the six years since Discord debuted its chat platform, it has seen explosive growth. And, over the past half-decade, Discord's chat options have expanded to include GIFs, video, audio, and streaming. With this growth and these expanded offerings, there have come a number of new moderation challenges and required adapting to changing scenarios.Discord remains largely text-based, but even when limited to its original offering -- targeted text-oriented forums/chat channels -- users were still subjected to various forms of abuse. And, because the platform hosted multiple users on single channels, users sometimes found themselves targeted en masse by trolls and other malcontents. While Discord often relies on the admins of servers to handle moderation on those servers directly, the company has found that it needs to take a more hands on approach to handling content moderation.Discord's addition of multiple forms of content create a host of new content moderation challenges. While it remained text-based, Discord was able to handle moderation using a blend of AI and human moderators.Some of the moderation load was handed over to users, who could perform their own administration to keep their channels free of content they didn't like. For everything else (meaning content that violates Discord's guidelines), the platform offered a mixture of human and AI moderation. The platform's Trust & Safety team handled content created by hundreds of millions of users, but its continued growth and expanded offerings forced the company to find a solution that could scale to meet future demands.To continue to scale, Discord ended up purchasing Sentropy, an AI company that only launched last year with the goal of building AI tools to help companies moderate disruptive behavior on their platforms. Just a few months prior to the purchase, Sentropy had launched its first consumer-facing product, an AI-based tool for Twitter users to help them weed out and block potentially abusive tweets. However, after being purchased, Sentropy shut down the tool, and is now focused on building out its AI content moderation tools for Discord.Discord definitely has moderation issues it needs to solve -- which range from seemingly-omnipresent spammers to interloping Redditors with a taste for tasteless memes -- but it remains to be seen whether the addition of another layer of AI will make moderation manageable.Company Considerations:- What advantages can outside services offer above what platforms can develop on their own?
- What are the disadvantages of partnering with a company whose product was not designed to handle a platform's specific moderation concerns?
- How do outside acquisitions undermine ongoing moderation efforts? Conversely, how do they increase the effectiveness of ongoing efforts?
- How should platforms handle outside integration of AI moderation as it applies to user-based moderation efforts by admins running their own Discord servers?
- How much input should admins have in future moderation efforts? How should admins deal with moderation calls made by AI acquisitions that may impede efforts already being made by mods on their own servers?
Issue Considerations:- What are the foreseeable negative effects of acquiring content moderation AI designed to handle problems observed on different social media platforms?
- What problems can outside acquisitions introduce into the moderation platform? What can be done to mitigate these problems during integration?
- What negative effect can additional AI moderation efforts have on "self-governance" by admins entrusted with content moderation by Discord prior to acquisition of outside AI?
Resolution: So far, the acquisition has yet to produce much controversy. Indeed, Discord as a whole has managed to avoid many of the moderation pitfalls that have plagued other platforms of its size. Its most notorious action to date was its takeover of the WallStreetBets server as it went supernova during a week or two of attention-getting stock market activity. An initial ban was rescinded once the server's own moderators began removing content that violated Discord guidelines, accompanied by Discord's own moderators who stepped in to handle an unprecedented influx of users while WallStreetBets continued to make headlines around the nation.Other than that, the most notable moderation efforts were made by server admins, rather than Discord itself, utilizing their own rules which (at least in one case) exceeded the restrictions on content delineated in Discord's terms of use.Originally posted to the Trust & Safety Foundation website.
Read more here
posted at: 12:00am on 02-Dec-2021
path: /Policy | permalink | edit (requires password)
Take-Two Interactive Appears To Be Morphing From Game Publisher Into IP Troll
Furnished content.
There is this thing that sometimes happens to companies that are wildly successful where they stop focusing so much on making the things that made them successful and turn instead to intellectual property trolling. Think Atari, for instance. Atari was once a behemoth in the gaming industry, but have since been reduced to trying to bully and/or sue everyone who comes even remotely close to referencing one of its properties, rather than making any real hay in the industry.This process of devolving from competing business to grifting troll is at its most obvious in the end-state, where Atari is now. I won't pretend that a company like Take-Two is anywhere near there just yet, but it is notable that we have seen a recent uptick in the company engaging in this restrictive and trollish behavior. Whether it's fighting its own modding community to release shitty versions of its own older games or filing trademark oppositions for axe-throwing companies (yes, seriously), this just isn't where the focus should be for a company known to make amazing AAA video game titles.There is apparently more here than had been previously known. NME has a decent writeup discussing all the IP action Take-Two is getting involved in, some of which includes the examples mentioned above. However, it also includes a couple of other examples of trademark bullying I hadn't seen previously. The first deals briefly with an opposition for a trademark application by Hazelight Studios for its game It Takes Two.The owner of Rockstar Games has filed hundreds of trademark disputes to the United States Patent and Trademark Office over words and phrases like “2K”, “Rockstar” and “Take-Two”.This included the publisher attempting to file a trademark dispute over It Takes Two with Hazelight Studios. After being published in May of last year, the trademark was abandoned on March 20 this year. As the post notes, the application was abandoned by Hazelight Studios in March of this year. There is no detail on offer as to whether Take-Two's threat of opposition was the reason for the abandonment, but that's mostly besides the point. That point being: this is a silly opposition to begin with. The studio's name being somewhat similar to the name of the planned game doesn't really point to any serious concern about public confusion mixing up the two. This appears to be pure trademark maximalism, or bullying.The second might be closer to a valid opposition, though it still all seems rather silly.A more recent and open trademark filing from Take-Two concerns Max Pain, a business which according to opencorporates.com promotes Twitch streamers, Youtube content creators and Esports teams. Max Pain was given an incorporation date (the date it can commence business) of June 29 2020.Whilst the US Patent and trademark office doesn’t appear to list what this specific trademark is over from Take-Two, it appears as though the publisher is filing in connection to Max Payne, the series developed by Rockstar. The trademark dispute has been extended, and appears to be ongoing. In this case, Max Payne and Max Pain sound the same when spoken, except that the services rendered appear to be significantly different. Sure, streamers stream video games sometimes. Yes, eSports involve video games. But is anyone really looking at a promotion agency for streamers and eSports teams called Max Pain and thinking that this has anything to do with Take-Two's video game franchise? That seems pretty unlikely.But wherever you come down on these individual instances, it sure feels like Take-Two is becoming more and more focused on being the IP police rather than doing what made the company the giant it is in the industry: making outstanding AAA games. And that, in turn, feels like it could be the starting point for Take-Two Atari-izing itself.And that would be a damned shame.
Read more here
posted at: 12:00am on 02-Dec-2021
path: /Policy | permalink | edit (requires password)
|
|