Global AI AutoMod does not understand context
Right now making mature jokes in chat can get chatters permanently suspended globally on Twitch completely without any human intervention that requires context.
Clarification:
This is not about regular AutoMod but the HIDDEN global AI AutoMod present in all of Twitch channels.
The AI based AutoMod Twitch currently uses everywhere does not understand important contexts and takes all text at face value. Streamers who have a strong focus on the LGBTQ community find themselves having to tell their chatters not to express themselves freely, because what they joke about can be seen as hate speech even though they are strong supporters and allies. For moderators it seems they are not allowed to do what they are supposed to do, which is to determine what is appropriate in their streamers chats.
Anything of the following in chat currently gets flagged by AI based AutoMod:
- Hints at self harm, even if clearly a reaction to a video game or made as a joke.
- Threat of violence even if clearly in-game or as a joke. The context of the game being played may portray or make the same joke that is being commented on.
- Jokes towards a protected group, even if the person who is making the joke belongs to that protected group themselves.
- Sexual advances, even if clearly consensual, requested or welcomed by the streamer.
Right now it seems anything of the above in combination with a manual report will get your account permanently suspended. Partnered and mods for the channel it was flagged in appear to get more leniency.
While it is Twitch responsibility to protect all members of their community the current AI has become overly sensitive, especially for streamers who focus on mature themes and stream games that contain mature themes.
As long as streamers have their content classification set appropriately, should that not also extend to Twitch chat?
I suggest allowing "mature content" streamers to:
- Turn off the global AI based AutoMod for their chat.
- Reduce global AI based AutoMod sensitivity for their chat. (at the very least)
As long as they have set their stream content classification properly to:
Mature-rated games; Sexual themes; Drugs, Intoxication, or Excessive Tobacco Use; Violent and Graphic Depictions; Significant Profanity or Vulgarity; or Gambling. https://safety.twitch.tv/s/article/Content-Classification-Guidelines?language=en_US
Being able to discuss or joke about topics on screen of a sensitive nature is no longer possible in Twitch chat and streamers are seeing their most active chatters and supporters get permanently suspended for participating in completely consensual conversations.
I hope this can lead to further discussion regarding the matter.
Outsourcing moderation to AI, while it may save money, is not a good idea.
Hi, thanks for taking the time to share your feedback with us. We wanted to clarify a couple of things. AutoMod can’t ban people from Twitch. Mods can use the tool to identify chat messages that break their channel’s rules. But AutoMod can’t be used to timeout, ban, or mute users from any channel, or from Twitch – those aren’t AutoMod capabilities.
We also wanted to share a bit more about how we enforce our Community Guidelines. These are the policies that apply to all of Twitch, and help make Twitch safer. We have human teams that review our content moderation decisions. We call this “human in the loop.” These reviewers help ensure that our policies are being applied accurately. We don’t use a “global AI” system to enforce our guidelines, and we manually review suspension decisions.
We recognize that, while the majority of enforcements we issue are correct, sometimes we may get it wrong. If you feel an action taken against your account was not warranted, we recommend submitting an appeal through our appeals portal for our teams to review.
Again, thanks for sharing your experience. It’s important that we’re open about how we enforce our policies. It’s also on us to ensure that our enforcement decisions are accurate. Our teams issue and review content moderation decisions in real time for that reason.
-
Ntrpy__
commented
**** twitch, show nip
-
DuvalFunk
commented
This is already negatively impacting streams. Viewers are becoming afraid to chat and partake in jokes on stream for fear of getting auto banned.
-
kevvywevy
commented
I find the issue is that automod just takes the literal meaning of words without context. If this to be fixed would have to take 2-3 other messages to understand the meaning.
-
Sir_LongButt_McFugly
commented
AI moderation is faulty, you’re just being cheap. Keep dropping in quality and your platform suffers. This effects streamers and viewers. Stop being so greedy and use human moderation
-
wumboinator
commented
If every step of the moderation system is AI then don’t supply the illusion of an appeal process
-
arcthelink
commented
Ya like jazz? Don't let robots control forever decisions about real people.
Yours truly, highly regarded.
-
icanmakebagels
commented
Twitch, you never cease to let down your most endangered communities while proclaiming to fight in their honor.
-
Rancor515
commented
AI has trouble understanding the nuances of human interaction and conversation, thus is not suitable to automatically decide whether or not a user should be banned. Furthermore, streamers foster different communities and different types of jokes or speech, which AI cannot distinguish between.
-
notsafefordavid
commented
It's broken, and you wanting it to not be broken doesn't make it magically fixed.
-
kingmartell
commented
If anything, this ban should apply to:
The one chat only
Be limited to 7 days
Be able to be over riden by that
streamer/mods on the spot -
nyakittyy
commented
hey man your system is failing miserably, maybe consider crossing this bridge differently.
-
xhighjinkss
commented
If you implement this I hope you're all ready to kill your platform even more, all I'm going to say.
-
iiroro
commented
Let any streamer decide how they want moderation to occur, do not make new options automatic or opt out. Make them opt in only, and send better notifications of new moderation tools.
-
Trashgerbil
commented
Banning any user, especially moderators with years of clean history, for speaking on something in context and following both a streamers and the platforms TOS is a fast way to lose yourself tons of money. Just allow people to moderate their own communities and IF they need more assistance, make yourself available then.
-
silentriku
commented
Context is extremely important in regards to what is chatted and without context an indefinite ban can and will harm thousands of users that can end up banned for talking about the game a streamer is literally playing. Twitch is a website built by it's users and if paying users are banned then the platform is bound to fail
-
NightmareMan328
commented
Free Senor
-
Chocodave
commented
Can we add to this... ANY appeal of an AI-ban should be reviewed by a human completely, with context, as a bare minimum.
-
thumb_stick
commented
It's very unreasonable to have something without critical thinking making forever decisions. It's also making me lose respect for the platform when there is zero communication to creators on what Twitch is doing on the backend so they can advise their community.
This needs revision.
-
Keabz
commented
using an AI for moderation is straight up irresponsible
-
darkershadow96
commented
This is such a dumb idea. Let streamers moderate their own chat. AI is not going to work here.