Global AI AutoMod does not understand context
Right now making mature jokes in chat can get chatters permanently suspended globally on Twitch completely without any human intervention that requires context.
Clarification:
This is not about regular AutoMod but the HIDDEN global AI AutoMod present in all of Twitch channels.
The AI based AutoMod Twitch currently uses everywhere does not understand important contexts and takes all text at face value. Streamers who have a strong focus on the LGBTQ community find themselves having to tell their chatters not to express themselves freely, because what they joke about can be seen as hate speech even though they are strong supporters and allies. For moderators it seems they are not allowed to do what they are supposed to do, which is to determine what is appropriate in their streamers chats.
Anything of the following in chat currently gets flagged by AI based AutoMod:
- Hints at self harm, even if clearly a reaction to a video game or made as a joke.
- Threat of violence even if clearly in-game or as a joke. The context of the game being played may portray or make the same joke that is being commented on.
- Jokes towards a protected group, even if the person who is making the joke belongs to that protected group themselves.
- Sexual advances, even if clearly consensual, requested or welcomed by the streamer.
Right now it seems anything of the above in combination with a manual report will get your account permanently suspended. Partnered and mods for the channel it was flagged in appear to get more leniency.
While it is Twitch responsibility to protect all members of their community the current AI has become overly sensitive, especially for streamers who focus on mature themes and stream games that contain mature themes.
As long as streamers have their content classification set appropriately, should that not also extend to Twitch chat?
I suggest allowing "mature content" streamers to:
- Turn off the global AI based AutoMod for their chat.
- Reduce global AI based AutoMod sensitivity for their chat. (at the very least)
As long as they have set their stream content classification properly to:
Mature-rated games; Sexual themes; Drugs, Intoxication, or Excessive Tobacco Use; Violent and Graphic Depictions; Significant Profanity or Vulgarity; or Gambling. https://safety.twitch.tv/s/article/Content-Classification-Guidelines?language=en_US
Being able to discuss or joke about topics on screen of a sensitive nature is no longer possible in Twitch chat and streamers are seeing their most active chatters and supporters get permanently suspended for participating in completely consensual conversations.
I hope this can lead to further discussion regarding the matter.
Outsourcing moderation to AI, while it may save money, is not a good idea.
Hi, thanks for taking the time to share your feedback with us. We wanted to clarify a couple of things. AutoMod can’t ban people from Twitch. Mods can use the tool to identify chat messages that break their channel’s rules. But AutoMod can’t be used to timeout, ban, or mute users from any channel, or from Twitch – those aren’t AutoMod capabilities.
We also wanted to share a bit more about how we enforce our Community Guidelines. These are the policies that apply to all of Twitch, and help make Twitch safer. We have human teams that review our content moderation decisions. We call this “human in the loop.” These reviewers help ensure that our policies are being applied accurately. We don’t use a “global AI” system to enforce our guidelines, and we manually review suspension decisions.
We recognize that, while the majority of enforcements we issue are correct, sometimes we may get it wrong. If you feel an action taken against your account was not warranted, we recommend submitting an appeal through our appeals portal for our teams to review.
Again, thanks for sharing your experience. It’s important that we’re open about how we enforce our policies. It’s also on us to ensure that our enforcement decisions are accurate. Our teams issue and review content moderation decisions in real time for that reason.
-
Bpapman
commented
I was wondering why I got a 7 day ban randomly for unsolicited sexual advances. Seems making a joke about a 'twitch 7 day vacation' is now ban worthy. Appeal was denied in 20minutes after giving context so that might be from a bot too, or an employee who was active at 2am pst on a Saturday surely. Either way insane to think this is a global ban without any user triggering it. Or if users can trigger it you can target ban people making similar jokes even knowing the context. Why have automods ban keywords in chat if you can receive a global ban for it anyways? Just a dumb system
-
GaryOak_
commented
Using AI to automod is a catastrophically stupid idea. Let the moderators keep control of their chats, and if users are flouting the ToS, they can be reported and banned.
I find it inane that people are being banned automatically for comments that are perfectly fine in the context of the chat and/or the stream's culture (both of which operating within ToS, mind you), while a channel I moderate has a ban evading user with literally over 200 accounts who comes back within a minute of being banned. Several people have manually reported him, yet he's there every single stream.
Why are real bad actors allowed to exist, while innocent people are being banned by an automod that is incapable of understanding context? Are you trying to drive people toward Kick? I do not even have a Kick account, but developments such as this only encourage me to make one.
-
AverageBritishWhiteBoy
commented
The queer community is one of the most targeted by this, nearly every week there is a person that has been banned from obviously something that isn't warranted. People are purposely reporting members of queer streams even when they know they are innocent, its purely being used in these communities as another way to harass and silence queer voices that are already diminished. It really does say something when a queer person gets banned for purely existing. Let mods dictate what is tolerable in communities and maybe focus on streamers and communities that harass those that aren't straight white men.
-
kirbskirby1
commented
People have been abusing the automod to get innocent chatters banned. It does not understand context and the appeals process also is not very helpful. It should at least show the message that was reported and just not repeated Twitch ToS terms. I have seen at least 10 innocent chatters banned from abuse of the system. This includes 10 year old banned accounts from when we still had Justin.tv. These people are harassing accounts in LGBTQIA+ communities. What is the point of an AI moderator scanning chats with no context or intention when there are channel moderators in streams?? I'd honestly rather dodge homophobia and racism on Kick if this isn't fixed.
-
AveryWumbo
commented
People will abuse this and have innocent chatters banned. They harass entire communities and they can't be stopped because of how easy it is to abuse this. Twitch can afford human mods to review these reports.
-
louisluv
commented
I got banned for 2 weeks and I barely type in twitch chats, just occasionally in one (with a large LGBTQ+ community that I know has a huge problem with this) and another that I rarely type in.
I definitely do not message hate speech or anything bannable, but taken out of context I can guess that maybe I had said something that could be seen as hate speech/violent.
On top of that, when I messaged twitch to find out what I said to get me banned for so long, I got that automated response because clearly they do not care to make sure their bans are deserved.
The worst part about all of this is not that I couldn't type for 2 weeks... but the ads! I pay for my subscriptions to not have to sit through them, but for 14 days I had to because twitch doesn't care about fixing a problem they've had for YEARS!
This problem is clearly targeting communities of minorities because trolls know mass banning random people will work. -
Bachlavuh
commented
Let the mods mod. This is completely harmful to long lasting and loving communities that have built diversity and inclusivity as part of their daily programming. AI should not be doing the job of human policing and free speech for the simple fact that AI is incapable of sarcasm, context, and building lasting relationships /personas within the community.
-
Jeebleschrist
commented
I just don't understand why Twitch feels there needs to be an automod feature using AI to scan every chat. AI depends on data, which are often imperfect or incomplete. It has no place in moderation, which requires professional judgment. The internet has gone 30+ years without needing automodding on forums and websites, so stop pretending like it's needed now. Mods appointed by streamers do their jobs more than well enough, please stay away because you're doing more harm than good. This isn't the past where there were no legitimate competitors to Twitch, sites like Kick are within striking distance largely due to Twitch's heavy-handed approach to moderation.
-
foxycheshire
commented
Horrible and needs to be immediately removed, the AI can’t tell if the chatter is responding to the streamer or a movie. More importantly when talking about politics the AI will not be able to differentiate the talking points and is banning people! I will not financially support twitch if this continues. There are humans who are mods in each community !
-
sad__chatter
commented
The number of people I've seen falsely banned in the last week and a half has been insane - and if you try to appeal your wait time is longer than the ban itself (and these are like 14 day bans). Everyday another chatter get's got, it's posted about on the streamers discord, the streamer themselves doesn't agree with it, they get what's being said that's why the mods didn't ban it. If that chatter is also a streamer, they're off to kick, same with the community that supports them. This here, is like twitch directly attacking the individual communities, and the first time.
-
dastarfruit
commented
The number of people I've seen falsely banned in the last week and a half has been insane - and if you try to appeal your wait time is longer than the ban itself (and these are like 14 day bans).
-
sporeian94
commented
The industrial revolution and it's consequences has been a DISASTER for the human race...
-
SkuggiXD
commented
Everyday another chatter get's got, it's posted about on the streamers discord, the streamer themselves doesn't agree with it, they get what's being said that's why the mods didn't ban it. If that chatter is also a streamer, they're off to kick, same with the community that supports them. This here, is like twitch directly attacking the individual communities, and the first time.
-
Wraeththix
commented
I've been an active Twitch user for quite some time, and I've also been a large user of the gifted sub/bits/etc system. I've found myself moving to alternative sites, just purely because the community around me is being slowly picked away by this system. It may seem like it's saving money for now, but I know specifically it's costing money from users like me.
-
IndyNavy
commented
Get rid of this asap
-
SuprMeatDude
commented
We don't need twitch automod when every channel already has moderators...
-
LsTheRoberto
commented
Every channel has their own ‘in jokes’. They may not make sense out of context, but what jokes do. Using AI to moderate an online messages, that does not have a concept of context will always over moderate.
Why can the streamer not control the level of moderation from this tool? Why can the streamer not un banned those banned by this tool in their chat? It just makes no sense.Either allow this streamer to control the tool (set allow/deny list words, turn on/off auto ban vs a hold of the message, etc). or allow the streamer to unban those who did not break the streamers community guidelines. Or allow the streamer to turn off this tool in their chat.
-
wubbysrightnip
commented
FREE THE NIP. WUBBY7
-
Mitsota101
commented
great way to kill your userbase twitch, I don't even use the chat system anymore because I'm affraid of what your AI will randomly consider an inappropriate comment
-
don_kiwi
commented
free the nips