Global AI AutoMod does not understand context
Right now making mature jokes in chat can get chatters permanently suspended globally on Twitch completely without any human intervention that requires context.
Clarification:
This is not about regular AutoMod but the HIDDEN global AI AutoMod present in all of Twitch channels.
The AI based AutoMod Twitch currently uses everywhere does not understand important contexts and takes all text at face value. Streamers who have a strong focus on the LGBTQ community find themselves having to tell their chatters not to express themselves freely, because what they joke about can be seen as hate speech even though they are strong supporters and allies. For moderators it seems they are not allowed to do what they are supposed to do, which is to determine what is appropriate in their streamers chats.
Anything of the following in chat currently gets flagged by AI based AutoMod:
- Hints at self harm, even if clearly a reaction to a video game or made as a joke.
- Threat of violence even if clearly in-game or as a joke. The context of the game being played may portray or make the same joke that is being commented on.
- Jokes towards a protected group, even if the person who is making the joke belongs to that protected group themselves.
- Sexual advances, even if clearly consensual, requested or welcomed by the streamer.
Right now it seems anything of the above in combination with a manual report will get your account permanently suspended. Partnered and mods for the channel it was flagged in appear to get more leniency.
While it is Twitch responsibility to protect all members of their community the current AI has become overly sensitive, especially for streamers who focus on mature themes and stream games that contain mature themes.
As long as streamers have their content classification set appropriately, should that not also extend to Twitch chat?
I suggest allowing "mature content" streamers to:
- Turn off the global AI based AutoMod for their chat.
- Reduce global AI based AutoMod sensitivity for their chat. (at the very least)
As long as they have set their stream content classification properly to:
Mature-rated games; Sexual themes; Drugs, Intoxication, or Excessive Tobacco Use; Violent and Graphic Depictions; Significant Profanity or Vulgarity; or Gambling. https://safety.twitch.tv/s/article/Content-Classification-Guidelines?language=en_US
Being able to discuss or joke about topics on screen of a sensitive nature is no longer possible in Twitch chat and streamers are seeing their most active chatters and supporters get permanently suspended for participating in completely consensual conversations.
I hope this can lead to further discussion regarding the matter.
Outsourcing moderation to AI, while it may save money, is not a good idea.
Hi, thanks for taking the time to share your feedback with us. We wanted to clarify a couple of things. AutoMod can’t ban people from Twitch. Mods can use the tool to identify chat messages that break their channel’s rules. But AutoMod can’t be used to timeout, ban, or mute users from any channel, or from Twitch – those aren’t AutoMod capabilities.
We also wanted to share a bit more about how we enforce our Community Guidelines. These are the policies that apply to all of Twitch, and help make Twitch safer. We have human teams that review our content moderation decisions. We call this “human in the loop.” These reviewers help ensure that our policies are being applied accurately. We don’t use a “global AI” system to enforce our guidelines, and we manually review suspension decisions.
We recognize that, while the majority of enforcements we issue are correct, sometimes we may get it wrong. If you feel an action taken against your account was not warranted, we recommend submitting an appeal through our appeals portal for our teams to review.
Again, thanks for sharing your experience. It’s important that we’re open about how we enforce our policies. It’s also on us to ensure that our enforcement decisions are accurate. Our teams issue and review content moderation decisions in real time for that reason.
-
Abyssal_DPRKed commented
My account got indefinitely suspended by the AI, and twitch appeals rejected my appeal in which I said "I'm a moderator at the chat in which it happened and I had no intention of being malicious with my message." Any suggestions?
-
joshtron07 commented
Hi
-
KnightKeeeper commented
my main account was banned not that long ago and for a month i been making new accounts yeah evading is against TOS but since they denied my application for an appeal after not only giving it me back the day prior. for context i was banned firstly for "unsolicited sexual advances" got that appealed then the next day im banned for "hate speech" to which my appeal was denied so im in this limbo point of making new accounts and them being banned every few days cause twitch has zero way of contacting them. if anyone can help me with this id love the advice. the streamers i chat too and the mods have 0 clue why i would be banned as they looked at my message logs and saw nothing wrong so yeah idk what the heck to do really.
-
Cragsand commented
@saysera
This is what Twitch claims but the truth is that a manual report in combination with an AI flag does get you suspended. I've seen it countless times and I have a social media post with examples I won't link here. Using low pay workers with zero incentive to conduct a proper investigation or look at VODs to get the complete context is another factor we don't have transparency on.Since this global AI AutoMod remains an undocumented "feature" of Twitch chat from a while back a lot of the conclusions I've listed in the thread are based on deduction from watching active chatters get suspended and tell their stories on Discord and social media.
Most can luckily get their account reinstated after appealing but it relies on having an actual human look at the timestamp of the VOD and take their time to figure out what actually happened as well as get the complete context of what was going on on stream when it occurred. I've seen many apologies from Twitch moderation sent in emails after appealing, but if you get unbanned, an apology or stay banned seems mostly random.
Being banned like this will also make it much less likely that you want to participate and joke around in chat in the future, leading to a much worse chatting experience.
I see some discussions are arguing that all AI flagged moderation events are actually reviewed by humans (but poorly) and this is a possibility. Because of the lack of transparency from Twitch regarding how this works it's very difficult to know for sure how these reviews are done. A manual report in combination with an AI flag is almost certainly a ban. One thing is sure though, and that is that too much power is given to AI to judge in these cases.
Seeing as permanent suspensions from accounts who have had active paying subscriptions for YEARS on Twitch can be dished out in seconds, either those reviewing are doing a lousy job, or its mostly done by AI. Even worse, if those reviewing are underpaid workers who get paid by "number of cases solved per hour" there is little incentive for them to take their time to gather more context when reviewing.
A similar thing happened on YouTube at the start of 2023, where they went through all old videos speech-to-text transcripts and issues strikes retroactively. It got a lot of old channels to disappear, especially those with hours of VOD content where something could get picked up and flagged by AI. For the communities I'm engaged in, it meant relying less on YouTube for saving Twitch VODs. It was brought up by MoistCritical about a year ago since it also affected monetization of old videos.
A more reasonable alternative to permanent bans on paying customers is global muting. That would also be an improvement.
-
Saysera commented
"Right now making mature jokes in chat can get chatters permanently suspended globally on Twitch completely without any human intervention that requires context."
"Outsourcing moderation to AI, while it may save money, is not a good idea."
For the record, this is not the case at all.
Every single suspension on twitch is done by a human moderator either directly employed by twitch, or an employee of a 3rd party firm twitch pays to help them moderate the site (they've mentioned it in their moderation transparency reports fyi)
So yeah literally no magic "rogue AI that perma bans people"This user voice post should be deleted as it's pointless (asking to remove something that isn't there).
The fact that some people are so confidently wrong is crazy.
-
RandomRed38 commented
I watched a streamer playing "Super Seducer" recently. Some of the things we, chat, were typing, OMFG, they look really bad out of context. But if you know what the streamer just said, and what was on screen, and what the character just said, it's perfectly understandable. But an automod would absolutely slay that person
-
xLiamLiamx commented
This sounds like a really bad idea to let AI moderate to this extent...
-
ZenicReverie commented
Free Shoepert!
AI auto-mod should be opt-in or channel owner/mods should be allowed to overturn/petition the auto-modding.
-
nofalberg commented
Wow, this needs more attention, I know many people who got suspended because of this AI's sensitivity and Twitch needs to do something about it, like multiple accounts are being banned while they did not mean any serious violation. Dumb idea, Twitch. Be better.
-
Never_trust_a_cat commented
There's value in the AI mod, but it should be channel-based bans and it should be opt-in or opt-out. There should be a way to turn off the AI mod. The AI mod doesn't know the context, it doesn't know what the steamer said. It is incredibly stupid to ban an account for a joke with no malicious intent.
People who have been banned say that they weren't given a clear indication of what exactly they were being banned for. So if you're too afraid to show people exactly what they're being banned for by the AI mod, why does the AI Mod have the power to ban accounts site-wide in the first place?
-
JabroniDogBoy commented
the jabroni_mike chat will turn into a desert if this goes through
-
IreneSmith13 commented
I certainly agree with the conclusion here. There is no way for AI to accurately discern what is inappropriate, based on one or two sentences. I think the responsibility for moderating that type of content should rest solely with the channel's moderators. If nothing else, what appears to be improper posting should be brought to the attention of the channel before anything permanent is done.
-
Barnacules commented
Wow, this is not okay! I didn't realize that Twitch had a global auto mod in every channel that could take private action against viewers without the channel owner being notified. This would explain why some people have just disappeared and never came back in the middle of a conversation about something that could be seen as hitting one of these lack of context moments discussing important topics.
I will share this with my audience and ask them to upvote this so hopefully it gets Twitches attention. However, since it's a uservoice page and not a Twitch official page I doubt they will listen regardless of the number of signatures unless people start bombarding them on social media and official support channels.
If you have a manager for your partnered Twitch channel then please contact them and let them know that this isn't okay. If they feel something like this needs to exist then it needs to have humans at the helm looking at the video and making the call on context or at the very least employ a GPT trained on multiple lingual contexts that takes the closed captions said around the time of the chat the message and compares it in that reference frame with what was posted in chat to make the final call.
-
theodoros_1234_ commented
This is a terrible way to moderate the platform. The AI is missing a lot of important context, and is 100% going to wrongfully ban many people. Twitch, please let the moderators of each channel do their jobs.
-
MrBuga12 commented
this issue just happened to me, I am a fairly small creator with only 400 followers, but when I got an email explaining that my account's chat connection was "automatically temporarily flagged to prevent abuse" I logged into the appeal portal, but there was nothing to appeal, so I am at a loss for what my next steps could be.
-
cruzk0ntrol commented
doubt twitch will even look at this but I hope they do
-
Kinsi55 commented
I'd love to be as blunt as possible about my feelings towards this "feature" but I wont because I'm not a fan of getting banned for words not directed at any individual or group so you'll have to use your imagination: This is utterly redacted
-
ZexMaxwell commented
A global "bot" would be nice. but a Global "AI" is not the answer. the tech is NOT at the stage where it can be reliable. assuming "reliable" is defined as an overly aggressive fail safe that bans people regardless of language nuance.
AI is STILL too far out for something to be helpful. we are talking tech that is at least 5 to 10 years in dev time. assuming we even have the Video cards to even process this data. even IF a company has this processing power to do this task, its going to cost way too much to do it now. A flagging bot that is managed by twitch staff is the current answer for what you need.
-
Metalus69 commented
This website continues to make the most out of touch business decisions, it's almost like they want to kill the site off. If you want to make money, this is not how you do it.
-
Cragsand commented
Here it is:
"Spirit AI" using "proactive" technology. I fear this has backfired.
Quote from 2022 blog post:"Fortifying the technology that detects harmful text of all kinds on Twitch.
We recently completed the acquisition of Spirit AI, a leading natural language processing company who will help us continue to refine AutoMod and other proactive detection for catching harmful text or phrases sent on Twitch."
https://safety.twitch.tv/s/article/An-Update-on-Our-Work-to-Stamp-out-Targeted-Hate?language=en_US