Global AI AutoMod does not understand context
Right now making mature jokes in chat can get chatters permanently suspended globally on Twitch completely without any human intervention that requires context.
Clarification:
This is not about regular AutoMod but the HIDDEN global AI AutoMod present in all of Twitch channels.
The AI based AutoMod Twitch currently uses everywhere does not understand important contexts and takes all text at face value. Streamers who have a strong focus on the LGBTQ community find themselves having to tell their chatters not to express themselves freely, because what they joke about can be seen as hate speech even though they are strong supporters and allies. For moderators it seems they are not allowed to do what they are supposed to do, which is to determine what is appropriate in their streamers chats.
Anything of the following in chat currently gets flagged by AI based AutoMod:
- Hints at self harm, even if clearly a reaction to a video game or made as a joke.
- Threat of violence even if clearly in-game or as a joke. The context of the game being played may portray or make the same joke that is being commented on.
- Jokes towards a protected group, even if the person who is making the joke belongs to that protected group themselves.
- Sexual advances, even if clearly consensual, requested or welcomed by the streamer.
Right now it seems anything of the above in combination with a manual report will get your account permanently suspended. Partnered and mods for the channel it was flagged in appear to get more leniency.
While it is Twitch responsibility to protect all members of their community the current AI has become overly sensitive, especially for streamers who focus on mature themes and stream games that contain mature themes.
As long as streamers have their content classification set appropriately, should that not also extend to Twitch chat?
I suggest allowing "mature content" streamers to:
- Turn off the global AI based AutoMod for their chat.
- Reduce global AI based AutoMod sensitivity for their chat. (at the very least)
As long as they have set their stream content classification properly to:
Mature-rated games; Sexual themes; Drugs, Intoxication, or Excessive Tobacco Use; Violent and Graphic Depictions; Significant Profanity or Vulgarity; or Gambling. https://safety.twitch.tv/s/article/Content-Classification-Guidelines?language=en_US
Being able to discuss or joke about topics on screen of a sensitive nature is no longer possible in Twitch chat and streamers are seeing their most active chatters and supporters get permanently suspended for participating in completely consensual conversations.
I hope this can lead to further discussion regarding the matter.
Outsourcing moderation to AI, while it may save money, is not a good idea.
Hi, thanks for taking the time to share your feedback with us. We wanted to clarify a couple of things. AutoMod can’t ban people from Twitch. Mods can use the tool to identify chat messages that break their channel’s rules. But AutoMod can’t be used to timeout, ban, or mute users from any channel, or from Twitch – those aren’t AutoMod capabilities.
We also wanted to share a bit more about how we enforce our Community Guidelines. These are the policies that apply to all of Twitch, and help make Twitch safer. We have human teams that review our content moderation decisions. We call this “human in the loop.” These reviewers help ensure that our policies are being applied accurately. We don’t use a “global AI” system to enforce our guidelines, and we manually review suspension decisions.
We recognize that, while the majority of enforcements we issue are correct, sometimes we may get it wrong. If you feel an action taken against your account was not warranted, we recommend submitting an appeal through our appeals portal for our teams to review.
Again, thanks for sharing your experience. It’s important that we’re open about how we enforce our policies. It’s also on us to ensure that our enforcement decisions are accurate. Our teams issue and review content moderation decisions in real time for that reason.
-
Lazzie
commented
The ai system has fully ruined alot chat experinces for myself and communties im around. My friends are afraid to banter joke or anytihng
-
vocaab
commented
Let streamers turn this off because clearly its not working correctly.
-
TangoBunny
commented
Completely agreed. People getting suspended indefinitely for something an automod _mistook_ as violating any terms of conditions is ridiculous enough, but the fact that the support site claims you have to provide "wait six months then provide an acknowledgement that you violated the Community Guidelines" is an absolute insult.
If an AI or member of Twitch Staff got something wrong, the concept of being punished and forced to apologise for something they never actually did is completely immoral and cruel.
-
FrommDK
commented
I absolutely agree, there is nothing wrong with using AI to flag content for review, but letting AI or algorithms have a final say on suspensions with no human oversight is insanity, I expect better of twitch and am deeply troubled with the idea that I could get suspended from interacting with my favorite streamers for saying something completely innocent, Do better twitch...
-
kdragon
commented
I haven't gotten banned (as of yet) but seeing other people getting banned for simply repeating exactly what the streamer just said but in the chat is absurd. As someone who enjoys chatting, the specter of AI that might ban me at any time for (often inexplicable) reasons has definitely curtailed both my enjoyment and sometimes my willingness to engage.
-
GrimRiver
commented
Using AI as a replacement for human moderation is probably the worst idea ever conceived and doesn't NOT understand context.
I seen so many people get wrongfully banned without cause or reason just to be unbanned an hour later and twitch's response was "oops lol" while others get permanently banned for making lighthearted jokes with the streamer, when there has been people that has literal sex on stream or dangerously waved firearms around get only a 3 day ban. Actually unprofessional on twitch's part.
Remove global AI moderation or give an option to turn it off, to be honest there is no reason for it anyway when the streamer can ban the offender or have actual living people moderate the chat not to mention channels have access to auto mod. So in reality global auto mod is rendered utterly irrelevant and more of a hindrance than anything. Just remove it.
-
cyb3rofficial
commented
+1 definitely should be considered to have global stuff tuned down or off per streamer basis, opt in by default, but should be able to opt out and have some of the responsibility placed on the streamer as compensation for failure to moderate. Thats why most streamers have mods, to take out the trash in chat. We have tools in the mod panel to report such people in chat too, I can not see any negatives other than lazy moderation from staff.
-
Livvydoodlez
commented
AI should never be used for moderation. It can easily be used to pick up false flags. User's accounts should not be jeopardized from false flags being made. Context matters. Using AI to make judgement calls, which it literally cannot do, is unhealthy for the chatting experience. All chats should be up to the streamer's, and their moderators decisions.
Any bans globally, should still be reviewed, and processed, by a person, and used only for EXTREME ToS violations.
-
Zalathar
commented
Any time I want to discuss the game being played, I first have to ask myself “is a bot going to misinterpret this legitimate good-faith comment as violent hate speech”?
Because I’ve seen it happen to other people, and once it happens, you’re done. Instant site-wide indefinite ban for a first-time “offence”, with no meaningful review or appeal process (despite Twitch’s claims to the contrary).
Sadly, I’ve seen no indication that Twitch cares about the harmful effects this is having on individuals and communities. They seem to care more about people who actually utter real hate speech (and then wait six months) than about people who have been incorrectly banned.
-
DanaCrysalis
commented
I mod for a number of channels, and the AI has gotten more than just a few people banned for months or more, there is currently another moderator who is on "indefinite suspension" because he made a comment that was almost repeating exactly what the streamer had said, IN CONTEXT, it made perfect non-hateful sense. But the AI doesn't care about context, and it doesn't care about it being a moderator with a 12-year old account with 8000+ hours streamed and far more than that watched. And this would even be forgivable, if the appeal process weren't ALSO completely nonsense and instantly denied, possibly by AI as well (but who knows, there's no transparency so...). The streamer even attempted to contact Twitch to get him unbanned as well, and nothing! #freeshoepert
-
IronDJT
commented
This is very true. People abuse the AI to report people they don't like out of context and get them banned. Twitch really needs a better way to go about global bans. The current system is very exploitable.
-
ChillaxingJay
commented
Cragsand is completely right. I've seen too many Twitch chatters, many of whom are my friends, get their accounts permanently suspended over joke comments that are taken out of context. The fact they get their accounts banned so easily and sometimes even get their ban appeal denied is ridiculous. Something should definitely be done about how sensitive the AI AutoMod is for chat rooms, especially for mature content streamers.
-
klaatu1
commented
I was personally banned after a joke comment I made in a friend's channel, even as a moderator Twitch indefinitely suspended my account. Many users I am either personally friends with, or adjacent to have also been affected by this issue with an increased frequency over the last 18 months. This has permanently changed the scope of Twitch chat & forced people to participate in communities they've been a part of for years with an unprecedented & highly unnecessary level of caution.
-
Cragsand
commented
@LogicWeasel Thank you! All comments and direct accounts of how this affects more people help spread awareness of the problem.
-
LogicWeasel
commented
Cragsand eloquently presents how many chat communities feel right now: that twitch's AI automod creates a hostile chat environment that feels like a paranoid secret police that bans far too many good chatters and cannot tell jokes from real hostility online. It goes against what the streamer and their mods want and doesn't even give a channel the authority to permit the chatter in context as instead a chatter is banned silently.