Call of Duty enlists AI to eavesdrop on voice chat and help ban toxic players starting today

captain price
(Image credit: Activision Blizzard)

Call of Duty is joining the growing number of online games combatting toxicity by listening to in-game voice chat, and it's using AI to assist the process. Activision announced a partnership with AI outfit Modulate to integrate its proprietary voice moderation tool—ToxMod—into Modern Warfare 2, Warzone 2, and the upcoming Modern Warfare 3.

Activision says ToxMod, which begins beta testing in North American servers today, is able to "identify in real-time and enforce against toxic speech—including hate speech, discriminatory language, harassment and more."

Modulate describes ToxMod as "the only proactive voice chat moderation solution purpose-built for games." While the official website lists a few games ToxMod is already being used in (mostly small VR games like Rec Room), Call of Duty's hundreds of thousands of daily players will likely represent the largest deployment of the tool to date.

Call of Duty's ToxMod AI will not have free rein to issue player bans. A voice chat moderation Q&A published today specifies that the AI's only job is to observe and report, not punish.

"Call of Duty’s Voice Chat Moderation system only submits reports about toxic behavior, categorized by its type of behavior and a rated level of severity based on an evolving model," the answer reads. "Activision determines how it will enforce voice chat moderation violations."

So while voice chat complaints against you will, in theory, be judged by a human before any action is taken, ToxMod looks at more than just keywords when flagging potential offenses. Modulate says its tool is unique for its ability to analyze tone and intent in speech to determine what is and isn't toxic. If you're naturally curious how that's achieved, you won't find a crystal-clear answer but you will find a lot of impressive-sounding claims (as we're used to from AI companies). 

modulate toxmod

(Image credit: Modulate)

The company says its language model has put in the hours listening to speech from people with a variety of backgrounds and can accurately distinguish between malice and friendly riffing. Interestingly, Modulate's ethics policy states ToxMod "does not detect or identify the ethnicity of individual speakers," but it does "listen to conversational cues to determine how others in the conversation are reacting to the use of [certain] terms."

Terms like the n-word: "While the n-word is typically considered a vile slur, many players who identify as black or brown have reclaimed it and use it positively within their communities… If someone says the n-word and clearly offends others in the chat, that will be rated much more severely than what appears to be reclaimed usage that is incorporated naturally into a conversation."

Modulate also offers the example of harmful speech toward kids. "For instance, if we detect a prepubescent speaker in a chat, we might rate certain kinds of offenses more severely due to the risk to the child," the site reads.

In recent months, ToxMod's flagging categories have gotten even more granular. In June, Modulate introduced a "violent radicalization" category to its voice chat moderation that can flag "terms and phrases relating to white supremacist groups, radicalization, and extremism—in real-time."

The list of what ToxMod claims to be detecting here includes:

  • Promotion or sharing ideology
  • Recruitment or convincing others to join a group or movement
  • Targeted grooming or convincing vulnerable individuals (ie, children and teens) to join a group or movement
  • Planning violent actions or actively planning to commit physical violence

"Using research from groups like ADL, studies like the one conducted by NYU, current thought leadership, and conversations with folks in the gaming industry," says the company, "we've developed the category to identify signals that have a high correlation with extremist movements, even if the language itself isn't violent. (For example, 'let’s take this to Discord' could be innocent, or it could be a recruiting tactic.)"

Modulate is clearly setting its goals high, though for Call of Duty's purposes, it sounds like ToxMod will simply be the middleman between potential offenders and a human moderation team. While the machinations of AI decision-making are inherently vague, Activision says its enforcement will ultimately abide by Call of Duty's official Code of Conduct. That's not dissimilar to how Riot and Blizzard have been handling voice chat moderation in Valorant and Overwatch 2, though Riot has also been gathering voice chat data for over a year to develop its own AI language model.

ToxMod will roll out worldwide in Call of Duty at the launch of Modern Warfare 3 on November 10, starting with English-only moderation and expanding to more languages at a later date.

Morgan Park
Staff Writer

Morgan has been writing for PC Gamer since 2018, first as a freelancer and currently as a staff writer. He has also appeared on Polygon, Kotaku, Fanbyte, and PCGamesN. Before freelancing, he spent most of high school and all of college writing at small gaming sites that didn't pay him. He's very happy to have a real job now. Morgan is a beat writer following the latest and greatest shooters and the communities that play them. He also writes general news, reviews, features, the occasional guide, and bad jokes in Slack. Twist his arm, and he'll even write about a boring strategy game. Please don't, though.

Read more
A zombie santa with six fingers leaps at the screen.
Call of Duty admits it's using generative AI to 'help develop some in-game assets', and suddenly all those poorly made calling cards make sense
Black Ops 6
Call of Duty devs admit Ricochet anticheat 'did not hit the mark for integration' at the start of Season 1, and want to do better
LoL summoner art
It's high time League of Legends got full voice chat
Razer Project Ava key visual
Razer has released a backseat gaming AI bot called Ava, and I'm not sure whether using it should be considered cheating or not
A young Asian woman opening visual aids to give her audience a better understanding while holding a podcast session.
Logitech has announced an 'intelligent streaming assistant' in Streamlabs to tell you when your live stream sucks
black ops 6 season 1
Call of Duty: Black Ops 6 has now banned 136,000 accounts as part of the ongoing struggle to ensure fair play but still says that IP banning isn't an option
Latest in Call of Duty
A soldier looks out over the Verdansk map, as a single tear rolls down his cheek.
The original Verdansk map is returning to Call of Duty: Warzone, to celebrate which we get a soldier crying to Nat King Cole
black ops 6 season 1
Call of Duty: Black Ops 6 Season 3 has been delayed, as the devs say they're 'taking the time to deliver a great experience' for what will be a 'big moment' for Call of Duty
A zombie santa with six fingers leaps at the screen.
Call of Duty admits it's using generative AI to 'help develop some in-game assets', and suddenly all those poorly made calling cards make sense
The Teenage Mutant Ninja Turtles in Black Ops 6.
Call of Duty's Teenage Mutant Ninja Turtles crossover costs like $90 and even the die-hards are in shellshock: 'Cash cow-abunga!'
Ghost, from Call of Duty: Modern Warfare 2 (2022), looks bleakly at a fellow passenger in a transport.
For COD’s sake: One player’s 763-day legal quest to make Activision unban their account ends in total success: ‘Worth the effort’
Call of Duty: Black Ops 6 Season 2
Call of Duty: Black Ops 6 Season 2 will let players battle on boats and bullet-trains, with the Terminator entering the fray 'shortly after launch'
Latest in News
Key art of Kent Paul in Grand Theft Auto Vice City.
'My own voice was driving me f***ing insane': GTA Vice City actor admits even he couldn't get past his own, notoriously difficult mission
Nvidia RTX 4060 Ti graphics card
Specs for Nvidia's new RTX 5050, 5060, and 5060 Ti GPUs leak out and that 5060 might actually be half decent. If it's priced right
Pipboy holds up an open padlock.
A BIOS update could be all that's stopping you or someone else from jailbreaking your old AMD CPU
Asus's new ultrawide sucks as hard as it blows
Asus' new monitors purify 90% of airborne dust from your desktop and I've definitely seen some gnarly gaming setups that would benefit
A screenshot from Sony's PlayStation 5 Pro announcement video, showing a stylized processor against a dark background with glowing lines streaming from its edges
The AMD x Sony collab gave us FSR4 and a version will appear in PlayStation next year, too, having 'already started to implement the new neural network on PS5 Pro'
Pedro Pascal as Joel in a coat in winter looking unhappy
'Don't you know what he did?': The truth comes out in The Last of Us Season 2 trailer