Storing trillions of your Discord messages is waaaaay harder than you thought

Discord forum channels
(Image credit: Discord)

In this age of utterly ubiquitous connectivity, it's easy to forget that enabling it all is actually, you know, hard. Enter Discord, the voice, video and text chat app, and its tale of toil when it comes to storing and retrieving the trillions of messages its users generate.

The specifics of the company's struggles have been outlined in forensic detail in a blog by Discord's Bo Ingram, a senior software engineer for the platform. 

On big public Discord servers, most of the the displayed messages are recent and cached which is good for performance. But smaller servers used by just a few friends can often involve old messages that aren't cached being displayed or read from the database every time a user logs in.

According to numbers from 2019, Discord handled 25 billion messages per month.

According to numbers from 2019, Discord handled something like 25 billion messages per month, 850 million every day, and 600,000 per minute. You can expect those numbers to be even bigger now.

Discord started off using the MongoDB database to support this activity, but then moved to Cassandra. Its appeal was as a NoSQL database that had support for clustering, which means you can have multiple instances of a given database working as one to improve performance.

Both Apple and Netflix, among other big tech companies, both reportedly use Cassandra. So, it's got proper blue chip cred. Surely it's good enough for Discord? Initially, yes. Once Discord had ironed out a few issues, the Cassandra databases were writing new messages within one millisecond and read back old messages to users in five milliseconds. Quick enough for your typical text chat, that's for sure.

But the good times didn't last. Discord now has something in the region of 150 million monthly active users. By early 2022, Discord was running 177 Cassandra nodes storing trillions of messages.

The result was hotspots within the database when users interacted with Discord servers in certain patterns. The upshot is latency as the server falls ever further behind with user queries—in other words, users sending and reading text messages.

Discord DB

The NoSQL bone is connected to the Channel ID bone. (Image credit: Discord)

The way that Cassandra marks data for deletion before eventually removing it via garbage collection routines also added dramatically to latency when reading messages. In really rough layman's terms, the tombstones marking the location of data marked for eventual deletion slowed down the process of getting at actual live data for reading.

The solution for Discord is another extremely difficult migration to a new database, known as ScyllaDB. It has several advangtages, including being written in C++, which is much faster than the Java coding language used by Cassandra. It's also Cassandra-compatible and also deletes data directly rather than using a garbage collector.

Your next upgrade

(Image credit: Future)

Best CPU for gaming: The top chips from Intel and AMD
Best gaming motherboard: The right boards
Best graphics card: Your perfect pixel-pusher awaits
Best SSD for gaming: Get into the game ahead of the rest

Anyway, the switch over to ScyllaDB happened in May last year, and no doubt you didn't even notice when it happened. Which is a good thing. According to Bo, latencies are much improved compared to the tail end of the Cassandra implementation.

"It's been a quiet, well-behaved database (it's okay to say this because I'm not on-call this week). We're not having weekend-long firefights, nor are we juggling nodes in the cluster to attempt to preserve uptime. It's a much more efficient database—we're going from running 177 Cassandra nodes to just 72 ScyllaDB nodes," he says.

"Our tail latencies have also improved drastically. For example, fetching historical messages had a p99 of between 40-125ms on Cassandra, with ScyllaDB having a nice and chill 15ms p99 latency, and message insert performance going from 5-70ms p99 on Cassandra, to a steady 5ms p99 on ScyllaDB."

So, there you have it, the slightly-more-complicated-than-you-might-have-thought story of how to keep a message service from collapsing under the weight of millions of users. You can read about it all in much more detail over on the official Discord blog.

Jeremy Laird
Hardware writer

Jeremy has been writing about technology and PCs since the 90nm Netburst era (Google it!) and enjoys nothing more than a serious dissertation on the finer points of monitor input lag and overshoot followed by a forensic examination of advanced lithography. Or maybe he just likes machines that go “ping!” He also has a thing for tennis and cars.

Read more
discord
Brace yourself for Discord to get worse: Reports swirl that the company is in talks with bankers about opening itself up to shareholders
The new TeamSpeak overhaul.
TeamSpeak could finally be so back, since it can't stop teasing a new makeover while sneakily encouraging you to delete your Discord account
Discord Social SDK
Your Discord friends list may soon appear directly in the games you play
DeepSeek
Today I learned I can run my very own DeepSeek R1 chatbot on just $6,000 of PC hardware and no megabucks Nvidia GPUs required
New Discord desktop client themes.
Discord drops big update with 'completely new' in-game overlay and new dark themes for the desktop client
AMD Epyc 4th gen server processor on motherboard
AMD sees record revenue of $25.8 billion in 2024 thanks to data center growth—gaming last seen tumbling into a ditch
Latest in Platforms
A screenshot from game Mudborne of a little humanoid frog in a marsh
Five new Steam games you probably missed (March 24, 2025)
midnight murder club
Five new Steam games you probably missed (March 17, 2025)
Screenshot of Children of Clay showing a mysterious clay model
Five new Steam games you probably missed (March 10, 2025)
discord
Brace yourself for Discord to get worse: Reports swirl that the company is in talks with bankers about opening itself up to shareholders
The Spy from Team Fortress 2 holds up a folder with an accusatory expression.
Steam users react ecstatically to update that lets them access their heaving game notes via the web, also it fixes Monster Hunter Wilds video recording
HasanAbi
Twitch streamer Hasan Piker suspended after saying Republicans would 'kill Rick Scott' if they really cared about Medicare fraud
Latest in News
A long bendy arm stealing money from people in a subway car
'You're a very long arm. You steal things. It's a comedy game,' explains developer of comedy game where you steal things with a very long arm
The heroes are attacked by monsters
Pillars of Eternity is getting turn-based combat to mark its 10th anniversary, and that means PC Gamer editors will soon be arguing about combat mechanics again
Image of Ronaldo from Fatal Fury: City of the Wolves trailer
It doesn't really make sense that soccer star Ronaldo is now a Fatal Fury character, but if you follow the money you can see how it happened
Junah beginning a battle in Metaphor: ReFantazio.
Today's RPG fans are 'very sensitive to feeling like they wasted time' when they die, says Metaphor: ReFantazio battle planner—but Atlus still made combat hard anyway
Image of Cersei Lanniser from Game of Thrones: Kingsroad Steam early access trailer
A new Game of Thrones RPG is coming to Steam today with a cast of 'familiar faces,' which is good because it's really the only way to tell it's a GoT game at all
The new Prime Asset featured in the upcoming update for the Outlast Trials.
The Outlast Trials puts its already paranoid players under surveillance for a time-limited story event