Massive Entertainment's The Division 2 launched in 2019 and, while it's not the type of game that makes headlines every week, it has since tootled along quite nicely thanks to regular seasonal updates, and the promise of more to come from the series. The most recent update, however, has been a disaster on a scale that is still not yet fully clear: Perhaps even to the developer.
In a perfect encapsulation of the butterfly effect in coding, the newest season had been delayed due to a localization issue. "This past Saturday, in the process of creating the update which would resolve the issue, we encountered an error that brought down the build generation system for the Division 2," said the studio in a statement posted to Twitter. "As a result, we cannot update the game until this system has been rebuilt."
You can see how bad the problem is by the fact that Massive has been unable to simply extend the current season's activities—something the studio has previously done in response to a new season's delay. "Unfortunately this is not possible in the current situation, as we are unable to make server or client side updates until the build generation system is restored."
- 'We made one big mistake': Destiny 2 developer reveals how a small team dedicated to player retention led to a 20 hour server outage and character rollback
- Path of Exile fans are having a very normal one after an expansion was delayed thanks to Path of Exile 2: 'I need to make peace with myself and let go of this burden'
That is, in trying to fix a problem with a new update, the entire system for updating the game has keeled over. So now Massive has to fix the system that updates the game before it can update the game to fix the other problems.
Question being, what is a build generation system and just how bad can this be? I spoke to a senior industry coder who explained that this is potentially really bad. Like, cats and dogs living together bad. A Build System is a bunch of machines and scripts that take the source code and assets and turn it into something that will run on a client machine, like a console.
A message from #TheDivision2 team. pic.twitter.com/KuPiz7t9PLFebruary 9, 2023
One problem could be hardware failure, "which isn't a major issue. From what they have said though, it doesn't sound like that is the case, it sounds like they're saying they've had to rebuild the build system from scratch."
The surprising thing, from our insider's perspective, is that either there wasn't a failsafe in place, or the failsafe has failed. "A hardware failure should be anticipated with a recovery plan ready to go. Rebuilding a build system from scratch is something you schedule and plan around, you maintain the existing one and switch over seamlessly. Something very unusual has happened and it sounds like they are much less prepared than they needed to be."
This could be down to underestimating the knowledge, skills and above all experience needed to maintain a live code base, because Massive's main focus is arguably elsewhere on upcoming Division titles.
"I think the industry as a whole still has a lot to learn about operating these living code bases & it sounds like The Division 2 team has just had to learn one lesson the hard way, I hope they get it sorted with minimal disruption & hopefully it's seen as an opportunity to grow the knowledge base of the team and not a stick to beat the team with."
To be clear, the Division 2 is still up-and-running, for now at least. Over on the subreddit the community has taken the news with a dose of black humour, as well as some gratitude that at least Massive is being transparent about what's going on and what to expect. Unfortunately this means current issues such as a crash bug when returning to the DC location remain a problem, with some players permanently locked out of returning to the game's central location until a fix is applied.
"We are working to resolve the situation as quickly as possible," said Massive. "We have made good progress over the last 96 hours and have been able to restore critical aspects of the system and will share the timeline for resolution when available."
It promises another update later today and, when the issue is fixed (fingers crossed!), some form of in-game compensation. Until then, the agents of the Division face a problem too insurmountable for even them.
"I knew it was fucked when they didn’t immediately roll back and extend the season," said player HotSpicedChai. "But I never expected it to be this fucked."
