So it’s that time again. That time when a major game launch goes completely in the weeds due to a required online mode, and major server issues befall the game upon launch. SimCity is only the latest example of a phenomenon which is becoming increasingly common, and was well-showcased by Diablo III last year.
In the past, this phenomenon was typically restricted to MMO games, which are understood to have large and complex server backends and require a connection to play. People were understandably pissed in the communities for those games, but the level of vitriol in the Internet at large was typically restricted. The reason for this is setting of expectations: we know that this is the case with an MMO. In an MMO, I’m never alone… even when I don’t see another player character on screen, I see other players chatting in the General or Trade channels.
But Diablo and SimCity have a different problem – namely that we have an expectation formed from the previous games in their respective franchises. In both cases, the previous games had a very rich single-player experiences, which could be completely enjoyed without being online. However, in both cases, the designers of the new iterations completely focused on them being online experiences. In fact, in SimCity, we’ve already seen some interesting cases of what I’d call “emergent griefing” show up in the press. Could Diablo III or SimCity be enjoyed as a single-player experience? Surely. The previous titles in their series show this. However, both titles chose social interaction as a main focus to expand upon the previous iterations, Diablo by adding features such as join-in-progress, and SimCity by adding the entire multiplayer component.
One of the things that always bugs me when one of these happens is the assertion that the the cause of the outage is “always-on DRM”. Would you say that World of Warcraft has always-on DRM? It does, by the definition used to complain about Diablo III or SimCity, but you (probably) won’t ever see that phrase in connection with that game. The phrase implies both that the primary purpose of the required internet connection is rights management, and that the game could be entirely enjoyed without being online. While the latter point is debatable since these titles may not be creating enough value from their connectivity, I don’t think the former is.
Hanlon’s Razor states:
Never attribute to malice that which is adequately explained by stupidity.
A corollary to this is to not attribute to malice that which is adequately explained by laziness either – and the problems plaguing SimCity players now can be most easily attributed to laziness.
Programmers are lazy. We have to be. Otherwise, projects would never get finished. Software is the most complex endeavor of humanity. The number of lines of code which go into a AAA title like SimCity far outweighs the number of components in the most complex of automobiles, especially once you add in every dependency taken (DirectX, Windows, Visual C++ Runtime, etc. etc.)… and that doesn’t even touch on the hardware. To help us create such monstrous feats of engineering, we have to choose the tools which we believe are best suited for the job.
For the game client, a native-compiled C++ application was used, which is very proven tech and the best (some would say only) choice for high-performance AAA games. For the game server, however, there were a ton of choices the SimCity team had. Where do they host their servers – on premises? In “the cloud”? Which cloud? Do they code in Java? Ruby? .NET? node.js? Where do the store their data? Oracle? MS SQL? MongoDB? DynamoDB? MySQL? Windows Azure?
I have no inside information about what the SimCity chose to use as their technical solution for their server backend, but I’d bet dollars to doughnuts that the code wasn’t something which could be easily moved over to their game client. And even if the backend code could be, it’s even less likely that the data store could be moved over trivially. And then there’s the data conflict resolution aspects to consider.
So, as an engineer, when the game designer comes to you and says “we need a system to generate new cities, with 500 parameters which we can tweak whenever we want”, you build it into the service, since the service can have its code and parameters tweaked easily for all users at once, and it runs in a much more trusted environment. You can’t trivially port this back over to the client code. You might, as an engineer, express a concern about this, but the reality at this point in the project is that the server has already been made required both by fiat and by previous engineering decisions. You can either make a major engineering effort to fix this, or you can work on say… the firetruck AI (and 5 other features besides).
What percentage of the time do you think the producer is going to choose new features vs. making existing, working features work in an offline mode which relatively few of your players are likely to use (provided your service is working as planned)?
Now, I’m not trying to give the SimCity team a free pass here. Their service infrastructure is clearly not handling the load which their release generated. If the service had come through all of this with flying colors, there wouldn’t be nearly the amount of complaining going on about the required server connection. I’d be building a virtual city instead of writing this blog post. When building a service, it’s always important to remember who owns your availability. That’s not to say that anything about it is easy. I’ve built services for Zune, for Halo, for other games on Xbox 360, Windows Phone, and Facebook. They are hard. Scale is incredibly difficult to predict and build for. Estimating the flow of data into/out of data centers, between servers, to and from services which you depend on – all of these are very hard problems. “The Cloud” doesn’t solve any of this automatically, especially at large scales. Somethings things don’t work as they did in QA, sometimes you get DoS attacks, sometimes you self-DoS because you forgot to put back-off code in the client. Sometimes you have to take drastic measures to try to fix it.
It’s fine to be upset that you can’t play the game you bought. Just don’t call it “always-on DRM” as a knee-jerk reaction. They’re almost certainly not doing it to stop people from pirating the game – they’re doing it to give you the best game to play. The fact that it’s a particularly effective form of rights management is a (probably classified as “happy” [unfortunately]) accident of the engineering “laziness” on the project.
Disclosures: At the time of this writing, I worked on Valve’s Steam which is a competitor to EA’s Origin. All of the views expressed here are my own and do not represent the views of Valve. Steam gets its own set of DRM-related vitriol at times, and does not carry SimCity. At the time of this writing, I had not played SimCity, but I did play it once the server problems died down and found it to be an adequate but not phenomenal city builder.
