Epic Games Primer (Pt IV): Epic Online Services
This essay contains the fourth portion of the six-part ‘Epic Games Primer’ with Jacob Navok, which covers every element of the Epic Games Flywheel, as well as the company’s aspirations. Specifically, this entry walks through Epic Online Services.
The ‘Epic Games Primer’, Part 4: Epic Online Services (2019-)
What Are Online Services?
For the first few decades of gaming, revenue was generated either on a pay-per-use basis (as was the case in arcades) or a pay-per-copy (e.g. buying Mario Kart 64). In this latter case, it didn’t much matter whether a player played 5 or 500 hours. What mattered was the $60 upfront fee. Even when a game had a strong online experience, such as Age of Empires or StarCraft, revenue was almost entirely (if not entirely) from one-time, upfront fees.
Today, the biggest games typically generate the bulk of their revenue from ongoing operations. There might still be an upfront fee, but the goal is to continue generating revenue from ongoing play. This might be through season passes, downloadable content updates, micro-transactions for extra lives or outfits, or ad impressions.
Online, live games require a myriad of different services to work and produce compelling experiences for users. This includes account management, authentication, and security; entitlements; achievements/leaderboards/stats; player data storage; player-to-player communications; social networking and friend-lists; matchmaking; player reporting; game analytics; version management; and so on.
The Online Services Landscape
Just as most game designers use third-party engines for their games, most also use third-party online/multiplayer services providers. This reflects the cost of building these services, the difficulty of operating them reliably, and the fact that the major platforms use such services to differentiate their ecosystems.
The best example is likely Valve’s Steamworks, which is the most used set of game services in the world. Steamworks offers almost all of the services a publisher needs to run online games and supports millions of them across every conceivable PC configuration.
And crucially, all of Steamworks is free to both game makers and end consumers. However, Steam offers these services at no cost in order to attract both publishers and players to Steam and then keeps them there and away from alternatives. For example, the use of Steamworks meant that a game had to be played through Steam, even if it was bought through another digital store (like Amazon). This meant a player could never remove Steam, was constantly exposed to opportunities to buy more games on Steam, and would have little reason to keep buying games elsewhere. In addition, Steam would continue to collect more player data and fortify its player network, too.
There are other consequences, too. As an example, consider Steamworks’ Achievements services. This service enables a publisher to easily offer badges, trophies, and other digital items to players based on their in-game accomplishments, such as beating a game, winning a tournament, or being a day-one buyer. Although these achievements could sometimes be shifted from Steam to a game if a developer desired and built the requisite functionality, there was no way to take all (or even most) achievements across all games to another store.
In addition, Steamworks’ player networks and matchmaking services mean that if a player switches stores, they need to rebuild their entire friend and teams lists. This is true even if they moved only a single game (e.g. Call of Duty) or if the only purpose of Steam was to play that single game in the first place.
Similarly, we can consider the example of team-based communications. A “squad” of three players can open Steam and talk, then decide to play Call of Duty together and continue talking to one another. That’s great. But although this team can then play a fourth friend who plays Call of Duty via Activision Blizzard’s store, they can’t talk to him/her through Steam. Instead, they need to use a separate communications app like Apple’s FaceTime or Discord, or switch to the in-game livechat (which doesn’t use Steamworks and therefore requires the publisher to use another third-party provider or build live chat services themselves).
Like Steam, Microsoft used online multiplayer services to drive its platform. The company launched its Xbox Live service years before Sony’s PlayStation Plus, and in addition to offering Steamworks-like services for free to publishers, it even covered the most expensive parts of online multiplayer services: the servers that host the game itself. This technical decision also meant that Microsoft was able to offer Xbox players then-unprecedented reliabiliability. This mix of a first-mover advantage, free services suite, and technical excellence allowed Xbox to quickly become the preferred platform for multiplayer console gamers and thrive alongside Sony, which excelled in single-player offline gaming. What’s more, Microsoft built an enormous business around charging consumers for this service. Today, Xbox has some 90MM players paying $5–10 per month. Eventually, both PlayStation and Nintendo replicated this model.
At the same time, the console approach produced a number of limitations. For example, achievements were still locked to the platform they were generated on. If a Call of Duty player switched from PlayStation 4 to Xbox One, they would have to redo/re-acquire their badges and trophies (if still possible).
Most important, however, was the fact that matchmaking was also tied to a single platform. For all its limitations, Steam allowed players to play with non-Steam players (be they on PC or another device). However, even if the games were identical, a PlayStation player could only play PlayStation players; an Xbox player could only play Xbox players.
There were a few drivers of these constraints. Microsoft’s early decision to literally host all online gameplay rather than just offer a suite of à la carte services meant that its Xbox Live service wasn’t designed to support and integrate into other online player networks, for example. Still, Microsoft could have built the technologies required for cross-play online multiplayer over time. Alternatively, another third party could have enabled them.
However, the macro problem was that of platform incentives. Steam, for example, was the dominant PC store in large part because of the lock-in effects of Steamworks. Valve could sell this back-end infrastructure as a whitelabel services to third-party stores, but doing so would erode its network effects and facilitate losses in market share. And while it could build a for-profit business selling these services to non-PC games, console games were closed and couldn’t use them anyway.
Sony, meanwhile, had an enormous lead in the “console wars” and thus didn’t want to weaken its player network effects (if changing to another system meant losing your friend group, you’d be less likely to leave). To point, the president of Sony Interactive Entertainment admitted in 2016 that “the technical aspect could be the easiest” part of enabling cross-play and portable player data.
While Microsoft has openly supported cross-play for years, it resisted such a move back when its Xbox Live network led the marketplace. To this end, many analysts see Microsoft’s reversal as an active effort to deflate PlayStation 4’s dominance in the eighth console generation. If your network is weaker, there’s more to gain and less to lose from partnering with other ones; the stronger the leader was, the less they’d want things to change.
Terraforming Online Play
Although the forces keeping online multiplayer “closed” remained strong overall throughout the 2010s, the pressures to open up continued to grow. Microsoft, for example, began to integrate Xbox-based games with Windows-based ones, and even enabled cross-play with Steam for select games. The enormous growth of mobile gaming throughout the 2010s was also important since nearly every mobile title supported cross-play (unlike gaming consoles, few would ever buy a specific phone to play a specific game, and the major phone makers neither developed their own games nor licensed them exclusively anyway).
As a result, ubiquitous cross-play seemed to be more of a how/why/when rather than an “if.” The answer here was Fortnite, which was unique in several ways.
It was the first mainstream AAA game that could be played on any major game device, be that the PlayStation 4, PlayStation 3, Nintendo Switch, Mac, or iPhone. As a result, most players had multiple ways to access the game. In addition, Fortnite was free and online only. This mattered because by late 2018, each of Sony’s competitors had embraced Fortnite cross-play. As such, PlayStation didn’t just have the “worst” multiplayer version of Fortnite, but PlayStation owners had many alternatives at their fingertips.
Collectively, this meant that while Sony’s decision to gatekeep the Call of Duty multiplayer experience might have modestly reduced the number of $60 copies bought by gamers, a failure to participate in Fortnite’s online meant missing out on most revenue and driving players to its competitors. Worse still, Fortnite had become incredibly popular and was generating more revenue on a monthly basis than any game in Western history. And thus, by the end of 2018, PlayStation had embraced cross-play for Fortnite. And shortly thereafter, it enabled cross-play for many more games, too.
The history above explains how we ended up in a world where games can be played across multiple devices. One where multiplayer services, from accounts to trophies and achievements, with some exception, need not be tied to a platform.
However, this doesn’t mean it’s easy or cheap for a developer to support such flexibility. In fact, it means they need an alternative solution to Xbox Live or Steamworks. After all, cross-play, by definition, means that online services/operations can’t be siloed or centralized in a single platform.
To this end, Epic had to build these services themselves. And it had to do so at unprecedented scale. In April of 2020 alone, Fortnite amassed more than 3.2B hours of playtime. This compares to “only” 2B for Minecraft and Roblox, and 2.4B hours spent playing Call of Duty: Modern Warfare and Call of Duty: Warzone from October through April. In addition, the popularity of Fortnite meant that Epic was able to create one of the world’s largest social networks/account systems, spanning some 350MM users and 2.3B total connections.
The Epic Online Services Strategy
By mid-2019, Epic began beta-testing a version of its Fortnite services for outside developers.
This meant that just as Epic’s Unreal Engine allowed game developers to easily release their games across all devices and platforms, Epic Online Services (EOS) would allow these developers to operate multiplayer online services for these games across all devices, platforms, and engines. In addition, these developers would get access to Epic’s enormous player network and social graph.
EOS, in other words is “Fortnite services in a box.” By using it, a developer can instantly make their games available anywhere Fortnite is available, operate with comparable multiplayer functionality and at a comparable scale, and use its entire friend/team lists, too. And they get to do so for free.
And crucially, EOS is available without restriction for all games, with any engine, and through any store. This includes, as an extreme example, games based on a competing engine that aren’t even available for sale on the Epic Games Store. And unlike Steamworks, use of EOS does not require a player to ever have, let alone use EGS.
Note, too, that over the decade, a number of large-scale businesses have been built and invested in to specifically sell Streamworks-like services outside Steam and to support full cross-play and portability. Amazon’s AWS and Microsoft’s Azure, which built large gaming hosting businesses, also began developing their own gaming services solutions and buying existing market leaders such as Gamespark and PlayFab. Unity also built up a large suite of services including entitlements, communications, and analytics. And Epic now offers its entire hyper-scale suite — save for game hosting — for free.
"Right now, you have Xbox Live and PSN and Nintendo Switch Online and Steam, each with a competitive user base with friend systems limited to that one platform. Here we're trying to connect all the platforms for all developers and have a shared user base. The Fortnite friend system is the core of that ... Now all developers can use [it]. And when another developer uses the system in their game, they contribute their users accounts and friend connections to the system and it grows to benefit everybody. It's very much in line with our open strategy of wanting to connect ecosystems instead of building bigger walls around them.” - Tim Sweeney
The Value of EOS to Epic
It might seem odd that Epic would offer its cutting edge services suite at zero cost and with zero restrictions, except to use Epic ID for elements that would structurally require it (e.g. player-to-player matchmaking). In addition, it was already a market leader that serviced the market at scale and invested heavily in cross-play functionality.
According to Sweeney, EOS is “more valuable [to Epic] if free.” Much of this has to do with Sweeney’s belief that by breaking down the walls between every platform and player and making it easier for publishers and players to change platforms and stores, the gaming ecosystem will grow in value and be healthier overall.
However, the direct benefits to Epic are also considerable. As EOS grows, so too does Epic’s account system, player network, and player data. EOS also makes all other parts of Epic, from Unreal to EGS and EGP (below), more attractive to a developer. It is a world-class, end-to-end shop… and the cheapest in the market.
”What we get out of Epic Online Services is building up a persistent user base that transcends platform boundaries. The bargain is we give every developer access to the full Fortnite player base and social connections. Now, when people who play Fortnite come into their game they are immediately connected with all their friends. [Before], you had to rebuild a friend system for every multi-platform game that launches — if you play Fortnite and Call of Duty and Rocket League, you've had to build up these systems.” - Tim Sweeney
As mentioned above, Epic Games Store also provides Epic with added insight into which games are being played, as well as the future pipeline of games. This, in turn, helps with the development and optimization of Unreal. EOS enriches this dataset further and expands it beyond PC to all gaming platforms.
Finally, EOS makes it even easier for Epic to connect various virtual experiences together. After all, they’ll already be running on the same player network, with the same server tools, analytics, chatting software, and more. More to come in section six.
This essay is the fourth portion of the six-part ‘Epic Games Primer’, which covered every element of the Epic Games Flywheel, including Unreal Engine, the Epic Games Store, Epic Game Publishing, and Fortnite, as well as Founder/CEO Tim Sweeney’s vision for the future and unprecedented aspirations.
Matthew Ball (@ballmatthew) & Jacob Navok (@jnavok)