APIs for RTS: Ladder Edition

A wishlist for real-time strategy developers, part I

Over the years I’ve written a number of different integrations with APIs offered by real-time strategy games, including client and ladder APIs, replay parsing libraries, achievements data, etc. I spent an especially large amount of time with StarCraft II, a market leader in offering comprehensive, high-quality information.

For folks who are unfamiliar, an API is a contract offered by a piece of software that enables developers to build more software on top of it. For example, Blizzard offers community APIs that serve information about the StarCraft II ranked ladder, enabling the creation of websites like RankedFTW. I’m interested in any capability a game developer offers for community members to build stuff on top of a game. This includes everything from the above, to support for community-built mods in Age of Empires II, to the HTTP server exposed by StarCraft II which surfaces the game state.

(Basically, to keep things simple, I’m using “API” as an umbrella term to refer to any supported specification you can build against to enable new functionality. Apologies to anyone out there bemoaning the lack of precise language.)

I know some folks will think this is a topic that only engineers can understand, but I don’t agree. I don’t think you need to know how to build RankedFTW in order to know that RankedFTW is a useful website, right? Same goes for the value of Twitch overlays, or automated replay analysis, or a whole gamut of other useful tools. Part of the reason I write this article is to try to make this topic more accessible for non-technical folks, partly for your own knowledge and benefit, and partly so you can contribute ideas on what other services would be useful so that developers can support those use cases.

Today’s article focuses on APIs around the competitive ladder, but in the future I’ll cover things like replays and in-client APIs. One thing I’ll mention before starting is that user privacy is important; while ladder games are public data*, most other things - co-op, custom games, achievements, inventory, etc - should be taken case by case, and in most cases ought to be gated behind something like OAuth to ensure players give access only to tools that they trust.

Ladder APIs - The Basics

The most basic feature is a way to fetch the leaderboards - 1v1, 2v2, etc - with relevant metadata like ELO, wins and losses, team members, etc. This enables community members to build leaderboards websites around the game (like Nephest for StarCraft II or AoE2.net for Age of Empires II), which are popular in every competitive RTS I’ve played. It also enables basic analyses, like activity (games per day, regional trends, etc) or personal tracking (where on the leaderboard you currently sit).

I think it’s OK to keep this simple and delegate game understanding to the client. For example, the StarCraft APIs structure data in terms of metaphors from the game, like ladder divisions. And while that’s nice, it sometimes makes dealing with the APIs a pain; the data is generally unusable as-is after ingestion, and needs to be manipulated to answer common questions. For example, to determine a player's percentile rank in their region, the divisions data ingested from the API needs to be merged into a single ranking. It would be easier (and require fewer API calls, maybe) to just fetch the leaderboard page-by-page.

Another nice feature is a way to get data on individual players. Any data served from this endpoint can be used to build a profile page for a player on a community website, showing their wins and losses, recent games, win rate across matchups, and so forth, which I think is popular among players. Public data like ladder statistics could be offered to all callers, while everything else could be gated behind a user-authorized mechanism like OAuth.

Notably, achievements data is an underrated gem in the StarCraft II player profile data set - it enables analysis on how players are engaging with the game, even if it’s limited to just ladder players. For example, you can determine what proportion of ladder players purchased the StarCraft: Remastered graphics upgrade pack, because players receive an achievement for it.

Individual player match history is arguably the holy grail, because with that data, you can reverse engineer everything else - but I’m guessing it’s cost-prohibitive to offer more than just recent games for free. StarCraft II notably does not offer full match history; the Age of Empires APIs do (or at least appear to). Personally, I’d be willing to pay a fee every month to have access to this data.

Related to match history is the actual match data that’s exposed. Generally, games default to basic information - map, races, etc. However, some games involve permanent decisions in their match progression, like landmark choices (Age 4) or heroes (Warcraft III), that are low-cardinality enough to be surfaced by API, too. Not high priority, but it would be interesting stuff for community members to analyze.

More Nice-To-Have Stuff

I think a common client use case is building overlays for Twitch streamers. An ongoing matches endpoint can be helpful here, because it allows you to build an overlay showing the current player, the opponent, the races, the match-up, etc. This makes it easy for the viewer to catch-up when joining mid-stream. Not all streamers like this feature, of course, but it’s great to have the ability to build it.

Relatedly, ongoing matches can be a complement to in-game spectating (if a game supports it), allowing community sites to implement the heavy lifting search functions. I don’t think I’ve ever used Age 2 or 4’s in-game search features to find a match to spectate.

I also think it’s helpful to think about Twitch chatbot use cases. For example, a common thing in Age 2 streams is where one chatter might ask about so-and-so’s rating, and another will reply with “!lookup <player_name>” to confirm. This is already possible from above - a client could ingest the leaderboard into a table and then put a search API on top of it - but I think it could be good to shift applicable use cases to lightweight queries instead of requiring every API client do a periodic full ingestion. I’m not sure, it depends on how common this ends up being. (Similar to the above, some Twitch steamers dislike this feature.)

I also think it’s good hygiene to offer a read-only inventory API via OAuth, so that players can show off their in-game items if they authorize a website to pull that data. The use case I’m thinking of is a tool that pulls player’s skin data and allows them to mix and match how their army looks; this isn’t possible in StarCraft II, because you only see individual skins in isolation. This seems like a good example of a low priority feature, from a developer’s standpoint, that’s cheaper to enable by offering up the data via an API and letting a community member build it for free.

Write APIs for inventory are controversial and I won’t offer an opinion here, because it’s a game and business decision. One large use case is third-party item marketplaces, which have trade-offs, as can be seen in CS:GO.

Co-op, Custom Games, and Arcade

StarCraft II never offered good co-op data via its API, and I think that’s too bad. I think most of the value proposition in supporting community websites can be achieved through the above APIs - just include co-op data in ongoing matches and match history, and append co-op metadata like commander levels and masteries to player profiles. Same for custom games and the arcade. This enables all of the benefits you get from ladder data without needing to do that much more work. There is some extra work due to differing privacy considerations, though, so I get why some developers may just skip past this.

I think it would be great to surface aggregated data, too, but I guess it depends on how the data is organized. Co-op, for example, doesn’t have a natural leaderboard metaphor like ranked games. In the absence of an intentional design decision to have co-op leaderboards, I don’t think it’s actually necessary to offer an “aggregate” view of overall co-op activity the way leaderboards do for ranked play. As a data nerd I would of course love to have it - say, total games played in the current season broken down by commander and map.

One thing I will add here is that a publisher API for custom game makers would be helpful, too, for tracking player counts, trends, etc. That said, it may be cheaper to just have a standard recommendation on how mapmakers and other creators can implement their own instrumentation.

Historical Data

Ladder data can be tricky for games that reset their leaderboards periodically, like the seasons systems in StarCraft II and Age of Empires IV. If you’re late to the party and start querying the data mid-way through the game’s lifecycle, you’ll be missing historical data from previous seasons. From a client perspective this creates a “data hoarding” mindset - you’re incentivized to comprehensively invoke every API and store every response, on the understanding that even if you’re not using the data now, you won’t be able to get it in the future.

In an ideal world, developers would just surface all data about a game in a generic way - for example, every API request could take an optional “season” attribute, which defaults to the current season if not provided. Maybe that’s cost-prohibitive, so alternatively it would be great to surface high-level historical data for folks looking to catch up. I think this actually reduces API maintenance cost in the long-run, because all users know that they can catch-up on anything they miss, and therefore only query for what they need.

I think it’s fine to keep this basic - maybe even just the final leaderboard for each historical season. Aggregated rating history for individual players (maybe sub-sampled as data gets older) would be nice, too, especially if it’s organized on a season-by-season basis to enable progression tracking sites.

Esports

I haven’t personally seen a game support esports APIs natively, like results look-ups, player profiles, etc. I think this is partly because there isn’t good data - when it comes down to it, esports tournaments in RTS are just a long series of seemingly random custom games. Data is fragmented, and collected and organized manually by fans.

I think if the client natively supports esports - say, an in-game bracket maker, or a separate tournament ELO, things like that - exposing it via an API would be useful, although I think it would be a heavy lift to fully replace community websites that support the same via manual tracking.

A cheap hack here would be to enable tagging of custom game lobbies, and then ask tournament organizers to tag all their tournament games with, say, the name and date of the tournament, as part of the game’s licensing agreement. It’s not a perfect solution (you can’t reverse engineer a bracket) but it enables a lot of interesting design and balance analyses. Plus, with the right wiring you could automate the creation of tournament replay packs.

Developer Support

I also poked around asking a few other knowledgeable folks from various games their thoughts on this topic. One thing I heard that resonated with me was a desire to have an open line of communication between API users and the developers. I get that many people feel this way (pro players, streamers, content creators, etc), but the thing I’d highlight in this case is that because this can be structured as a pure engineering discussion, it may be less resource-intensive for developers to support. And since APIs are typically used to build services on top of a game, saving developer time and adding value to the title, it’s probably win-win for all parties.

My personal take is that I’ve chatted with many developers over the years, and it’s always been a good experience, so I of course agree with this feedback on a personal level. I also see it as a general best practice in API design. But I also recognize how challenging (perhaps even annoying) it is to take feedback on what is probably not an easily resourced, high priority feature. In the case of StarCraft, for example, I would guess the API serves divisions data because that’s how the data is structured internally, and Blizzard never staffed building an alternate solution after MMR was made public. And that’s totally fine. Resourcing is a trade-off and if the perspective of a developer is that they have time to put something out, but not time to iterate on it, well, I get it.

Final Thoughts

Whew, I think that’s it. That’s a long wishlist! Before I go, I wanted to thank the many intrepid engineers who have enabled RTS APIs over the years. It’s appreciated. I’ve joked in the past about Age of Empires having an undocumented ladder API, but maaaan does that beat the alternative of being in the dark. Thank you to folks who keep stuff like that going!

Until next time,
brownbear

If you’d like, you can follow me on Twitter and Facebook and check out my YouTube and Twitch channels.

* The idea here is that ladder games are public both factually (they are explicitly included on leaderboards, match history, etc) and from the perspective of user expectations (their handle is going on a leaderboard, their rating is publicly displayed, competitive titles have historically supported public access to ELO for community sites, etc). Users may not have the same expectation of, say, a coop game with a random partner, even if it’s online and there is a replay. With privacy I think it’s important to meet users where they are and make conservative assumptions about what they consider public data.