Official development blog

Special Mode Design: Battle Royale

Every good game needs a Battle Royale mode, right? Okay maybe that’s something you’d be more likely to expect from an actual online multiplayer AAA title looking to jump on a bandwagon, rather than a traditional roguelike :P

Nonetheless, I did build one for Cogmind! It was going to be Cogmind’s next April Fools Day event, with the idea originally sparked by a player’s “what if”-style comment on Discord earlier this year.

If you’re not familiar, “Battle Royale” is basically free-for-all survival of the fittest gameplay across a shrinking play area filled with random gear. So here you’d basically be up against a bunch of other Cogmind-like bots trying to survive. I’d never actually played any of the staple games in this genre, my first and only relevant experience coming during 7DRL reviewing for 2019, when I played B-Line, a take on the BR genre within a traditional roguelike framework (original video, when I was going back and playing the top 7DRLs for the year).

I say “was going to be” the next AFD event because although it reached a playable and somewhat fun state, I ended up cancelling it! I’ll talk more about why later on, but first let’s look at the development process and its challenges and solutions…

Exploration

Like many ideas, as a pure concept “Cogmind: Battle Royale” sounded pretty interesting, but I wasn’t sure just how feasible it would be on a technical level. I mean sure it’s doable, but would the cost be worth it? A pair of very important questions needed to be answered first, so development started with an exploratory phase directed at answering them before actually building this as a special mode.

The first question had to do with factions. Cogmind’s whole relationship system is based on a limited set of hard-coded factions, and there are only 10 of these internally. How to build a system to support free-for-all hostilities between numerous robots? The most straightforward solution would be to dynamically create a unique new “faction” for every single robot. What might this break? I had no idea, since lots and lots of assumptions had been made and reinforced since development began in… 2012 for 7DRL xD. I wasn’t sure how flexible the faction system was, but this was a natural place to start since it would be much easier to test than question #2 to come. I simply created a bunch of extra factions and dropped a single Grunt in each, heading off the few problems that might result and sure enough, each one was running around shooting anything that moved and otherwise behaving as expected. Yay!

cogmind_battle_royale_first_faction_experiment

Grunts successfully wreaking havoc on -10/Materials, and each other.

The second question would unfortunately be much much harder to answer. It would take a while to explore the feasibility of an AI with Cogmind-like behavior, having the ability to put together and maintain a build from found and salvaged parts, and also manage inventory.

Now if I was just doing this purely for a short special event then I’d be less willing to venture into complex AI territory, essentially because it’s an endless time sink (!), but I liked how this little detour could bring me one step closer to another project I’ve thought about off and on over the years: Having a bot that can actually play Cogmind.

So it was worth a shot and I spent some days hacking together an AI much more complex than anything else in Cogmind (robots normally have relatively simple and predictable behaviors unique to their class, where interesting scenarios are produced by interactions with numerous AIs at once rather than especially smart and capable individuals).

AI Development

Technically I wasn’t starting from scratch on this AI. Quite conveniently there was already another smaller stepping stone in place: Abominations from last year’s Halloween event.

Abominations in that mode are capable of gathering parts to build themselves, but do so very haphazardly and take a lot of shortcuts, being capable of telekinetically attracting any parts they see and pretty much ignoring balance because they don’t need to worry about energy or heat and always have a permanent form of natural propulsion and built-in backup weapon. For build management purposes, architecturally there’s a pretty big difference between an AI allowed to both decide on and complete an action in a single moment, and one that makes a decision then follows it up over multiple turns in a changing environment.

In short, Abominations are a far cry from a true Cogmind-like inventory management AI. But they did require repurposing the player item autoswap feature for AI use, which would form an important backbone of the new AI. As small as it was, at least that bit was already done. To recap, back in Alpha 5c (2015) I added the ability to use a series of simple and predictable rules and to automatically perform obvious inventory management actions on request.

cogmind_upgraded_autoreplace

The original diagram I made to explain the smart/automated inventory management system back when it was first added.

Nearly all the same logic could be used by AIs as well, so we have a pretty solid foundation to work from!

The new AI clearly still had a wider range of needs, though, so at this point I didn’t actually start creating the new mode or a Battle Royale map, instead basic AI testing simply happened in a regular Cogmind map. In the first phase of development, I added the following features in this order--the AI could:

  • use core hover (normally all robots other than the player are immobile if they have no external propulsion)
  • switch to different faster forms of propulsion if more efficient
  • collect parts to fill empty slots and also upgrade existing attached parts (however, because this is the AI rather than a player, it can take even more liberties with the swapping, for example removing inactive propulsion that doesn’t match the current movement mode to make room for more of the active type)
  • manage mass vs. support, for example avoiding adding so much mass it goes past a desired overweight threshold, and tries to shed heavier parts if it’s suddenly gotten heavier due to build changes, to maintain speed
  • wander the map while improving its build without fleeing under fire (since it needs to prioritize searching for and collecting weapons if it’s been disarmed)
  • seek out Matter and Protomatter
cogmind_battle_royale_AI_first_attach_sample1

The first-ever AI to pick up its own parts, when I first started the project.

 

cogmind_battle_royale_AI_first_attach_sample2

A slightly less stupid (but still very stupid) later version.

Here I also considered adding ramming to their basic abilities, so that like Cogmind the AI could also have a useful attack while disarmed, but found there were too many complications so although I figured out how it would be possible, ultimately decided it wouldn’t be worth it for this event.

I didn’t get into personalized behavior of any kind, like AIs with a preferred style, since there’s already plenty of randomization and variety coming from the many many parts and their conditions at a given time. This would be a natural future step for a more involved project, however.

The complete non-combat portion of the AI can be summarized with pseudocode like so:

cogmind_battle_royale_AI_build_management_pseudocode

Cogmind Battle Royale AI pseudocode (non-combat state).

Once the Battle Royale AI reached this stage, it had started getting complex enough that it was really annoying trying to fine tune it while working in regular Cogmind/0b10 maps where there are too many distractions. About time to make the new map!

The Arena

The idea behind this mode was that there’d be only one map, the first ever Cogmind mode to pare down the game into such a small “world.” Initial plans set it at 200×200, the same size as Factory/Research, based on my experience designing maps of that size relative to movement speeds, the ranges of sensors and weapons, and perhaps most importantly the total number of participants. 30 seemed like a good number.

At first it was going to be like a typical Cogmind room-and-corridors style map, with a final central room on which the last living bots would naturally converge as the accessible area shrank. Here I was just starting out getting the map generator building out from the central room:

cogmind_battle_royale_mapgen_early_wip

Early WIP Cogmind Battle Royale map layout sample with central room.

As I was fooling around with trying to get it to do what I wanted, a different better idea popped into my head: What if instead of using the normal subterranean-style maps we instead directly converted it to a sort of overland map where rooms are instead “structures”?

This would significantly open up the map, creating additional pathways and a sort of “organized chaos” in the form of a little town without clear straight roads, but all locations are still accessible.

cogmind_battle_royale_mapgen_wip_4

Sample Cogmind Battle Royale map layout.

 

cogmind_battle_royale_mapgen_wip_1

Sample Cogmind Battle Royale map layout.

I like it! Some of the strategy implications of this approach:

  • makes it much easier to navigate and reach the center (if necessary one can technically even just blow through walls to reach the center once you have orientation)
  • makes it easier to escape if you’re faster than a pursuer, harder if you’re slower (and harder to give chase if you’re slower)
  • easier to spot enemies and be spotted because generally more open
  • less likely to get unnecessarily trapped by the closing circle of death

Plus it’s also kinda neat just to have a different kind of map to play around with.

I called the map “Wartown.”

Mechanics and Features

We’ve got a basic AI and a basic map, next up are features!

  • Everyone spawns in their own room, usually out near the edges of the map.
  • Everyone starts with the same base stats and slot count, but completely randomized gear. It’s more interesting to be forced to adapt, but rather than having players gradually collect their parts from around the map, that entire process can be sped up by simply starting with it. This is more appropriate for Cogmind anyway, because one or two parts alone isn’t enough for an interesting or fun build--you really need a larger collection to be at all effective or to more quickly convert into something effective.
  • Random parts are also scattered across the map (some items found in the regular game are excluded, however, generally things that wouldn’t make sense in this mode).
  • There is no Matter cost associated with attaching parts. This rule change was partially put in place to make inventory management a little easier on the AI (no need to consider Matter waste alongside repeated build tweaking), and also to speed up gameplay in general because it would otherwise be easier for players to run low on or out of matter.
  • Attacks enforce RPGLIKE-like damage redirection where some part damage is transferred to the core, but in this case the rate is only 50% (and armor transfers only 25%). I wanted parts to still take plenty of damage and be destroyed, but at the same time didn’t want too much damage focused on target cores because otherwise bots wouldn’t last long enough in a confrontation, at least not long enough to allow for some tactical variety.
  • Cogmind’s parts do not have their usual high-integrity critical strike protection, and no bot can have their core automatically destroyed by a critical strike.
  • All bots use the normal player-Cogmind rules for increasing system corruption when his with EM damage (so it’s no longer as much of a threat to other robots as it normally is).
  • In addition to salvageable parts, destroyed bots drop both Matter and Protomatter, the latter of which does not decay and is applied in full each turn where necessary/available just by standing on it. Borrowing Protomatter from RPGLIKE mode was essential in maintaining balance because without a way to repair, attrition would inevitably lead to death after one or more confrontations.
  • On destroying a target, the attacker gains a new random slot (awarded on the following turn). This allows for a form of progression, ensuring it’s less advantageous to simply hide from everything and pop out at the end to kill the last remaining enemy, since whoever they are they’re quite possibly more powerful by then.
  • Ambient heat slowly spreads from the four corners of the map towards the center. This is the so-called “circle of death” mechanic important to Battle Royale games in order to force players into eventual contact with one another while keeping it from getting boring. I like the idea of using heat rather than damage to achieve this goal because it builds gradually as it approaches and builds with good dissipation can afford to stay near or in the hot area for longer, not to mention heat has different kinds of side effects. In the end no build can withstand the full intensity of the heat, of course, it just allows for more or less flexibility in that area. The AI is aware of the heat area and acts accordingly.
  • Once only a single bot remains, an exit appears in the center of the map.
cogmind_battle_royale_heat_sample4

Heat encroaching on the map’s center room.

The heat’s color effect is achieved via additive RGB based on the actual ambient heat per cell, so maximum heat actually appears yellow (although the animation still plays out for all hot locations so it’s not a constant yellow).

cogmind_battle_royale_heat_sample2

Full heat has filled the map, though it cannot enter the central room.

 

cogmind_battle_royale_heat_sample3

Catching the heat mid-glow after it’s covered the map. AKA making procedural art.

 

cogmind_battle_royale_heat_spread_sample1

Recorded bird’s eye view of heat expanding from one corner over time.

Each player is also assigned a random name with their player number attached. For this I wrote up a list of hundreds of Cogmind-relevant words for use in compounds.

cogmind_battle_royale_name_sample2

Sample generated Battle Royale “player names.”

Cancelled!

Normally I would’ve cancelled an idea a lot sooner in development than this (like the real-time arcade version of Cogmind I never even wrote about here but released to patrons at one point :P) but due to the investment required to develop the necessary AI, it took a while to reach an MVP state so I didn’t get to testing whether it was actually a good idea for a while xD

As soon as it was semi-playable I immediately realized it wasn’t quite going to work, at least not without a huge amount of additional investment that just wouldn’t be worth it for a special mode.

In the end, Cogmind’s mechanics and gameplay weren’t designed to be all that compatible with 1v1 fights between equals, which is what Battle Royale tends to boil down to--you have a bunch of essentially identical players all vying for the win, but Cogmind is meant to be a more tactical one vs. many game (or many vs. many, with factions) in which most bots die quickly and the situation keeps changing as a result, rather than just sitting in one position pelting enemies, which is often what BR mode using Cogmind mechanics ends up as. Certainly you can do a fair amount of stationary farming in regular Cogmind, too, but there’s still a lot of thinking that goes into the plan before the fighting even starts, and that determines how the fight will be likely to go--this is something the AI can’t really do on an equal footing with the player, and it ends up being less interesting as a result.

In BR mode, AIs spot a target and then sit there doing their best to pummel it to death if they’re capable of attacking. Without a much smarter tactical AI that can come up with environment-related tactics and exercise more forethought and better react to spacial disadvantages, it just doesn’t work well.

In hindsight this all should’ve been obvious or could be realized via thought experiment alone, but I was clearly hyperfocused on the AI development aspect and really wanted to give it a try anyway :P

But Playable!

Before stopping work on BR mode, however,  I actually put some more hours into it so that it’d be a more complete and fully playable prototype for patrons who’d like to try it out, and feedback was actually decent! (maybe I was too harsh on the mode and should bring it back one day with a bit more development? :P) The heat mechanic, among some others, is an example of a feature that wasn’t even implemented yet when I first decided to cancel--I admit I also really wanted to see what it might look like :D

I released a special Battle Royale build on Patreon here, and since then have updated the build several more times (in place on the original page) to address bugs and further improve the AI.

cogmind_battle_royale_fight1

A rather quick and explosive skirmish in BR mode.

Features missing from this mode, as released, but that I would have addressed later:

  • prefabs (including special stockpiles and especially better treatment for the center area)
  • more attention to machines (all I did was throw some random machines into the mapgen pool, where ideally the types and compositions would be selected for both better gameplay and atmosphere)
  • message log details about who is destroying who
  • Haulers that teleport in and carry lots of great gear, and everyone gets a ping of their locations (works as a mechanic to force confrontations and give extra rewards those who succeed)
  • different enemy colors to reflect their state
  • different game over screen, a leaderboard ranking all the “players” (unique scoring system) and showing their build classes
  • even better AI :P
  • a more balanced experience overall, or at least “balanced” to maximize FUN

Results

Fortunately most of the work that went into this mode didn’t go to waste. Cogmind Battle Royale may have been cancelled, but the AI lives on! I’ve continued to improve on it (vastly!) for use in a different special event, and there is at least one part of the regular game where I’m interested in incorporating just such an AI. So I’d consider this project a success overall.

This side project also resulted in some general AI improvements and even discovering and fixing one relatively new AI behavior bug that hadn’t yet been reported.

I’m not sure whether I’ll ever get to that AI that can reasonably play the game, although the next little step towards that goal is only barely beyond what we have now--just take the first exit spotted ;)

Posted in Design | Tagged , | 6 Responses

What is a Traditional Roguelike?

In the beginning there was the Berlin Interpretation… Just kidding, believe it or not there were arguments about what constituted a roguelike long before 2008, but in the decade since then we’ve entered a new era in the wider roguelike genre in which that lone word itself has become increasingly diluted as mainstream games easily overtook the original more closely related set of games in popularity, allowing the common definition to shift and expand almost beyond recognition. Depending on who you ask now, “roguelike” can refer to games as disparate as these:

nethack_vs_tower_of_guns

Spot the roguelike.

At the top we have NetHack, and on the bottom is Tower of Guns, where the latter and other equally distant examples (borrowing maybe one or two aspects of roguelikes) are generally referred to as “roguelites” by people familiar with the time before this newer expansion.

Personally, I am very much in support of the expansion of the genre in every direction, injecting roguelike values into other genres to create new types of enjoyable experiences. Although in some cases these games may veer pretty far from what some will accept as roguelikes, games that aren’t too far out on the fringes are also known to sometimes lead new players to discover “traditional roguelikes,” which is great!

Note: This article will sometimes simply use the word “roguelike” to mean roguelikes in the traditional sense unless otherwise stated.

One thing worth pointing out right away is that although I’m writing up a narrower take on roguelikes here and even recently started a community dedicated more explicitly to “traditional roguelikes,” I don’t recommend that developers try to adhere to a strict definition of what anyone thinks roguelikes are or should be. Too often in dev communities we hear from prospective developers worried that their game is “not roguelike enough.” Over on r/RoguelikeDev the definition is basically “whatever you think it is,” because interesting game development is often about testing and breaking rules and boundaries, after all. Trying to shoehorn a game for that perfect fit into a specific definition isn’t a really meaningful endeavor if it’s going to warp the game idea in a bad way or keep it from reaching its full potential. Develop the idea, not for a genre.

On that note, it’s also important to realize that “traditional” doesn’t mean that we’re necessarily talking about only old games here! (I would dub those older representatives “classic” roguelikes.) New traditional roguelikes are popping up every year, featuring plenty of innovation while remaining squarely within the traditional sphere.

r-TraditionalRoguelikes_banner_image

For r/TraditionalRoguelikes I put together a banner of images from various roguelikes, a mix of old and new spanning decades (click and zoom for full-size).

Definition?

Giving “roguelike” a definition to satisfy everyone is an impossible task. Even within the core community of roguelike players well aware of Rogue, whether or not a game should be considered a roguelike is commonly attributed to each individual’s “feeling” and whether a given game adheres to the “spirit” of the genre as they understand it. Naturally these opinions are going to vary as much as these players’ respective experiences, even more so in a world surrounded by an increasing number of games each day. As I’ve seen over years of repeatedly watching relevant arguments play out online, many different perceptions of the roguelike spirit are clearly irreconcilable. The reason behind this was pretty eloquently summed up in this comment by silverlarch, reproduced here:

“Traditional roguelikes […] share specific qualities with Rogue, namely being grid/turn-based with procedural generation and permadeath. That’s not all Rogue is.

Let’s compare some games.

Rogue is a grid/turn-based dungeon crawler with full procedural generation and permadeath in a fantasy setting. It’s about descending through a multilevel dungeon to find the Amulet of Yendor and return it to the surface. It has no RPG mechanics: the only stats are health and strength. Your character is defined by the equipment you find along the way, and health increases with experience gained from killing monsters. You have to deal with identifying potions, scrolls, and items that may be beneficial or harmful. The dungeon is filled with monsters, traps, and secret rooms.

Tales of Maj’Eyal is a grid/turn-based dungeon-crawling RPG with permadeath and some procedural generation in a high fantasy setting. It has character classes and talents, experience-based leveling, meta progression in terms of unlocking new classes, an overworld map, quests, a skillbar, non-dungeon areas like towns, interactive friendly NPCs, and extensive shops. ToME is a traditional roguelike, but not a whole lot like Rogue.

Cataclysm: Dark Days Ahead is a grid/turn-based open world survival game with full procedural generation and permadeath in an apocalyptic zombie sci-fi setting. It has no end goal, RPG mechanics and a wide array of skills for character progression, base-building, vehicles, potentially friendly NPCs, quests, special abilities via mutated or bionic character upgrades, weather and time of day, injury and morale systems, and extensive crafting. C:DDA is a traditional roguelike, but really nothing like Rogue.

Brogue is a grid/turn-based dungeon crawler with full procedural generation and permadeath in a fantasy setting. It’s about descending through a multilevel dungeon to find the Amulet of Yendor and return it to the surface. It has no RPG mechanics: the only stats are health and strength. Your character is defined by the equipment you find along the way, and health and strength are increased by potions. You have to deal with identifying potions, scrolls, and items that may be beneficial or harmful. The dungeon is filled with monsters, traps, secret rooms, and minor puzzles. It has recruitable NPC monster allies, minor stealth mechanics, and environmental interaction in the form of gas, liquid, and fire mechanics. Brogue is a traditional roguelike, and about as close to Rogue as modern roguelikes get.

Unexplored is a realtime/pausable non-grid-based dungeon crawler with full procedural generation and permadeath in a fantasy setting. It’s about descending through a multilevel, branching dungeon to find the Amulet of Yendor and return it to the surface. It has no RPG mechanics: the only stats are health and strength. Your character is defined by the equipment you find along the way, and health and strength are increased by potions. You have to deal with identifying potions, scrolls, and items that may be beneficial or harmful. The dungeon is filled with monsters, traps, complex puzzles, and occasional secret rooms. It has minor stealth mechanics, occasional small shops, minor crafting, minor meta progression in the form of unlocking new basic starting equipment, and environmental interaction in the form of gas, liquid, and fire mechanics. It’s basically a realtime Brogue clone with a better proc gen engine and a few added mechanics. It is not a traditional roguelike, but very much like Rogue.”

Bullet points can’t accurately capture the breadth of the genre, even in its traditional sense.

At the same time, as I thought about ways to describe the “feeling” that adequately captures the roguelike spirit, I realized that even such a description can’t be meaningful in a vacuum--a list of core features is still a necessary component.

Now I normally don’t bother wading into the endless arguments on this topic, but having created a new community more explicitly centered around them I essentially forced my own hand and needed to write something xD. The aim is to try to be inclusive of at least a decent percentage of “personal definitions” floating around the core roguelike community.

Here goes!

xkcd_standards

xkcd: Standards

What is a Traditional Roguelike?

First of all, hopefully you’ve actually read the above background info for all-important Context, and didn’t just skip down here :P

While “roguelike” is a somewhat more permissive term as used these days, traditional roguelikes have a stricter focus requiring all of the following elements:

  • Procedural content: Some content, often the environment, is procedurally generated. This increases replayability by keeping the game interesting and surprising on repeated runs, while also making gameplay expertise more about mastery of tactical and strategic knowledge. Much of the play time in a roguelike is spent exploring, so procedural maps in particular are an effective tool for generating fresh experiences both on a local (different enemy compositions and room layouts for tactical analysis) and grander scale (different routes or orders to visit world locations?), especially where there are many types of environments and inhabitants to draw on. Maps can technically be procedural to different degrees, where one may have static vaults/prefabs integrated into larger generated maps in order to provide particular points of interest or even areas outwardly recognizable for their gameplay meaning. Note that even in identifying procedural content as a vital element in traditional roguelikes, consistency is still important in order to allow for learning and eventual mastery of the relevant systems.
  • Permadeath: Death is permanent. To retain the element of surprise in a procedural world, consequences matter. Traditional roguelikes are “run-based” games in which save/load features exist only to continue playing at another time, rather than to redo actions and therefore lighten their meaning. As run-based games there is also little to no metaprogression--you grow as a player through knowledge and skill, but these aren’t explicit benefits carried over from one run to the next. A small amount of metaprogression is not a problem where there is a hard cap which is quickly reached and serves some other purpose in the design without breaking the consistency of its replayability. Growing ever more powerful with each run is a very anti-roguelike metamechanic, whereas unlocking lateral game content is technically fine. Note that some traditional roguelikes allow players to optionally forgo save/load restrictions, either to avoid permadeath or even to redo actions; at that point they’re not being played as a TRL, although this option doesn’t negate the game’s status as a traditional roguelike altogether unless that is the default and expected method of play for which the game was designed.
  • Grid-based: Traditional roguelikes play out on a discrete grid with cell-wise movement and actions, aiding in tactical calculation. Combat must be non-modal and take place on the same map grid where exploration occurs, but non-modality is not a requirement for interaction with shops, towns, or a world map.
  • Turn-based: Actions are carried out in a turn-wise manner, with the player taking an action followed by all other entities acting in turn. Turn-based action gives players unlimited time to react to changing and unexpected circumstances and weigh the permanent consequences in formulating a response, testing analytical skills rather than physical reflexes and quick thinking.
  • Single player-character: The player controls a single primary character represented on the map, and the death of that character is the end of the game. It’s okay to have AI-controlled allies, partial control over allies, or even switching player control to an alternate character by means of in-game abilities, but the normal state of the game should not be controlling multiple disposable “units.”

As you can see from the list above, “traditional roguelike” isn’t nearly as limiting a category as one might think. Even within these confines there is practically limitless space for innovation on mechanics and content and theme.

But again there is also a second component to fully describing a traditional roguelike, the feeling that’s hard to put into words but both born of and complementing the list of features above.

Darren Grey touches on some of this in his description here:

“…in my view it’s inherently replayable, capable of surprising the player on many playthroughs. It rewards cleverness and tactical thinking. It cannot simply be learnt by rote, but it can be mastered with experience. It emphasizes gameplay before aesthetics, concentrating on making that replayable experience fresh and engaging on each play. It’s unforgiving, but all the more rewarding when you perform it well, offering an honest sense of achievement and satisfaction. Much of this satisfaction comes from the internal knowledge of having done well at the game itself, rather than artificially constructed rewards.”

(This could still be said to describe games that aren’t roguelikes, but it at least further links many games that also share the list of roguelike characteristics.)

With regard to “gamefeel,” I find roguelikes are both challenging and addictive in a way similar to many multiplayer games in that you’re up against uncertain challenges that are operating within parameters established by the game world, only this is the slow singleplayer version of that experience. Roguelikes rely on multiple or even numerous overlapping systems to create a complex web of emergent possibilities, increasing both the number and variety of challenges as well as the number of tools available for problem solving in any given situation. As a result they’re highly replayable, and you’re frequently encountering new and changing conditions. It’s the player’s job to navigate those systems both as the in-game character and as the meta player. This is where the heavy decision-making focus comes in (lots of variables to consider!), and while roguelikes are often played quickly, you sometimes need extra time to figure out the best course of action (hence the focus on turn-based action on a discrete grid), since any negative consequences could either immediately cut the run short or potentially reverberate throughout the rest of the game, thus adding weight to each decision.

/u/aaron_ds also has a good overview covering the mix between feeling and feature with roguelikes rooted in “accurate threat assessment in tactical scenarios generated by systems.”

In the end all the elements of a roguelike detailed in the earlier list strongly complement each other to produce a satisfying “roguelike gestalt,” and removing or altering any of these is likely to destabilize that balance and fundamentally change the experience. The result might be like a roguelike, but is likely already more appropriately classified as something else at that point. For example basing the gameplay on multiple controllable characters is more of a tactics game (X-Com!) with a different style of pacing, removing permadeath pushes a game into more typical CRPG territory where consequences are not nearly as pronounced, and adding real-time elements requiring quick reflexes or at least limiting decision-making time belongs in the explosive “action roguelike” subgenre.

Examples of classic roguelikes (more here):

Many classic roguelikes have since gained one or more optional tilesets, although all began with ASCII as shown below.

roguelike_sample_rogue

Rogue (1980)

 

roguelike_sample_nethack

NetHack (1987)

 

roguelike_sample_angband

Angband (1990)

 

roguelike_sample_crawl

Crawl (1995)

 

Examples of modern roguelikes:

Among more modern roguelikes you can see a significantly wider visual variety, including a number that don’t even support ASCII, or are at least don’t use it by default.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

In the r/TraditionalRoguelikes sidebar I’m also maintaining a list of of excellent newer/underplayed roguelikes people can explore. As of this writing the list includes:

 

 

 

 

 

 

 

Features that might suggest a roguelike, or be related to them, but aren’t absolutely necessary for most players:

  • ASCII: Minimalist aesthetics with visuals boiled down to sometimes little more than the essential tactical factors are pretty common among traditional roguelikes, but not required since graphical fidelity is tangential to roguelikes being rooted in gameplay mechanics.
  • Identification system: All the way back to Rogue, many roguelikes include a system of unknown items that may be identified by external methods, risking their use, or deduced through other means. This is one of many systems to interact with, and adds yet one more strategic aspect to exploring a procedural environment.
  • Inventory management: Although very common in roguelikes since an inventory offers the player a familiar and intuitive way to collect and organize the various tools for tackling challenges, technically a roguelike could achieve that goal using other means such as abilities and other types of resources.
  • RPG elements: Stats, character classes, and leveling--roguelikes tend to have some amount of these, if not many, to provide the player with pathways for progression and further increase options for replayability and hooks for interactive systems.
  • Monster-player similarity: Mechanics governing players also apply to monsters, e.g. both can have inventories, equipment, use items, cast spells, etc. Although not a direct requirement, on the development side this approach does a good job of facilitating the wider range of clever interactions for which the genre is known, especially where other entities are using the same internal architecture as the player and therefore a greater number of features and interactions can easily be applied equally to both.
  • Dungeon environment: In the early days a majority of roguelikes took place in fantasy-themed subterranean maps composed of rooms and corridors. Today fantasy roguelikes still outnumber other types, but even among traditional roguelikes the genre has since expanded into many different types of environments as well.
  • Keyboard support: Ideally most or all actions are accessible without using the mouse, allowing players to quickly transition from one important decision to the next.
  • Quick action resolution: Actions carried out by both the player and other actors are usually instantaneous and involve minimal or no animation (or at least easily skippable animations), giving the player full control over the pacing. By extension there’s also usually a message log recording details of events as they occur in case the player misses something or needs additional details.

Examples of features/game types that aren’t considered traditional roguelikes:

While many of these might fall under the wider roguelike umbrella, and people who enjoy traditional roguelikes will likely enjoy them, this doesn’t make them traditional roguelikes :P

  • Real-time elements. Physical reaction times shouldn’t be an influential factor in roguelike play.
  • Games with no local map at all (lots of card-based games do this).
  • Puzzle-type roguelikes. These are often deterministic and play out on a small board without involving exploration, featuring lower levels of resource management (if any) and fewer abilities and problem-solving tools in general.
  • Modal combat, for example JRPG-style encounters which occur in a space off the explored map.

History and References

Previous definitions:

Other references:

r-TraditionalRoguelikes_banner_images_block

Also I like collecting traditional roguelike images, got tons of ’em :)

Posted in Uncategorized | Tagged | Leave a comment

Special Mode Design: RPGLIKE

cogmind_beta_9.3_rpglike_logo

Banner image from the Beta 9.3 “RPGLIKE” release.

This winter holiday season I released Cogmind’s seventh special mode, “RPGLIKE,” and quite unlike other events which have not necessarily been designed as permanent parts of the game to promote going forward, this one definitely is, and serves a specific purpose in that regard.

A number of people are interested in the world of Cogmind, but find that it’s extremely unique mechanics don’t click for them, most specifically the rampant item destruction, heavy item management requirements, and generally decent level of tactical prowess required to succeed. There are difficulty settings to somewhat mitigate these aspects, including even more recently the ability to quick save/load in all but the hardest mode, but those still don’t change the fundamental nature of adapting to loss and dealing with natural attrition, characteristics that permeate the normal Cogmind experience by design.

So I thought I’d try an experiment in attempting to convert Cogmind gameplay into an optional experience more similar to other traditional roguelikes. This means a number of things, most importantly a system for permanent progression, the ability to heal yourself, and items that don’t break.

Early Concepts

The earliest ideas for this mode were even more extreme than the final result (which is itself a pretty extreme departure from regular Cogmind :P). They assumed your parts would be completely immune to damage, and that you could simply heal either by “resting,” or gradually over time--basically your usual CRPG/roguelike mechanical tropes.

However, the design went through a number of iterations before implementation even began, because as I tried to run the numbers and make sure there would be some semblance of balance (also taking into account likely player strategies given each scenario), it became apparent said tropes in that form really couldn’t exist in the world of Cogmind and keep the game fun at all. I mean sure they’d be fun insofar as it can sometimes be fun to just run around practically invincible destroying everything, but there would basically be no challenges unless I also put a lot of focus on changing the nature of the existing challenges.

But with a mode already planned to be this big, it wouldn’t be reasonable to take on yet another huge amount of work on top of it, just for a special mode, so those starting points inevitably had to be scaled back… but we’ll get to that later. The implementation really started in earnest with one of the less questionable aspects: gaining XP and using it to choose permanent upgrades.

Upgrades

Instead of the usual evolution effects with each new depth as Cogmind approaches the surface, even basic stats like slots, max core integrity, and heat generation would be gained as XP-earned upgrades. But this being “RPG-like,” we definitely need more stats to modify on level up…

Types

I started with a broader and longer list of factors the player could theoretically be given control over as they leveled, but pared it back to below 26 because I wanted them all to fit in a single window, visible without scrolling, while still being entirely accessible via letter keys. (This is yet another of the many cases in which Cogmind’s design is guided by UI restrictions, rather than prioritizing content over ease-of-use, but that’s honestly the best way to go about it because there’s plenty of design space within even much tighter restrictions, so usability should generally be the priority.)

cogmind_source_rpglikeUpgradeTypes_enum

The internal enum listing all valid upgrade types.

Notice that many of these upgrades overlap with existing item effects, at least in part if not entirely. This honestly isn’t an ideal way to build this mode, which would benefit more from having every single upgrade provide or improve a unique ability unable to be acquired via any other means. But the better alternative would be a huge undertaking, whereas basing upgrades on existing systems and code is fairly quick by comparison--altogether it took a couple days to build the upgrade UI and get it hooked in with the game’s data that it could modify.

Having some overlap does, however, enable the player’s progression strategy to emphasize one of flexibility (more slots, like regular Cogmind which is maximum flexibility) or safety/consistency (using permanent upgrades), especially in cases where the functionality they’re after can come from either a part or upgrade. Upgrades are naturally also the more accessible of the two options, since they only require XP and can be obtained at any point, rather than finding the relevant part (and better versions of it later).

You can actually graph when it’s more effective to have a part for a given effect vs. the upgrade, to a certain extent, weighing the cost of the upgrade vs. the cost of evolving the required slot, but the calculations change over time depending on the necessary part’s precise effect, and how many slots the build already has, since the latter get increasingly expensive. Trying to optimize these decisions can be complicated yet further by additional requirements in some cases. For example increasing raw energy generation is a simple straightforward way to get more power, but power sources can do the same thing and also come with more storage capacity for surplus energy, though also produce heat, require additional mass support, and occupy an additional power slot (yet another upgrade).

Relative costs are clearly the most important balancing factor here.

Costs

XP is used to increment upgrades, and each has a base cost, a modifier (a static value for the amount of stat increase per upgrade), and a cost modifier (a static value for the amount of increase in cost for each additional time an upgrade is incremented). For example, the first upgrade to sight range costs 100, increases by 1 per upgrade, and each upgrade after the first adds 200 to the cost (e.g. 300 to upgrade it a second time).

cogmind_rpglike_upgrade_costs_graph_sight_range

RPGLIKE cost upgrade graph for sight range. (Cogmind’s base sight range is 16, so its first upgrade brings it to 17.)

The result of this system is a linear increasing cost for repeated upgrades, essentially adding to the cost of specialization. 1,500 is the amount of XP available to spend at each new level raised, so according to the graph the 8th sight range upgrade, from 23 to 24, requires an entire level’s worth of experience! At that point (or even before) it’s probably more cost effective to add another utility slot and attach a part that extends sight range, depending on how expensive utility slots are getting :)

As for where the raw numbers for different upgrade costs came from, I wanted to start with a base cost of 500 for a single first slot upgrade, and use that as the basis for all other costs. This makes sense since slots are central to Cogmind’s balance to begin with. The particular value of 500 also 1) gave enough room below it to fit the other lesser upgrade base costs and 2) wasn’t too long in terms of digits (even after the intended increase in costs over time) to mess with the planned upgrade UI layout. As usual, before starting any of this I’d already mocked up the UI:

cogmind_rpglike_upgrades_ui_mockup_wip

Early WIP mockup of the RPGLIKE mode upgrade menu (incomplete). Created with REXPaint.

To be sure relative costs were about right without needing much actual playtesting, I built a spreadsheet in which I could drop in different base values, cost increments, and stat modifiers to see how they would affect the progression and player abilities at each level, especially in comparison to other upgrades.

cogmind_rpglike_upgade_costs_design

Interactive chart for testing relative upgrade costs in RPGLIKE mode. Variables go in the cells with colored (non-white/gray) backgrounds towards the left and everything else is calculated automatically, including some useful automated formatting to highlight various aspects.

Without an interactive chart something like this would take forever, and although I spent a good couple hours poring over these numbers, once it was done and I went in to actually playtest, very few changes were necessary afterward.

Inventory

One of the big questions early in the design was what to do about inventory capacity? Too large an inventory would be both boring and also doesn’t really contribute much since in RPGLIKE you don’t need to carry spare parts to combat attrition like you do in the regular mode--seeing as your parts won’t be destroyed, you just need a little space for the occasional consumable, or a handful of alternate parts for special situations.

Thus I decided it would be better to fold inventory capacity into the upgrade system and have each slot be very expensive, while starting with no capacity at all!

On the technical side, we already have the Pure Core challenge mode (no inventory allowed), which automatically converts any Storage Unit you step on into matter, but that was more of a janky solution which feels weird for the more “serious” tone I wanted to retain for RPGLIKE, since it’s obviously not realistic and you can still see Storage Units as parts, and on other bots which use them.

For RPGLIKE I went a step further and made it so that Storage Units essentially don’t exist. On startup, all robots equipped with Storage Units have them removed and their base inventory capacity is increased by a corresponding amount, changing the underlying game data in the process. Then of course I had to make sure it still wasn’t possible to get them via other means, like through fabricators, schematic hacks, or any kind of random spawning, and at first that seemed like it was going to take ages considering all the possibilities…

Then I remembered I could just use an existing “item distribution” setting and automatically convert all storage parts to “unique parts that can only be placed manually by the game”, as if they belong to the ranks of very high-rating special gear, and just like that they won’t be accessible through any other means :)

Of course this approach cannot handle cases in which Storage Units are actually manually added to the map via prefabs, so for those I fell back on the Pure Core method and just convert them all to matter, which is fine because the player will never actually see them as Storage Units to begin with.

With such low inventory sizes, Relay Couplers were also given special consideration. Builds focused more heavily on bothacking will want to carry spare couplers, but even at the high end inventory sizes in RPGLIKE are generally going to be only between maybe a quarter to a third of what you normally might have, so all Relay Couplers have their value doubled.

Caps

I prefer to avoid capping upgrades if possible, but there are some upgrades which would simply be too powerful without some kind of limitation. Technically we can enforce a more “natural” limitation by having those particular upgade costs scale very quickly, but because I decided to have costs scale linearly, trying to put later levels out of range would also put even earlier/mid-level upgrades out of range, so that approach wouldn’t work as well.

Instead I had to put an artificial cap on those upgrades which would be far too powerful if players focused on them, namely damage resistances and damage modifiers.

cogmind_rpglike_upgrade_cap

Example of upgrade caps in action.

Progression

“XP” for applying towards upgrades is made available in chunks, one per level, where each level raised both requires and provides 1,500 XP.

Because I decided to have earned XP translate directly to the value used for upgrading (i.e. there’s no additional/separate unit of value awarded at each level to use for upgrading), and upgrade costs increase with specialization, thus it makes the most sense for the amount of XP per level to remain static. Yes we lose the option of having inflationary XP-per-level thresholds often seen in RPGs, but it keeps things simpler both on the inside and outside :)

cogmind_rpglike_leveling_xp_thresholds

Target approximate XP gains per depth, with corresponding values.

The total XP target for each depth comes in handy for balancing costs and XP gains. To make sure the numbers would work out, I put together a separate “build theory” table with some sample slot arrangements likely for a given depth in a regular (non-RPGLIKE) game, and had it automatically tally what this build would cost in terms of XP:

cogmind_rpglike_leveling_build_theory_costs

Build theory calculation table, as used for balancing RPGLIKE mode.

Because basics like core integrity increases and minimal heat dissipation would no longer be included as benefits of Cogmind’s natural evolution, I also tallied the total cost including those corresponding upgrades (including also base inventory capacity) to arrive at the true XP cost of a regular build. By comparing the total XP cost to what a player would likely/hopefully have at a certain depth, and taking into account the cost of optional upgrades, I could see how much leeway there would be for builds compared to the regular mode and adjust for a desirable progression and power level that way.

Sources of XP

The latest April Fools mode, “Pay2Buy,” had a system one might liken to XP in the form of earning CogCoins for purchasing parts from the store. It was based entirely on accumulating alert, since alert in Cogmind already translates to difficulty, by design, so the more trouble you cause, the more you earn, and therefore the more you also have to deal with the consequences of your actions.

For RPGLIKE I wanted to use that same system, relying on alert increases to award XP, but because this was to be more than a little temporary event, the system needed to allow for a greater variety of play styles, and therefore required a somewhat wider variety of XP sources.

I thought of possibly adding very explicit sources of XP unique to RPGLIKE, such as certain enemies or special locations, but again, variety is better here and my eventual conclusion according to my notes was that “XP needs to be from whatever actions we want to incentivize, so it should really be a combination of things.”

So in addition to alert, which already encompasses a range of actions that tie into creating mayhem, I also decided to award XP for all sources of bonus points. Most of these resemble “XP bonuses” you might find in an RPG for plot-related encounters, and also tie into ally-caused destruction, allowing allies to help you earn XP as well just like they help raise your score (important especially for those builds/runs that frequently rely on help from allies).

But the largest single contributor to XP, or at least on par with alert-sourced XP for combat builds, is actually a separate category: Exploration. One of the defining factors of regular Cogmind progression is that your core improvements come from “evolution” automatically as you approach the surface, meaning you don’t have to grind out levels or worry about accomplishing specific tasks to remain effective. This leaves a lot more room for freedom to try a huge range of play styles, so I decided that exploration is far too important to pass up as a source of XP, since anything else would end up limiting creativity.

Simply visiting different maps gives significant XP rewards, where main maps are worth an entire level’s worth of XP, while the value of branch maps varies, but is always at least a third of that for each map. A few special (or especially dangerous) maps are even worth more than one level.

After playtesting with it I was very happy with the result, and from sample runs I’ve seen by other players in RPGLIKE mode so far, regardless of style their progression tends to be at least somewhat close to the intended targets outlined above.

Level Curve

That observation can also be explained by a built-in XP adjustment that tries to maintain a tighter “level curve.” (Level progression itself is indeed linear, although if we graphed the XP levels reached by different players at a given depth it would of course result in a curve.)

To keep players from getting too far ahead of the curve, diminishing returns on XP gains are applied beyond a certain point (specifically the “target max level” for a given depth). It’s not a huge immediate falloff, and one can still get some levels ahead, but the returns do get smaller and smaller until hanging around on the same map farming alert for XP is really not worth it, not when you can head up to the next depth and earn those same levels much more easily.

I also wanted to prevent some players from falling behind the curve, but decided this wasn’t necessary in Rogue mode (the hardest difficulty setting), so in Adventurer and Explorer difficulties the reverse of the diminishing XP formula is applied to XP gains by players who are below the minimum target level, allowing them to more easily catch up if they’ve somehow fallen too far behind.

(Here I’d like to share some sample graphs of level curves from our data, but sadly although the data has been collected, the leaderboard system to access it has not yet been completed…)

Timing

The upgrade menu is accessible at any time.

I originally thought of doing upgrades during Cogmind’s normal evolution process, but realized 1) this would require two new UI windows and be more complicated to implement and 2) was clearly thinking too much “inside the Cogmind box.” I mean it should be like most CRPGs where you can get your new goodies at any time, right? :)

So in RPGLIKE, whenever a new level is hit there’s a notification and the menu is always right there. XP can even be saved for later, accumulated for an upgrade that’s become too expensive for a single level, or for other strategic purposes.

Starting Conditions

Naturally for a mode as different as this, it makes sense to have unique starting conditions. It’s also an opportunity to add a bit of CRPG-like “character generation” right from the start.

Players start with a chunk of initial XP (more for lower difficulties) that can be spent immediately on upgrades to start defining their build. The base amount in Rogue mode is set at just enough to create what is more or less a normal starting loadout:

cogmind_rpglike_upgrades_regular_start

A Rogue mode RPGLIKE start with upgrades defining a standard starting build (7 slots + 4 inventory), minus one inventory slot since they’re quite expensive in this mode but also not nearly as important.

Starting core integrity is higher than usual, since that’s what takes the brunt of attacks instead of parts, unlike the normal game.

The base starting build before any upgrades is only 4 slots! This is the first time a Cogmind build has the option to only ever rely on a single Propulsion/Utility/Weapon slot. (Because I was curious and wanted to try it out, I actually did a complete 4-slot, upgrade-only, run recently :P)

There’s also no inventory capacity by default, but again it’s not as important in this mode, so most people will/should start with fewer than 3 upgrades there and reallocate the significant XP savings elsewhere on multiple cheaper but effective upgades.

UI

The upgrades UI appears in the same location that Pay2Buy did, and resembles it in some ways, but because the latter’s expanded form actually made use of a generic list class that I built for use throughout Cogmind, while the upgrading would require many more unique interactive features, I had to write this new one from scratch. No problem, I do rather enjoy designing and building interfaces :D

First, because it could be easy to miss the fact that you’ve raised a level in the first place, especially because it often happens when other things are going on, aside from the inevitable message log event there’s also extra feedback in the form of a unique sound effect and blinking indicator above the level/XP summary. I don’t want any of Cogmind’s notifications to be modal or anything quite so interrupting/in-your-face/immersion-breaking, so that’s as far as I went.

cogmind_rpglike_raised_level

Demo of raising a level and assigning a few upgrades with the mouse.

As usual I always want everything to be fully accessible via both mouse and keyboard, and in the case of the latter there are even three different input schemes. The standard Cogmind approach is of course lower/upper case characters for increasing/decreasing their respective values. Also with lists like these it’s nice to be able to navigate directionally, so both arrow keys and numpad can be used to move the selector up and down, with left and right incrementing and decrementing the current value.

cogmind_rpglike_selecting_upgrades_keyboard

Demo of pure keyboard control in the RPGLIKE upgrades interface.

As you can see references for key commands are built into the interface itself, making them easy to learn. The “Up/Down/Left/Right” in the corner isn’t really attached to anything, but players will generally intuit that it refers to key commands and try them out, seeing the effect it has on the UI.

Probably the biggest challenge with a UI like this is that upgrades cannot actually be applied until confirmed, but before that point it’s still necessary to show their state, modified cost, limits, etc. given the desired upgrade(s). This can get somewhat complicated, and also potentially require a fair amount of duplicated code for various purposes.

In the end I used a solution that simply retrieves direct memory references for respective stat values, which can be used to both query the current base value (and temporarily modify it as necessary) as well as apply each modification later once confirmed. The amount of code involved ended up being quite small, but could have been larger if the upgrades were more complicated than simple integers (and integers where positive values are always good, at that!). I did actually remove one planned upgrade partway through implementation because it would break the elegance of this system :P

Classes

Character classes are a staple feature of most RPGs, but at odds with Cogmind’s normal free-form build-as-you-go approach. This being “RPGLIKE” mode, it sure would be nice to fit classes in there somewhere…

Fortunately Cogmind already had a built-in feature that would help here: the automated build classification recently added with Beta 9. It analyzes your current build and assigns it a base class name with maybe a special modifier, but must be activated manually (doesn’t appear by default) because that particular feature didn’t really fit with the type of immersive atmosphere I wanted to create for Cogmind. Regardless, it’s certainly appropriate for a mode like this, so I made it active despite whatever the advanced config file says. A fair number of people who didn’t know about it before will probably even think it’s unique to RPGLIKE :P

cogmind_build_classification_test3

Sample “Skirmisher” class, a combat bot supported by infowar utilities. Se more samples and a breakdown of how the system works in Building the Ultimate Roguelike Morgue File, Part 3: Mid-run Stat Dumps, under “Build Categorization.”

The classification system does take into account the effects of RPGLIKE upgrades when deciding on a class, which took a while to merge with the classification system and wasn’t even present in the earliest prereleases since I wasn’t sure I’d have enough time to get around to it, but I’m glad I did because I think it’s pretty important for accuracy purposes given that many of the upgrades would be taking the place of various slots/parts.

cogmind_dominant_class_samples

Compound class types from a Cogmind run as they appeared in the Dominant Class section of the scoresheet.

“Healing”

Exactly how to restore core integrity would be a pretty central point of RPGLIKE gameplay, but I still hadn’t even figured out how it would work while already building the UI and setting upgrade costs. Since the first “purely RPG” ideas like resting-to-recovery or gradual recovery couldn’t really work balance-wise in Cogmind, it seemed I’d have to wait for a better solution to pop up, and at least there’d be spare time in between working on the other features to mull over different possibilities.

Healing over time would have clearly been boring in so many ways, not unlike it can be in some other roguelikes! But at least these other roguelikes are specifically designed around individual encounters, where you’re often at full capabilities for each one and when you go into a given battle, it’s you vs. that current enemy/enemies which you’ll probably either squash or which might have a chance to kill you if you make too many mistakes (maybe even just one mistake xD).

Cogmind has always been more about attrition and tactics vs. a much larger complex and numerous potential enemies, which individually don’t really pose a threat to you. So that whole central aspect of the game (and all the content and balancing associated with it) would have to change for rest/gradual-healing to be feasible, and as I mentioned earlier that’s outside the scope of this mode, and also unnecessary anyway since there are other methods.

Another problem particular to healing over time is that Cogmind doesn’t have the consumables most other roguelikes do in order to handle short-term variance, consumables like… health potions! It’s from here that I started taking a new approach to the idea of integrity restoration, and realized we’d need a whole new item to do it.

Protomatter!

Thus Protomatter was born, a whole new item permanently added to the game for a special mode, and in that sense the first of its kind. (One other event, the holiday mode from 2018, also added items, but that mode no longer exists so neither do the items associated with it.)

Protomatter is basically like a health potion, dropped randomly by some combat-capable enemies instead of regular Matter, and used to restore core integrity (or if one’s core is already maxed, it repairs any damaged parts).

I definitely didn’t want it to be like health potions in other CRPGs where you can simply chug them when in danger for an instant boost of HP, possibly even up to full. Having it take time to use would be more appropriate for Cogmind, and add a bit to the tactical decision of when and where to use it.

The original implementation (that patrons tried out in the first prerelease) was simpler but quite funny: It determined how much of the Protomatter was required for a full heal (or to use all of it even if not enough for a full heal), then calculated the required number of turns to do that, and applied it all as a single uninterruptible action. That was by far the easiest way for me to implement it rather than spreading its use over multiple turns, but the way the numbers worked out this meant you could in some cases end up sitting there applying Protomatter for 15 or more turns! One can imagine the potential consequences of such a long action--enemies you didn’t even know were there yet might round a corner and start shooting you to pieces :P

Instead I ended up having to take the more complicated route and have it apply on a turn-by-turn basis, with pacing determined by the player who could simply wait while standing on top of it in order to apply the next batch. Some of the complexity came from the application process itself (mainly where it involved attached parts), but also simply from properly handling the messaging system so it would show the desirable results once you’d finally finished applying Protomatter, rather than spamming the message log with too many individual messages.

At first I wasn’t sure what to call this item, but it soon became obvious it should be related to Matter because:

  • Protomatter would similarly be a pretty fundamental item throughout a run
  • Protomatter would sometimes be dropped as salvage (instead of regular Matter) (note that to avoid destabilizing the existing balance, all Matter drops were increased a bit as well)
  • Like Matter, it’s affected by salvage-modifying attacks (although unlike Matter it can never have its amount increased by a positive net salvage modifier)
  • I would need a tile for this new item, and the Matter tile would be a good choice here since it’s otherwise only used once and doesn’t fall into any other category
cogmind_rpglike_protomatter_drop

An enemy Programmer dropping salvageable Protomatter instead of regular Matter.

At the same time, I wanted to also differentiate it from matter because it not only has a different purpose but would also have different fundamental behavior, so I gave it a unique naming scheme, “Protomatter-XX” (e.g. Protomatter-56, where 56 is its remaining amount).

Protomatter behaviors that differ from Matter:

  • Can be picked up as an item and held in inventory (not in containers like Matter), important for allowing some backup repairs depending on how much inventory capacity you have and want to devote to this purpose.
  • Suddenly decays randomly, disappearing from the map (unless in inventory or currently under the player), as it’s important to prevent stockpiling Protomatter on the map itself, since in the right hands that just makes it easier to become invincible.
  • Cannot be merged or stacked, because that would also make it too easy to stockpile unless some kind of arbitrary limits were put in place, but may as well do with that altogether since it just unnecessarily complicates things.

On the topic of naming, the first options considered for a moment then thrown out were to just jokingly call these Health Potions, but even though it would be really in theme for this event, RPGLIKE is meant to be a more serious mode so I had to come up with something better. “HP Kit” or “HP Unit” were similarly passed over :P

cogmind_rpglike_protomatter_info

Protomatter info.

Damage Transfer

While working on the new “healing” mechanics, I decided this was an opportunity to change the initial idea that all incoming damage would go straight to your core, practically doing away with item destruction entirely and behaving more like your average roguelike with RPG-like progression.

Reducing the role of item destruction is definitely one of RPGLIKE’s primary goals, but completely removing the risk of that happening felt like it would unnecessarily gut a lot of the feeling of Cogmind--your parts always appearing in perfect shape would be fairly awkward, and we can get the anti-item destruction effect without going to that extreme, by just redirecting most of the part damage to the core, instead of every last bit. This also helps retain some of the coverage-related balance inherent in Cogmind’s part stats (there is definitely noticeable destabilization, however--that’s unavoidable).

So 80% of damage to parts goes instead to the core (it’s funny because there’s actually a part in the regular game which has this exact effect, although naturally the strategic implications are quite different there due to the lack of frequent core restoration). Even though it seems you don’t really lose many (or any!) parts in this mode, it still feels more natural if they’re taking a bit of damage from attacks, and it could still come into play if you’re playing very poorly or incorrectly managing Protomatter reserves and acquisition.

Honestly I think we could probably do with a lower percentage than 80, at least for Rogue mode, and it would make RPGLIKE more challenging while still retaining its uniqueness., but that’s a slippery slope that leads right back to the default Cogmind experience ;)

While on the subject of healing, I should mention that several times I considered allowing XP to be used to repair current core integrity as separate from the current/max integrity increase that it ended up as, but ultimately decided against allowing that since upgrades were meant to be instantaneous and that would be just like the “emergency chugging of potions” I was trying to avoid.

Records

As a more permanent official mode recommended for some players, RPGLIKE also got exclusive treatment as far as the scoresheet is concerned.

The history log records slot upgrades (since they’d normally be reflected in evolution messages but those won’t happen in this mode), as well as each level gained (since that aspect of progression is pretty important to the mode and can be interesting to see in the context of other events along with the turn counter).

cogmind_rpglike_history_log_excerpt

Excerpt from an RPGLIKE run scoresheet’s history log, showing level raising and slot upgrades.

RPGLIKE also gets its own section in the scoresheet, recording many other details on a per-map basis.

cogmind_rpglike_scoresheet_section

Sample of a dedicated RPGLIKE scoresheet section.

What’s Next?

Could RPGLIKE mode be better? Yes, in many ways! I mean this is no surprise given that it’s a mode designed on top of another game and basically turns it on its head :P

I do plan to still do some more work on RPGLIKE mode, or at least tweak it. Precisely how to improve it also really depends on the goal of doing so, since what constitutes an “improvement” is subjective, and in its current form it’s most definitely not the primary way I would want to play Cogmind.

I mean I’ve been having a lot of fun with it, as you can see from the already several streams I did in this mode:

  • RPGLIKE Intro Run -- An introduction to the new mode plus a relatively quick but complete run based around one of the overpowered EX-tech parts I’d never used before (spoiler: we torch 0b10)
  • RPGLIKE No-crutches Run w/Kinetics -- No FarCom, no RIF, no imprinting. Just a pure kinetic combat build out to shoot anything that gets in its way.
  • RPGLIKE 4-slot limit “Babymind” run -- No slot upgrades allowed, just 4 slots total straight to the end. We go mostly melee, a bit of hacking, and using allies to do much of the fighting. (Several big guys fight on our side because FLK :D)

An introduction to the new mode plus a relatively quick but complete run based around one of the overpowered EX-tech parts I’d never used before (spoiler: we torch 0b10)

But it’s far too easy. For the first iteration this was intentional though--I guided it towards the easy side for a couple reasons:

  • It’s a holiday event! For something like this it’s better to aim for fun over anything else. After all, it’s optional and not like it removes the regular mode, anyway.
  • In games there’s often a fine line between too easy and too hard when you have a healing system, certainly much finer than Cogmind’s usual death by attrition and an accumulation of long-term mistakes rather than one or two recent mistakes. Designing a healing system that can ride that line more closely would take a lot more theorizing and playtesting than there was time for. Heck it might even be impossible given Cogmind’s other mechanics.

My personal main goal for RPGLIKE’s theme was to add a system of permanent progression and all but eliminate part destruction, and it satisfied those goals well. It will never be heavily balanced like the regular game--that would require too many changes, but it’s still fun either way, and at least the challenge level can be tweaked. Those who enjoy this mode aren’t necessarily looking for complete balance anyway, at least it doesn’t sound that way from all the feedback :)

Bonus: RPGLIKE Strategy Notes

While playtesting RPGLIKE I was noting down various strategies and/or observations for later reference:

  • It seems to get easier over time, as is generally the case with other “RPG-like” roguelikes but quite the opposite of regular Cogmind (which tends to get harder as you progress)
  • That said, you also gain levels very quickly at the start (plus have extra XP to spend on upgrades you normally wouldn’t have) and as long as you can ensure you don’t destroy most of the Protomatter out there, it’s unlikely you’ll actually die.
  • Either way, you generally feel a lot more free to be reckless because damage can generally be repaired, and mistakes aren’t going to be as costly in the long run like they normally are.
  • Inventory capacity is going to be low throughout the run, but that’s okay because you don’t need spares. It becomes more of a simple storage for items needed in alternative situations, or even consumables (“health potion” Protomatter, and in one of my runs I had a PSU Rigger so I just kept filling my limited inventory space with nothing rigged power sources :P)
  • Can use more/different types of items you might not otherwise normally keep attached, since they won’t break! This is especially useful for unique parts that cannot normally be replaced.
  • There’s little reason to use anything slower than hover unless you want siege mode or other tread benefits, because propulsion is much more protected in this mode anyway, and even mass support can be gained via upgrading.
  • Repairing core and parts is separate, and core is often more important, so it can be advantageous to save your Protomatter specifically to repair core rather than worrying about parts, which can be replaced, after all. However if you’ve upgraded core integrity quite a lot and then lost much of it, getting enough Protomatter to repair both core and parts can become difficulty, putting attached unique parts at risk.
  • Because you generally have few to no spare parts (since you don’t normally need them, or can’t spare any inventory space), cave-ins are extremely dangerous, even more so than usual. To a lesser extent the same goes for other effects that remove and/or directly damage parts such as Blade Traps, Saboteurs, etc). The most dangerous effects are technically avoidable, though.
  • Honestly in it’s release state it’s kinda like a wizard mode for testing the effectiveness of builds, at least in an offensive capacity, since you can try them in different scenarios without fear of inevitably losing them after a period.
  • Can save XP for later when you really need it, either to buff some ability that will be of immediate use, or to increase core integrity in a pinch.

Some of these may no longer quite be as applicable after updates, but they’re relevant as far as the Winter 2019 event goes ;)

Addendum: Beta 9.4 Updates

Not long after this article went up I released Beta 9.4, mainly as an update for RPGLIKE mode. It didn’t involved any sweeping changes, and you can read about some of its other tweaks in the linked release notes, but here I did want to at least mention that the role of armor-type parts as protective gear was restored for Cogmind.

The solution was to simply have armor absorb more damage than usual (40% instead of 80%), turning it into a sort of consumable in the RPGLIKE world of otherwise far less consumable parts. This means builds can use armor as a way to avoid the need to rely on acquiring as much Protomatter.

Further changes will contingent on the size of RPGLIKE’s post-event player base and the data we see from these runs.

Posted in Design | Tagged , , | Leave a comment

Leaderboards and Player Stats with Protobufs

After finally ripping out the old leaderboard and metrics system, it’s time to build a new one!

I’d technically already completed the first significant chunk of the new system months before, described in the Ultimate Roguelike Morgue File series. In particular the “Preparation” section in Part 1 describes how I created an external file to store all the scoreTypes and their associated parameters. This will again become important later in the process.

Now as much as I love the new text format for scoresheets, we’re certainly not going to be uploading text files anymore. The new online data system needs to be elegant, more compact, more secure, and easier to manage overall, not just a bunch of text files that are uploaded by players then batch downloaded and manually run through a dedicated analysis program :P

Protocol Buffers

The first step on this new route is to decide what format to store the scoresheet in. Technically Cogmind’s score data already has its own binary format, a well-organized one that I use to keep it in memory and, of course, compressed in the save file itself. And it works well for this purpose, but it’s just my own format, and therefore not widely compatible or as easily usable outside Cogmind. So we need an additional format for scoresheet data, one that will be more suitable for upload, storage, and manipulation.

Will Glynn, the networking pro helping me with the server side of this new adventure, suggested we use protocol buffers, also known as “protobufs.” Protobufs are apparently a popular solution for this sort of thing (and storing data in general!), with a number of features we’ll find useful:

  • Protobufs are a compact binary format, which is important considering the sheer amount of values stored in the scoresheet for a single run--up to tens of thousands of data points and strings!
  • Protobufs can be easily converted to other formats like JSON (in fact they’re kinda like binary JSON data to begin with), and be parsed by a bunch of existing libraries for analysis or other uses, not to mention the accessibility benefits of support across many languages.
  • Protobufs are designed to be easily extensible while remaining backwards and forwards compatible. As long as no data identifiers are changed, we can add new data to later versions of the game and these scoresheets can be uploaded normally, even if the server doesn’t yet understand them--the ability to interpret new data can be added later if desired, and the data will still be there all the same. This makes it easier for me to put out new versions that extend the scoresheet without resetting the leaderboards, which I couldn’t do before since the analyze_scores.exe I talked about last time, used to build the leaderboards and produce stats, expects everything to remain unchanged from when a particular version was released. (This is why the leaderboards prior to Beta 9 were only ever reset with major update releases, and scoresheet updates always had to be postponed until that happened, even if minor updates could have benefited from including new entries.)

Setting up Protobufs

Protobufs it is, so let’s set it up! The docs are pretty good, and include API references and tutorials in a range of different languages, including C++ (which I’m using).

The first step is to download the library from GitHub, and in my case I had to use a slightly older version, the last to support Visual Studio 2010, since that’s what I’m still using to develop Cogmind. It includes a readme with setup instructions for both the protobuf library against which to link the game, and for “protoc,” the protobuf compiler, which is essentially used to read the game’s specifications for the protobuf content and generate a header-source pair of files for reading and writing the data from and to that format.

The data format is initially defined in one or more text files with the “.proto” extension, and anyone with that file can read and write a proper protobuf scoresheet. Cogmind just has the entire scoresheet data for a single run defined in one 1,811-line file.

cogmind_scoresheet_proto_excerpt_route

An excerpt from Cogmind’s scoresheet.proto. Notice the ID numbers associated with each entry. Once set and used, those can’t change if you want to retain compatibility across versions, since from that point on they identify the specified data type. Protobufs support structures (“messages”), nested structures, enums, and common data types like integers, booleans, strings, and more.

Protoc reads that file and turns it into code, altogether generating functions totaling 2.85 MB. And that’s with the .proto file specifying “option optimize_for = CODE_SIZE”! This results in lighter code at the cost of runtime performance, but Cogmind isn’t making heavy or wide use of Protobufs, just for a one-time upload of the run data, so tight performance isn’t a requirement. (Out of curiosity I compared the results, and without CODE_SIZE the protobuf files come out to 5.34 MB. Cogmind’s entire source itself is only 7.79 MB xD)

cogmind_source_protobuf_proto_generatedpng

An excerpt from the 22,948-line pb.cc file generated by Protoc, corresponding in part to the .proto file section shown before.

Loading Protobufs with Game Data

We have a format, we have the code, now to put the real data in the format using that code :)

Again the docs came in handy, although there’s more than one way to do it and the tutorial doesn’t cover them all so I tested several approaches and eventually also looked around at several other websites for some use cases and inspiration before settling on a final method.

In my own projects I like to use enums for everything, making it easy to write code for loading, moving, comparing, and retrieving large amounts of data, but protobufs have a separate set of functions for every. single. piece. of data. xD

This complicates things a bit, and at first I tested the idea of using macros to set values by retrieving a pointer to mutable protobuf data, or assigning a new protobuf object, where the macros are used to call the necessary function names.

cogmind_source_protobuf_loading_tests

Testing different methods of using the protobuf API to load values.

In both cases those code sections are merely populating values for the relatively simple Performance section of the Cogmind scoresheet, which you can see below.

cogmind_scoresheet_proto_excerpt_performance

Cogmind scoresheet.proto excerpt defining Performance data, and an inset demonstrating what that message represents in the scoresheet itself.

Looking back at the macro-powered source, either of these methods works fine if there’s a little bit of data--who cares, just get it done, right? But we’re going to need to also come up with a more efficient way to do this, one that doesn’t involve writing a ton of code for the huge amount of data needed to represent a scoresheet.

Taking the scoresheet’s Bonus section as an example, it currently contains 74 entries…

cogmind_scoresheet_proto_excerpt_bonus_partial

Cogmind scoresheet.proto partial excerpt of Bonus message.

Writing out all the code to load all these, even if just one line each with the help of macros, is kinda excessive, not to mention having to update them any time new ones or added or existing ones are modified in the game. And this is just one section! What if we could just have it update automatically any time there are changes?

There’s a much easier way that relies on the fact that I already have both enums and strings associated with all of these entries: Reflection. It’ll be slightly slower, but again the performance aspect is negligible because loading protobufs is not something the game is generally doing.

This bit of code loads the entire Bonus section, and doesn’t need to be touched in the future even if the section is expanded with more entries:

cogmind_source_protobuf_loading_reflection

Loading a scoresheet protobuf via reflection.

FindFieldByName() is our friend, a protobuf method that does exactly that, and I just have to feed it the right name based on the scoreType in question, properly converted from my own string format to that required by the protobuf API via protobufFieldName().

This approach works great for the majority of scoresheet sections, but unfortunately there’s still one (rather big) roadblock: the stats section storing separate sets of data for every map.

Now technically if I stuck to having the scoresheet.proto format reflect the internal data structures, all of the stats could’ve easily been handled in the same manner, and if it was just me working on this project that’s probably how it would’ve ended up, but Will convinced me that for the online system we needed to store the run data differently than I was doing it internally.

In short, this meant the stats section is going to need… a lot of code. Fortunately all of the relevant scoresheet data is defined in the game’s external text file described in Building the Ultimate Roguelike Morgue File, Part 1: Stats and Organization, so it can be used to generate the code to convert it. Yay, more generated code xD

So it turns out that part of scoresheet.proto--the messages related to general per-map stats, is generated by a script, and the source to load it at run time is also generated.

cogmind_source_protobuf_loading_stats_generated

Excerpt from the 762-line generated source file that loads all per-map stat-related messages and their submessages (and sometimes submessages of those submessages!) based on the game stats recorded at the time.

Here I should point out that all this last-minute large-scale conversion stuff could’ve been avoided if Cogmind simply used protobufs natively for all of the scoring data and stats rather than a separate format, which is what most people would probably do with protobufs. With that approach, when it comes time to upload everything is already packaged and ready.

There are a number of reasons I didn’t do that:

  • For internal use I prefer a stat-first organization over what was decided would be best for the online data (route-first and embedding a lot more meaning into the format structure itself).
  • It would’ve been far more work to go back and change how everything works, as opposed to just converting the existing data when necessary once at the end of a run.
  • I didn’t want the protobuf library to “invade” the rest of the code, so as an optional part of the architecture it can easily be disabled or removed.
  • The internal scoresheet data was originally built and maintained to create the text version of the scoresheet, so it’s more suited to that purpose than using the protobuf content as a starting point.

But anyway, that’s just me and this project. I’d still recommend protobufs as a pretty good solution to serve as the foundation for this sort of thing, or potentially even entire save files depending on the game!

Uploading via WinINet

Last time I described how for years I’d been using SDL_net to upload scoresheet files, but that’s not gonna cut it anymore. I defer to Will on networking matters where possible, and he said we need HTTPS, so we need HTTPS… Just gotta figure out how to do that xD. He listed some options to get me started, though for me it’s always a huge pain (or complete roadblock) to simply add libraries and get basic stuff actually working at all, so I wasn’t too confident in my ability to implement them.

gRPC

One seemingly particularly good option would be gRPC, since it’s been built to integrate with protobufs in the first place. Unfortunately it’s not compatible with Windows XP, which Cogmind still supports, and I think it would also require upgrading my project to VS2015 or so. While it’s possible I’ll eventually drop XP support anyway and probably one day upgrade to a newer version of VS, there’s no reason to do those things now if there’s a workable alternative, so I passed on this option.

libcurl

I was familiar with libcurl from my research years back before deciding on SDL_net, plus I’ve heard it recommended by other devs, but setting it up was nowhere near my idea of easy. Although I got pretty far in the process, it seemed that for SSL support I also needed to include OpenSSL, which is built against VS0217. Regardless, I eventually got it down to only one remaining linker error that I didn’t even understand, and ended up giving up on libcurl.

I’m also generally against adding a bunch more heavy DLLs to the game, especially for one little feature, another strike against libcurl. (Adding just protobuf support already increased the size of COGMIND.EXE by 15% xD)

WinINet

What better option than to cheat on download size and use native Windows DLLs? :P

Will couldn’t recommend WinINet, but I’m glad he listed it as a last resort because it’s right up my alley. Windows is good with backwards compatibility, so that also makes it more likely that I’d be able to use it easily despite using relatively old tech compared to so many of the other networking libraries out there. Remember, I’m not writing some persistent online application, or trying to do any kind of multiplayer networking--all I want is the occasional HTTPS data transfer, so it would seem kinda ridiculous to use anything more than the bare minimum necessary to achieve that goal.

For this option all I had to do was simply link “Wininet.lib” and start writing code! Love it :)

Well, the code part wasn’t as straightforward as I’d like. As with SDL_net, how to actually use WinINet properly took a little while to figure out, even more so since we needed SSL. The example in the docs was a helpful start, but didn’t cover everything I needed to know in order to conduct a successful transfer. I eventually came up with workable code by mixing and matching various solutions found on the net.

Of course it needs a nice C++ wrapper to simplify everything:

cogmind_source_wininet_https_connection_header

The header for a simple C++ class capable of handling HTTPS connections, along with a sample of how to use it (as of now that test will actually retrieve the latest Cogmind version number and news). The full header can be downloaded here, and the source here (file extension needs to be changed to .cpp--it was changed for hosting due to security reasons).

Altogether it’s less than 200 lines, and in addition to that test above, as the class was developed I ran a series of simple incremental tests until eventually reaching the point where I could encode and upload an entire scoresheet in protobuf form. (I’ll show an example of this further below when we get to crypto signing.)

Storing the Data Online

These scoresheets are uploading… to where, now? I can’t get into a lot of detail here, because much of it is currently a black box to me. The network side is Will’s area, but here are the basic steps I needed to finish hooking everything up:

  1. On gridsagegames.com I created a new subdomain that points to a Heroku service. This is easily done in cPanel with a simple CNAME record.
  2. I set up an AWS account which is where the Heroku service built by Will stores all the uploaded scoresheets for processing/reference, in an AWS S3 bucket.

For me, from there it was just a matter of using HttpsConnection to send a scoresheet and watch it appear in the bucket \o/

cogmind_AWS_data_first_scoresheet_upload

The first ever Cogmind scoresheet successfully uploaded to AWS S3, as seen in the console.

Although there probably won’t be any issues with the data on AWS, you never know what can happen and backups are essential, so in exploring backup options I found I can use the AWS CLI to download everything in the bucket to my local computer (instructions), from where it would also be included in my own automated offsite backups.

cogmind_s3_server_data_download_CLI

Downloading Cogmind score data via AWS CLI.

I had just set up a script to do that automatically every day, and of course I mention this to Will and he immediately writes a program to both do this as well as limit downloading to only new files, and zip everything up xD

cogmind_s3_server_data_download_program

Downloading Cogmind score data via AWS CLI. This was about a week after Cogmind’s first public version including the new scoresheet system.

Additional Features

Since the leaderboards based on this new system weren’t ready when Beta 9 released, but in the meantime I wanted people to at least have a chance to see the new data as it appears online (while being able to easily share their complete run info without sending a file), I had the idea to append URLs for their run data on the server to the end of their scoresheet once it was successfully uploaded.

cogmind_scoresheet_urls

URLs for a run’s data, in TXT and JSON formats, as appended to the end of a scoresheet.

See a live scoresheet sample here in text format, which is identical to the local version, and the same scoresheet data here as JSON, which essentially mirrors the protobuf format.

Depending on your browser, the JSON file might appear as a barely readable mess, or if your browser knows what it’s doing and has built-in functionality for parsing them there will be formatting options for readability and/or interaction.

cogmind_scoresheet_json_data_sample_firefox

Displaying the sample run data in Firefox in raw JSON, Pretty Print, and the interactive view.

Reuploading

One of the features missing from the old system which I kept waiting for the new system to actually implement is how to avoid losing potential run data. What I mean are those times that the website server happens to be unreachable for some reason, or maybe the player is currently offline or their own internet connection has issues. From then on the only way that score/data could ever make it to the leaderboards/stats is if they gave me the file and I manually uploaded it, which I did do for special runs by players on request, but now that we have the real, final system, this calls for something automated.

In cases where the player has uploads activated in the options but the process can’t be carried out or fails for some reason, the complete protobuf data is archived in a temporary subdirectory in their local scores folder. Each time Cogmind starts up, it checks that directory for scores that failed to upload, and tries them again. This saves time, saves headaches, and ensures we have all of the runs that we should! Now I won’t feel nearly as bad if the service is inaccessible for some reason :)

Cryptographic Signing with libsodium

My original dumb I-don’t-kn0w-networking approach for identifying the player associated with a given scoresheet was going to be to upload that scoresheet’s data alongside their player GUID, which would not be part of the public protobuf format. As a private value, it would prevent a malicious actor from spoofing another player and messing up their historical data, and our aggregate data along with it.

Unfortunately, keeping this unique identifier private would also decrease the accuracy of third-party data analysis (which, as we’ve seen, is something a number of Cogmind players like to do!).

At Will’s suggestion, however, there’s a way to add the same extra layer of security without hiding any relevant scoresheet values: have players sign their uploads with a local private key! That enables us to safely ban specific players as necessary since it’s no longer possible for someone to pretend they’re someone else, making it easier to ensure the integrity of the leaderboards and stats.

So what does it actually mean to “sign” something… I had no idea, either, but the libsodium docs explain it all pretty well, including examples of how to do it. Super simple. libsodium is definitely my type of library! Just drop it in, write a few lines of code following their examples, and enjoy the results.

Players store their own private key locally, and the public key (more or less like a GUID, insofar as it’s a unique identifier) is included with the protobuf along with a signature created based on the private key and the data to be transferred. The online service checks that the provided public key properly matches the data signature, to confirm that the data was, indeed, uploaded by the player they say they are, and then add that data to the AWS S3 bucket. If a data transfer fails that check, it’s ignored.

Here’s the code for using HttpsConnection from earlier to upload a signed protobuf scoresheet. It took me a while to figure out how to get the protobuf data in the right format for transfer over HTTP, so maybe this will help someone in the future:

cogmind_source_scoresheet_uploading

The function for uploading a signed Cogmind scoresheet to the server, using libsodium and the WinINet wrapper from earlier..

Also note that as per the libsodium docs there are two different ways to use signing, either “combined” or “detached.” The former actually inserts the signature at the beginning of the data itself, whereas detached mode allows you to send the signature separately. The sample use case above is using the detached mode, inserting the key in the HTTP header instead of messing with the protobuf.

And there you have it, the gist of Cogmind’s new handling for network-related stat stuff! It’s been a long time since I added someone to the Cogmind credits, but I couldn’t have done this without Will, so there he is :)

cogmind_credit_menu_leaderboards_rework

Cogmind credits page, 2019 version.

The AWS web interface itself only allows us to get a glimpse of what’s there (psh, viewing 1 to 300…), but there are thousands since the Beta 9 release.

cogmind_AWS_data_lots

Browsing the collection of Cogmind scoresheets in the AWS S3 web interface.

Next up is the new leaderboards based on all this data… And I’m still not sure how I’ll be doing the usual data visualization and statistical analysis now that everything is in a new format, but I’m sure it’ll work out.

Posted in Gamedev | Tagged , , , | Leave a comment

Leaderboards and Player Stats: The Old Way

It’s the end of an era! From May 2015 through October 2019, Cogmind used the same architecture for score uploading and leaderboard compilation that I’d patched together a few months before releasing the first alpha. That’s all changing as of Beta 9 and the introduction of a revamped “Ultimate Roguelike Morgue File,” so I thought I’d use this opportunity to share a rundown of how the old system worked before diving into the new one.

I must admit the old system was pretty dumb. I’m generally a proponent of brute forcing things using just what I know or can easily figure out and taking the simplest possible approach unless that really won’t work, and it happened to more or less work here, so I took the path of least resistance and it stuck that way for years.

Being terrible at web dev and and having trouble wrapping my head around networking and databases and whatnot, for this period I opted for a human-in-the-middle approach that can be summarized as:

  1. Copy scoresheets submitted by newly completed runs from the temporary directory to where they’ll be referenced by the leaderboards (the earliest versions of the leaderboards only offered name, location, and score, but this was expanded to link directly to the full scoresheet)
  2. Download all the new scoresheets to my local machine
  3. Run analyze_scores.exe to read in all the scoresheets and output the HTML data for the leaderboards
  4. Upload the latest leaderboards HTML file

I’m sure some combination of scripts could’ve handled this process automatically (and I did investigate and consider the possibilities), but I also justified keeping this workflow for years because having me in the middle for less than a minute each day at least allowed (forced) me to keep an eye on progress like how many runs were being completed each day, while also remaining vigilant in case something went wrong (this happened several times over the years as occasional bugs popped up, and I was usually able to respond before anyone else saw them because anything out of the ordinary had to make it past me first :P).

So literally once every day for 1,620 days I did the file copying and ran analyze_scores.exe on the data to produce the latest leaderboards. There were a couple days where I forgot or wasn’t able to update on time, but they were few and far in between.

I appreciate the many offers of help I got over the years (since a lot of experienced web devs happen to play Cogmind :P), but I wasn’t ready to take on help in this area when there was clearly lots more development to happen and changes to be made, and making these modifications alongside someone else would significantly slow the process.

I wanted to wait until closer to the end of beta when the scoring data was more stable and the format was finalized before doing something about it. “Doing something” being automating the process and making the whole architecture a lot more robust. For me, if long-term efficiency is the goal this sort of thing is best done once the specifications are clear and we’re past the point of making sweeping changes.

So Many Scoresheets

Over the years using this method we accumulated 89,904 scoresheets, a total 1.0 GB of raw text data.

cogmind_scoresheet_submissions_per_release_2015-2019_by_count

Cogmind scoresheet submissions per release, ordered by count (2015~2019).

Organizing the data chronologically probably tells a better story…

cogmind_scoresheet_submissions_per_release_2015-2019_chronological

Cogmind scoresheet submissions per release, ordered chronologically (2015~2019).

A number of factors affected uploads over the years and across different versions, contributing to a lower number of scoresheets than we’d have otherwise. Here are some of the more significant ones:

  • There were a few weeks in early alpha where I’d forgotten to actually save the scoresheets to my own records before wiping the server data for a new release, so while those leaderboards are still available, the underlying scoresheets have been lost to time.
  • We lost maybe a couple thousand runs shortly after the Steam launch in 2017 with Beta 3. Here I learned that storing this many individual text files on a web server is not something you’re really supposed to do, because they generally have a limit of 10k files in a single directory xD. I ended up having to quickly patch in support for multiple subdirectories, although this only affected by own analysis and data organization, not the game itself which could continue uploading to the same temp directory.
  • Score submissions were originally opt-in throughout Alpha, then later changed to opt-out when we joined Steam (to collect data from a wider group of players and get a clearer picture of the real audience), but then not long after we had to switch back to opt-in again starting with Beta 6 due to the new GDPR concerns.
  • The other reason behind the large drop heading into Beta 6 is that uploads were previously accepted from anonymous players, a practice that also ended with GDPR. Of all the scoresheets collected, 37,547 were anonymous (41.8%), though normally anonymous uploads accounted for two-thirds of the total within a given release, so if we were still accepting them now I’d expect to have about three times as many submissions from recent versions as we do now.
  • Earlier this year during Beta 8 the server went down for a couple weeks and wasn’t accepting score submissions, and even after being fixed there was still an issue that prevented some scores from uploading.

In terms of getting a clearer picture of the changes over time, even the chronological graph is somewhat distorted because it’s built on a per-release basis, even though some versions lasted only a month, while others persisted for several months or more. (Leaderboards are reset with each new release, rather than on a set time schedule.) Here I’ve compiled the monthwise scoresheet data, and annotated it with the above factors:

cogmind_scoresheet_submissions_2015-2019_monthly_annotated

Cogmind scoresheet submissions per month (2015~2019) (open for full size).

The compound effect of launching on Steam while also changing all new installs to upload stats by default really blew up the data, and it was interesting to collect and analyze some aggregate player metrics.

Of course the reverse happened half a year later upon switching back to opt-in and ignoring data from anonymous players. It’s true we could legally continue uploading anonymous data if it removes all unique identifiers completely, but to me data in that form is a lot less useful anyway, so for now I figure it’s best to just ignore it all rather than bloat the data set. Maybe later…

Uploading Files with SDL_net

On the technical side of things, how did these scoresheets actually get on the server in the first place? Again this was a big hurdle for me since for some reason this sort of stuff feels like black magic, but at least I found some resources online and was able to slowly patch together something that eventually worked.

Cogmind uses SDL (1.2.14), so when possible I tend to use SDL libraries for features I need. In this case SDL_net is the networking library so I added that in early 2015.

The solution for uploading a file ends up being pretty simple (once you have the answer xD):

sdl_net_http_upload_file

The basics for getting a file uploading via SLD_net. (You can download this as a text file here.)

That’s on the game side, but the server also needs a way to actually create the destination file itself and write the content to it, which is handled by a little PHP file targeted by the upload.

upload_file_creation_via_php

All that’s required on the server to accept the text data and use it to create a file locally (source, the extension of which would need to renamed).

And that’s it, just drop the PHP file in the target directory, and from the game call something like this:

Http connection(“www.gridsagegames.com”,”/cogmind/temp/scores/upload.php”);
bool uploadSuccessful = connection.upload(“filename_here.txt”,DATA_GOES_HERE);

This process ideally needs to be run in a separate thread to keep it from hanging the rest of the game in case the connection is slow or has other issues.

So there you have it, that’s how nearly 90,000 text files made their way to the server from 2015 to 2019!

analyze_scores.exe

The analyze_scores program is nothing to be proud of, that’s for sure. It was technically implemented within Cogmind itself as a special mode, making it easier to reference all the game data, structures, enums, and constants as necessary. Of course it wasn’t integrated throughout the game, just sitting there in its own file waiting to be ripped out one day.

By the time it was removed for Beta 9, it had bloated its way to over 6,000 lines, including lots of legacy code, unused features, bad organization, and confusing naming… basically a complete mess. I never bothered refactoring or designing it well in the first place because I knew it was going to be entirely replaced later on, though I guess at the time I didn’t realize “later on” would come over four years later, and in the meantime whenever I needed to go back and make modifications or additions it was a real headache to even make sense of whether I was doing it right or sometimes even what I was looking at.

Anyway, bad code. Good riddance.

By comparison the recently implemented replacement is both elegant and easy to follow, with all static data properly condensed into an external text file. I’ll be talking about that and more in a followup article (now available here).

Since I was stuck with it for a while but really didn’t want to actually rewrite the thing, an increasing number of features were just tacked on in place, some of the more notable ones including:

  • An external settings file. This was added so I didn’t have to actually recompile whenever I wanted to make minor changes to the program’s behavior.
  • A banned player list. I eventually needed a way to ban players by ID, so there was a text file against which the files were checked before analyzing any player data.
  • A file of alternate names used by a single player ID. I didn’t really need a file for this, but was mainly curious and added it anyway when finally implementing the ability for the leaderboards to detect and rank players by ID rather than name. The earliest iterations were based purely on names, which meant players could easily spoof other players, or individual players could easily occupy multiple places on the leaderboard by simply changing their name. Not very many players actually changed their name, and messing with the integrity of the leaderboards isn’t a big worry with a community like Cogmind’s, but it’s better to head off the inevitable.

A big part of the early bloat came from the  “Alpha Challenge 2015” tournament/event. For that I put together pretty broad-ranging “achievement” ranking system based on scoresheet data, so in addition to regular score-based leaderboards, players were also ranked in 66 different categories.

cogmind_alpha_challenge_2015_achievement_leaders_subset

Sample achievement results from AC2015. At first not all of the achievements were publicly defined beyond showing their name, so players weren’t entirely sure how or what they were calculating until the event’s conclusion, which helped prevent players from gaming the system which would’ve been easy (and therefore boring and unfair). Plus it was fun for players to guess what some of them might mean as they followed the event’s progress :D

I wasn’t sure whether I’d do another tournament like that, but I definitely wanted to, so the code stayed and it just sat there wasting time and space…

It was a lot of fun and I’d always looked forward to doing it again after first holding a similar mini-tournament in 2012 with Cogmind 7DRL, though in both cases we had a relatively small group of players so it was easier to manage, and “benefited from” no one being particular good at the game yet. Although I’d like to hold more competitions, there are two main roadblocks:

  1. A lot of the same people would win anyway--you can almost guess who they’ll be :P
  2. It’s not feasible to combat cheating, and there are enough players now that it would be a concern (especially if I offered real prizes, which I’d love to do as before).

There are some ways around these roadblocks, but various solutions comes with their own issues.

Anyway, back to analyze_scores.exe, it outputs a lot more than just leaderboards!

cogmind_analyze_scores_output_files

Files mostly created by analyze_scores.exe

Aside from the HTML file dropped onto the server, it also generates the materials to facilitate… you guessed it, analysis.

The userscores.csv file contains the complete contents of all submitted scoresheets for a given period, making it easy to examine the data in a single spreadsheet. There’s also the _best version containing each player’s best run so that each player only occupies one row, useful for different kinds of analysis.

cogmind_analyze_scores_output_userscores_best_sample

Sample _userscores_best.csv data.

Two other CSV files contain precalculated stats based on userscores data, organizing select factors that I wanted to keep an eye on across versions, and for easy conversion into graphs.

cogmind_analyze_scores_output_stats_best_sample

Sample _stats_best.csv data. “Best” stats are useful for exploring user preferences, since there’ll be only one set of data per player.

Although the userscores and stats analysis system were originally created for the tournament back during Alpha 3, it wasn’t until later starting with Alpha 8 that I decided to report on stats with each new major release. This is useful for keeping an eye on the general status of the community, and more specifically how recent changes have affected the game (and how players are interacting with them). Plus some players just enjoy having access to this information as well.

Stats have been published 14 separate times so far, all of them in this thread on the forums.

cogmind_alpha_beta_stats_history

History stats as linked from the leaderboards alongside the relevant period’s scoresheet archives.

I’m not sure how we’ll be doing stat analysis in future versions since analyze_scores is no longer a thing under the new system, but since from here on out we’ll have all the scoresheets collected in a brand new protobuf format rather than a trove of text files, it shouldn’t be too much trouble since presumably there is software out there we can use to run the analysis on that collection of data. More on that later!

Update 191226: The followup article covering the new system has been published, see “Leaderboards and Player Stats with Protobufs.”

Posted in Gamedev | Tagged , , , , | Leave a comment