Official development blog

Data-driven Development

As we transition from internal developments to content creation, let us first present an overview of how most of that content is organized.

Cogmind is an example of the “open data” model, wherein as many content-specific parts of the game are exposed in public files anyone can open up and read/edit with a text editor. Data-driven design gives access to the parameters of game objects and more as a way to both speed up development and greatly simplify modding.

Like the 7DRL prototype from 2012, the game comes with two versions of the data: One is the default set, encrypted to prevent tampering during competitions (oh there will be competitions!), and the other is that same default set but available as plain text. Changes small and large can be made to the latter set, then instruct the game to use that instead of the default set--bam, you have a mod.

cogmind_data_files

Current contents of the “data/objects/” directory--still all prototype/testing data, but the main categories are there.

There are other data directories, but a majority of the interesting contents that make up the “game” are currently found in the objects directory: Cells (terrain), props (terrain objects like machines), entities (robots), and items (parts) are the four basic types of objects used to subdivide the world in pretty much every game I’ve ever made. At the most basic level it’s easy to tweak the parameters of a specific weapon or change a given robot’s loadout, but the data files are far more powerful than that, allowing complete control of sound effects, particle effects, and even the animated interface (the appearance of which is mostly scripted, not hard coded).

Aside from objects, another important candidate for externalization is mechanics. Mechanics are often based on formulas, and these formulas in turn contain static values (e.g. Damage = Strength x “2.0”). Source code that leaves these values peppered throughout is a pain to deal with, and makes tweaking for balance difficult. To streamline the iterative design process these static values are always given names instead (e.g. Damage = Strength x “DAMAGE_MULTIPLIER”), and in code I keep almost all of them in a single file. (That file is literally 17% of the project’s code, by line count…) When it comes time to release, it’s pretty trivial to move them all to external files, thereby partially exposing a lot of mechanics.

Although theoretically any text editor will do, I use Notepad++ for multi-row editing (a huge time-saver), syntax highlighting (I maintain a highlighting scheme for Cogmind’s .xt data files), and the ability to be in more than one place in a single file at the same time (plus synchronized scrolling).

Because xml/json require annoying markup that gets in the way of what’s important (data) no matter how nicely you format it, Cogmind .xt files use a clean tab-delimited chart format. We like our data pure. After all, the game knows what kind of data to expect and comes with a fairly robust error checker that points out any unexpected/incorrect data when parsing the files, so why bother with all that unnecessary syntax?

cogmind_data_entities

Robot data from the 7DRL prototype (click for full size). Soon enough they’ll be sharing company with a new cast of friends.

cogmind_data_items

Item data from the 7DRL prototype (click for full size). Most of the prototype’s 500 items were made in a single day--I expect to spend a bit longer on them this time around ;)

Keyword highlighting is more apparent in files like the particle scripts:

cogmind_data_particles

A sampling of the particle effect scripts for rendering a few types of lasers. All animations, including the interface, are generated from scripts like these.

Technical

Internally the entire game is centered around data. How data is stored and represented is always the first and most important part of implementing a new feature.

Rather than risk the headaches and code bloat caused by inheritance (I’ve been there), the more complex Cogmind data objects are just a large collection of variables that are only partially used depending on the object’s subtype. Non-weapon items, for example, still have variables for damage and combat-related values, but these are ignored by the game. This is more straightforward to work with than the generic approach that is the composition model; it also doesn’t waste much memory because these extra variables only occur in static reference data.

All static reference data is read in at the beginning of the game and stored in “compendiums” for each type of object (from robots to sound effects). Unique instances of an object, such as dozens of robots of the same type roaming the map, need only store a reference to their static data that the game should access when it needs one of those values. Unique instances need only store dynamic data, so even ridiculously huge amounts of parameters describing a base object (or huge numbers of objects period) won’t consume exponential amounts of memory in use. This is a common method of handling data in games.

This can be taken a step further by allowing data files to specify “minor variations” of a base object by referencing the original object and indicating what specific values to assign to what variables (usually just a handful), while the game’s data loader automates the process of creating a new compendium entry. While not implemented on a large scale for Cogmind, a majority of particle variations use templates to generate the full particle definitions, thus simplifying recoloring and minor behavioral tweaks.

There are still plenty of aspects about data that haven’t been covered here, so feel free to ask any questions!

This entry was posted in Design and tagged , , . Bookmark the permalink. Trackbacks are closed, but you can post a comment.

4 Comments

  1. Posted February 19, 2014 at 4:57 pm | Permalink

    Sounds good, data driven is the only way to not get insane IMO ;)
    As for the format i’m more of an advocate of non-tab based data, I’ve experienced it with NWN and I never want to touch it again, after a certani point of data it just becomes too much of a mess IMO :/

    You mention encryption of the data for a competition; but how do you manage to really encure no cheating ? It would be probably extremely easy to open up the executable to grab the key and recrypt the data after alteration. Or if you use a private key to encrypt and a public key to decrypt to switch the public key in the code with their own

    • Kyzrati
      Posted February 19, 2014 at 5:46 pm | Permalink

      Interesting, didn’t know NWN (or any other popular game, for that matter) relied on tab-delimited data. I’ve been using it for the past 10 years so it’s kind of what I’m used to ;)
      If it ever got too unwieldy I’ve always planned to write an in-game object editor that could spit back out properly formatted files, but the closest I ever got was writing a tool that output long complex tab-delimited files based on… shorter tab-delimited files! :D

      Funny you mention cheating because just this afternoon I was rereading this post and thought “prevent” was definitely too strong a word to use there. More like “make it not super easy”. Without server-side control all you can hope to do is wrap things up in more layers of security that take longer to unravel. I think that’s pretty pointless unless you’ve got a popular competitive game--with a smaller audience (and little incentive to cheat) the percentage of players who are both willing to cheat and capable of getting beyond *one or two* layers of simple security will be small to non-existent. Heck, people can already cheat their way out of permadeath, which is also completely unpreventable and more likely to happen. My anti-cheating measures are minimal at best, because any more is a waste of time. It’s like fighting piracy: Meh.

      In short: My bad, wrong word ;)

      • Posted February 19, 2014 at 6:21 pm | Permalink

        Ah I see, good then, I do the same for ToME (though not with encryption but with MD5’ing). A bit of prevention is good but one can not prevent it all.

        Many of NWN-like stuff use tab-data actually, look it up it’s interresting at least :)

        • Kyzrati
          Posted February 19, 2014 at 7:17 pm | Permalink

          I also hash the values in the score file itself so it can be plain readable text yet still possible to easily tell if it’s been altered. Also not impossible to break, but no one’s going to bother… (now with ToME it might be a different story given the right incentive ;)

          Haven’t modded anything since the early 90’s. The files I do happen to have looked at these days are indie projects that all seem to use json/yaml/xml or some other syntax-reliant format. I always thought I was in the minority with tabs since anyone who’s ever asked me about data seems to say something along the lines of “why you use tab files, dude?” Wrong assumption, I guess ;)

          I can see how with a huge amount of data and a lot of users/modders you’d want a solution with clearer typing. Fortunately it’s just me! (Though the X@COM modders all seemed to do just fine with that project. We’ll see how it develops when the game is bigger in the future, since modding is a pretty major part of it unlike with Cogmind.)

Post a Comment

Your email is never published nor shared. Only the anti-spam entry is required. See here for the privacy policy.

You may use these HTML tags and attributes <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>