In software development, we build all kinds of tests to ensure the functionality isn't broken.
We have unit tests to make sure one very specific element isn't broken, integration tests to verify if two elements play well together and functional tests to check if one whole feature works from one end to the other.

With that in mind, I thought that maybe I could "functional test" my board game: Stardust.

Wait, WHAT?
Uhm... yes.

Don't worry! I'm not going to post any code here (although if that does interest you, let me know in the comments!).
I will explain the logic of what I'm doing in a more human way.

Before I go into all this, you might want to have a look at my last post or at the rules and the faction mats.

Obviously I won't be able to measure "fun" with this exercise and that is the most important aspect of a game. However I can gauge how balanced each faction is and how often do some weird scenarios happen.
This work should let me know if the mechanics function well together and if there are strategies that can get a player massively ahead (or behind) or the rest.

These few bits do contribute to fun (e.g. it's not fun to be so far ahead of everyone else and win easily), but they are not meant to give me a good picture at all. I'm making a game for people and playtesting feedback is a lot more valuable!

The other thing I won't be able to measure with this is how well would a person interpret the information on the card, that'll be playtest session work.

So, with that out of the way... It's automation time!

Automate all the things!

Where do I start?

I need to start by implementing the elements of the game in a programming language, test them to make sure they do what I want them to do, build some simple AIs that know how to take actions as a player would... and then get a log of everything that happened during the game for analysis.

If you're interested, I used javascript for this.

Implementing each element

Before implementing, I need to look at my rules and see what I've got, what's the same and what's different as far as a computer is concerned.

In my game I have the following components:

  • A few units but they can all be the same thing, but with different capabilities;
  • Hexagonal tiles to build the map dynamically. Each hex has a type (space, star, nebula and blackhole) and can contain one, none or multiple "slots" (planets, asteroids and moons);
  • Leader cards for some of the factions;
  • Dice (9x d3s, 1x d4 and 2x d6s);
  • Resource cards.

Asteroids and Moons work exactly the same (they just have different art), so for my implementation I can just use asteroids.

I also have some concepts and actions that need to be implemented. Some of them, to the mind of a human sound the same, and some of them work basically the same for a machine (I'll group them together from the program's perspective):

  • Factions (which have different abilities and some special rules);
  • Exploring (the act of adding a new hex to the map);
  • Moving (moving from one revealed hex to another);
  • Battling;
  • Building units / Colonizing;
  • Mining/Stardust gathering/Stealing/Population generation;
  • Adviser Voting (for the Republic Advisers);
  • Admiral intents (for the Travellers);
  • Adviser Elections / Admiral 'retirement' (i.e. discarding the leader cards);
  • Effects (usually awarded by the Leader Cards)

I'm probably forgetting something... but that about covers it.
There are a few other rules that humans might not have to think about but the program needs to know (I won't go too much into this).

Most of the physical components are easy to implement, they're just a collection of properties that belong to another element or to a player.
The only tricky one is the hexagon-based grid. Luckily, Amit Patel of Red Blob Games made a nice post about this and there's a nice javascript library that I can use made by Abbe Keultjes.

Game setup

All games usually start by each player getting their pieces into a pile somewhere close to them, organizing the rest of the pieces and then following the game setup (which in this case is shown in the faction mat).

Faction Mat for the Hivemind highlighting the Setup area
Each faction mat has setup instructions

Game setup is, obviously, where the game needs to start, so it's a good idea to implement the act of setting up first.

Getting the pieces into a pile, can be mimicked by generating aaaall the units in advance and put them into some sort of list for each player. This will make the units already have all their properties set up and, just like in real life, there is a finite amount of pieces in their pile.

Once everything is generated and assigned to each player, the AIs need to follow the faction mat's instructions, including the tricky part of adding their starting hex to the initial centrepiece (this sounds a bit like Exploring, so I can reuse this logic later).

After adding the tile, I need to spawn the starting units to that tile (sounds a lot like Building, doesn't it?).

Lets give it a whirl...

Output showing a log of each AI choosing a starting place depending on their personality.
Output of the starting places

I want the AIs to play the factions using different strategies in each game, so when I created the players I assigned them a personality. In the picture above, the Hivemind and the Pirates are aggressive, which means they'll try to place their starting hex close to someone else.

The Hivemind is the first to start, so it can't find an ideal position, but the Pirates decided to start close to the Machine Consensus. Everyone else tried to stay well away from each-other, as they should.

If I print out the initial status and overview of each faction, I can see that each faction created the right amount of units and got the right amount of resources as per their faction mat:

Output of the starting units

Everyone starts with 1 unit, except the Machines: they start with a Worldforge and a Cannon.

Playing through a turn

Now that the game is set up, I need to make the AIs execute their actions on each turn.

Each faction works differently, so I'll have to implement the actions separately and each faction uses them in different orders and/or with different restrictions.
In addition, the AI personality should attribute different weights to different choices (I want an Aggressive AI to seek out combat more often than an Explorer AI would, for example).

I'm going to start with the Hivemind, because it's the first player. Their turn is quite simple (yay):

Turn section in the faction Mat (Hivemind)
Turn section in the faction Mat (Hivemind)

Because the Hivemind starts with a drone on a normal hex with a planet, it will skip the Generate Resources and Breed phases. Technically it should be able to move, but the AI needs to be smart enough to save their units to colonize a planet if it has enough resources.
This brings us to the decision of whether to do battle (there's nothing to battle), colonize (yes! lets do that), explore or make an extra move.

This needs to be a repeatable process, i.e., the decisions that lead the Hivemind to create a new Hive on that planet and ignore the other types of actions need to be generic enough that they can be used throughout the whole game.

In order to do this, I need to deconstruct how a player may think.

  • If I have no Hives, my priority should be making one, otherwise I won't be able to make new drones;
  • If I have drones, I should move at least one of them in the direction of a planet or Explore to find one.
  • If I have drones on a planet tile, then I should keep at least 1 there so I can colonize it in the other stage.

Implementing this logic should be enough for the start of the game and it's still valid for the rest of the session. It still needs some more decisions to happen but those can be added and should work well with that we have.

I'll implement a little bit of the other actions with no logic other than printing some text to the screen. This will let me know whether or not the AI decided to do them or not.

With these rules, the Hivemind should've spent it's Drone and 3 Stardust to turn the planet into a Hive:

Hivemind now has 5 points (up from 1), 1 unit (down one and up another) and 2 stardust (down from 5)
Hives are Units as far as the game knows

That looks right: the Hivemind colonized the planet, losing 3 Stardust and its Drone. This let it spawn a Hive on the tile's planet slot, hence why it shows 1 unit in the output (lost a Drone but gained a Hive).
In addition, the Hivemind won 5 Power.

Before trying another turn, I need to implement Building, Moving, Battling and  Exploring.

The rules for building are simple for a Hivemind. Planets start with 8 pops and you want to turn them into drones as fast as possible, so every turn it's in your interest to build more drones right away.

Moving and exploring are similar: the only difference is that exploring creates a new hex in the direction that you're moving.
These depend a little on the personality.

  • An aggressive Hivemind wants to build up their forces a little bit and move towards an enemy tile so that it can destroy it and take their planet.
  • An explorer, coloniser or shy Hivemind might want to explore away from the centre so that it doesn't get bothered too much.

Regarding battling, aggressive AIs will want to prefer doing this while others may give it less weight when deciding what to do.
For a battle to happen, the units need to occupy the same hex (except cannons can participate in adjacent hex battles without risk of being struck back).
I don't want the AIs to blindly jump into combat, so they need to check whether or not they're outnumbered.

With all these rules implemented, we should expect the Hivemind to build 2 drones on it's 2nd turn.

Output shows Hivemind build 2 drones and has a Hive with 6 pops left

Looks like we were right!

I've changed the output a bit to let us see what units are present (and how many pops a planet has).

Now I need to let the other factions participate and I need to implement the other stages. I'll skip ahead of all the details and just give you the final result.
If you're interested in the thought process behind all the implementations, I can make another blog post in the future about this..

Findings

From the piles and piles of logs, I can see that the Pirates don't play well with Hiveminds or Travellers (or in a 2 player game).

Travellers struggle a bit when hexes run out if they don't manage to get enough ships.

Hiveminds always seem to do well in every game.

The Republic is well balanced due to the adviser system. I tried bypassing the votes to let the Republic do whatever it wanted and they just win the game really easily.

The Machine Consensus AI manages to grow to a nearly unstoppable size if no one keeps pressure on them. This happened 9% of the times, when both the Hivemind and the Travellers had an "Explorer" personality (which make them explore away from the centre) and nobody else had an "Aggressive" personality. Still, they only won one of those times.

Future work

The AIs aren't really that intelligent at the moment as they just follow some instructions and react to a few different variables.
Ideally, I want to make them learn from past games and be better at playing a given personality as time passes. Also, it'd be great to have them choose which personality fits them best.

The output is a bit of a pain to sift through. At the moment I'm dumping it into a database so I can query things a bit better.

This tool could be made more generic with some work and I could open source it once it reaches that stage and if there's any interest.

Thank you for reading! If you have any questions or suggestions let me know on the comments below or reach me via twitter/email.