MARCIN RUSINOWSKI — ONLINE SINCE 90s MAC OS · C++ · UNREAL ENGINE EN · PL*
Dev — ← back to Dev

Letting the game play itself

2026-10-05

The problem with a hundred decisions

N is a game where the player is hooked into decisions. Let's assume there are 100 decisions to make, but behind them sit about 1,900 plots spread over 63 data tables, plus random events that can cut into the story and stats that quietly open and close options as you go.

I can play that game. My testers can play it. What none of us can do is play it a thousand times. And a lot of the questions I actually need answered are only visible at that scale: which plots does nobody ever reach, how do games end, how long do they last, does the game really pick what the data says it should pick? One human playthrough tells you how a game feels. It tells you almost nothing about how the system behaves.

So I built a way for the game to play itself.

The idea: the game plays, a server decides

The Mechanical Test Mode splits a playthrough into two halves. The game does everything it normally does: it picks the next plot, rolls its random events, applies stats, runs the gameplay. The only thing it no longer does is wait for a human. Every time a decision page comes up, the game sends the current state and the list of choices to a small server running on localhost, and the server answers with one of those choices, or with "timeout".

The server is a short Python script with no dependencies. It reads the same source_json files the DataTables are imported from, so both sides look at one source of truth. It decides how to play (random, always the first option, always passive), keeps a log of every game, and checks what the game did against what the data allows.

That split matters. The game stays the thing under test. The server never chooses plots, never rolls dice for the game, never runs its own version of the story. It just plays, and watches.

The game plays itself; the server only picks choices and checks
The game plays itself; the server only picks choices and checks

Both sides read the same data; only the game decides what happens next.

One switch, and nothing when it's off

The first rule I set: when the test mode is off, the game must be exactly the shipped game. No menu skip, no HTTP, no extra logging, no timing changes. Test code that leaks into a release build is worse than no test code.

So the whole feature hangs on one compile-time constant:

// MechTest.h
#if UE_BUILD_SHIPPING
static constexpr bool bMechanicalTestMode = false;   // can never be on in Shipping
#else
static constexpr bool bMechanicalTestMode = false;   // set true locally for test runs
#endif

Every test branch in the game is wrapped in if constexpr (bMechanicalTestMode), so with the switch off the compiler drops it. The driver that talks to the server is a GameInstance subsystem whose ShouldCreateSubsystem() simply returns the switch, so in a normal build it never even exists. The game itself only knows the server's address; everything else (how many games, which mode, which seed, which policy) comes from the server when a game starts. Changing a batch never needs a rebuild.

Drive the real code, don't simulate it

The tempting shortcut is to write a fast simulator of the game's rules and run that a million times. I didn't, at least not yet. A simulator tests the simulator. The bugs I care about live in the actual code path: the HUD, the GameMode, the subsystem, the order in which they call each other.

So the test mode plugs into the same few places a player touches:

The timeout detail is a good example of why this matters. In N, a run of passive decisions has its own consequence. If the test mode faked timeouts on its own, it would test my guess of how that works. Because it drives the real timeout path, it tests the real thing.

When anything goes wrong (no reply, a choice that doesn't exist, a reply for the wrong plot) the game logs it and exits with an error code. It never falls back to waiting for a human. A batch that hangs silently overnight is the worst possible outcome.

Why leave it to chance

Before I get to testing randomness, I should say why the game is random in the first place.

It would be so much easier if it wasn't. A game that plays out the same way every time is simple to build and simple to test. You can walk through it once, tick the boxes and go home.

But it's also simple to forget.

Once you've seen the path, there's nothing left to find. The second time through is just a rerun of the first. Flat and predictable is boring, and boredom is the one thing a game can't afford.

So I didn't want every player to get the same story. In N the content comes out of a big pool of data and a bit of chance. Each time you play, you run into different situations, in a different order, with different details. Two people can start in the same place and end up somewhere completely different. And you can come back a week later and still find something you haven't seen.

That's what I mean by replayability. In a game made of decisions it matters even more, because a choice only means something if you can't be sure where it leads.

Of course, there's a price. I can't write every possible path by hand, and I can't tell you exactly what a player will see. Which is the reason this whole test mode exists. The same randomness that makes the game worth playing again makes it very hard to check by playing it a few times.

Randomness you can replay

That makes randomness the enemy of testing. Starting conditions, random events, which scene fills a gap: about seventeen places in the code roll dice. A bug you can't reproduce is a bug you can't fix.

The first step is simple: the server gives every game a seed, and the game seeds its random generator with it right before the game begins. Same seed, same data, same build should give the same game. I don't assume that holds. The report compares two batches run with the same seeds and counts how many games came out identical. If engine code draws from the same global generator, that number will tell me, and the next step is a dedicated random stream that every roll in the game goes through.

The second step is how I think about scripted test paths. A path can't be a fixed list of "choose option 2, then 1, then 3", because the game might insert a random event and every index after it means something else. So a path is a set of rules keyed by the plot that's actually on screen ("at this scene, pick that reaction"), plus an optional goal ("reach a given scene"). The server re-plans every step from wherever the game really is.

An oracle that reads the same data

Playing a thousand games is only half of it. The more useful half is that the server checks every step against the data.

Each time the game moves on to the next plot, it writes down what it decided and why: the scene it intended, whether a random event fired, whether a filler scene took the slot and from which candidates, whether the last step was forced to the ending, and the counters and state after the move. That record goes into the game's log and to the server.

The server then asks a narrow question: is this allowed? It knows which choices the stats should leave on screen, where each choice routes, which random events are allowed at which stage, which plots may fill a gap, and when the ending must be forced. If the game shows a choice the filters should have hidden, routes to the wrong place, or stages a plot and then shows a different one, the server flags it and names the kind of mismatch.

It doesn't predict dice. Where something is random, the oracle checks membership, not equality: the random event that fired has to be one that could fire, and the gap plot has to come from the allowed pool. That keeps it honest without making it a second copy of the game.

That's also the real risk with this kind of tool. An oracle that grows its own version of the rules will drift away from the game, and then you're testing two programs against each other. So it mirrors only the documented selection rules, and there's a line in my project instructions: when the selection code changes, the oracle changes in the same commit.

It earned its keep early. Building it, I found that the two alternative game modes showed the same plot again after every decision, and opened their first page on the wrong plot. Neither shows up when you only ever test the normal game.

What it answers, and where it stops

The questions I built it for are the ones a designer keeps guessing at:

There's also a handshake before any run: the game sends the row count of every DataTable, the server compares it with source_json, and refuses to start if they differ. A JSON file edited but not reimported is the most boring bug in a data-driven game, and the easiest one to catch automatically.

What it can't do is just as important. It tests structure and balance, not feel. A bot's choices are not a player's choices; even the planned personas (risk-taker, cautious, passive) only approximate people. It runs through the real UI and reloads the level for every game, so it isn't fast. And because the server lives on localhost, it runs on a desktop, not on a phone, even though mobile is the first target. Human playtests stay. This just stops me spending them on questions a machine can answer.

What do you call this?

When I tried to name it, I found there is no single standard term. It is a combination of known techniques, and each part of the design already has its own name.

In the game industry the umbrella is automated playtesting, or bot playtesting: bots play the game instead of people. Leaving the game running unattended to catch crashes and hangs is soak testing. Running thousands of games to tune the numbers is simulation-based balancing, familiar from card games, roguelikes and economy games.

In software-testing terms the pieces map neatly:

On the development side, building a game so it can be driven and checked automatically is design for testability. Putting the source of decisions (human, server, script) behind one interface is the Strategy pattern, or dependency injection of the input.

In the project I call it Mechanical Test Mode. To someone outside it, I would say: automated playtesting with a test oracle. Shorter still: simulation-based testing.

Closing thought

None of the pieces is clever on its own. A compile-time switch, a localhost server, a log line per step. Keep one source of truth, let the real game do the work, and put a cheap, honest checker next to it.

It is also built in phases. Seeding, the self-playing loop and the oracle with its reports come first. Personas, coverage-hunting runs and goal paths come next, then forced dice for exact bug repros, and eventually a logic-only mode fast enough for CI. Each step only makes sense once the previous one tells the truth.

That is the whole point. A question like "is my content really reachable?" stops being a gut feeling and becomes a report. The game can't tell me whether it is good. It can finally tell me what it is, and whether it works as designed.