Skip to content

Invariants

An invariant is a check that must hold after every step, whatever just ran. It is where you encode the properties a shuffled trail is trying to break: a cached column that must match its source, a quota that must balance, a status column that may only move along legal edges. Hand-write one with Invariant::make('name', fn (Context $ctx) => ...), or configure a built-in from the Invariants library.

public function invariants(): array
{
    return [
        // A cached/denormalised column must equal the value recomputed from source data.
        Invariants::cachedColumnMatches(Post::class, 'score', fn (Post $post): int => (int) $post->votes()->sum('value')),

        // A quota column must equal the starting allowance minus actual spend.
        // The allowance is a constant here, or a closure when it differs per row
        // (by plan, tier, tenant): quotaBalances(..., fn ($row): int => $row->plan_allowance, ...).
        Invariants::quotaBalances(Post::class, 'reports_remaining', 2, fn (Post $post): int => $post->reports()->count()),

        // A state column may only move along declared edges.
        Invariants::legalTransitions(Post::class, 'status', [
            'draft' => ['published'],
            'published' => ['locked'],
            'locked' => ['archived'],
        ], initial: ['draft']),

        // Soft-deleted parents must not keep live children.
        Invariants::trashedLeavesNoLiveChildren(Post::class, fn (Post $post): int => $post->votes()->count(), 'votes'),

        // No two rows share a column tuple — what a unique constraint (or a
        // firstOrCreate/dedup path) is meant to guarantee.
        Invariants::uniqueBy(Vote::class, ['post_id', 'voter']),
    ];
}

Built-in invariants

Every built-in is a static factory on Vusys\Runabout\Invariants returning an Invariant.

cachedColumnMatches($model, $column, $expected)

The cached/denormalised $column on every row of $model must equal the value recomputed by the $expected closure from source data. Catches counter and aggregate drift — a score, posts_count, or total that one write path updates and another forgets. The comparison is deliberately loose so it doesn't trip over driver-specific numeric types.

quotaBalances($model, $column, $starting, $spent)

The quota $column must equal the starting allowance minus actual spend. $starting is a constant (2 above) or a per-row closure when the allowance differs by plan, tier, or tenant (fn ($row) => $row->plan_allowance). $spent recomputes real consumption from source data. Catches quota drift where grant and spend live in different code paths.

legalTransitions($model, $column, $transitions, initial: null, stateOf: null)

A stateful invariant: it tracks each row's $column across the trail and objects the moment a row jumps between states no declared edge connects. $transitions maps each state to the states it may move to; initial lists the legal starting states. Handles plain strings and BackedEnum values. Pass stateOf to derive the state from something other than a raw column.

trashedLeavesNoLiveChildren($model, $liveChildren, $childrenDescription = 'children', $deletedAtColumn = 'deleted_at')

A soft-deleted parent must keep no live children. $liveChildren returns the count of children that are still live for a given parent. Catches soft-delete leaks where trashing a parent orphans rows that should have gone with it.

$childrenDescription only names the children in the invariant's description and failure message (trashed Post rows keep no live votes), and $deletedAtColumn is there for models whose soft-delete column isn't deleted_at. The query deliberately drops global scopes so it can see the trashed rows in the first place.

uniqueBy($model, $columns)

No two rows of $model may share the given column tuple — what a unique constraint, or a firstOrCreate/dedup path, is meant to guarantee but scattered code can violate under the right ordering.

Invariant checks count as assertions

Each passing invariant check adds one to the test's PHPUnit assertion count, so a journey checked only by invariants isn't reported as risky for performing no assertions. Checks made while shrinking a failure count too.

Stateful invariants live for one trail

Invariants are collected once per trail, so a stateful invariant like legalTransitions lives exactly as long as the trail it watches — it starts fresh each trail and accumulates observations only within it.

Baseline observation: fromStart()

Invariants are checked after every step — but not before the first one. If a state invariant watches a row that exists before the journey (rather than one created in a step), its first observation is the state the opening step left, so that step's transition is mistaken for the row's initial state.

Add ->fromStart() to check the invariant once at the trail's start too, so it sees the true baseline:

Invariants::legalTransitions(Post::class, 'status', $edges, initial: ['draft'])->fromStart();

Custom invariants

Anything the built-ins don't cover is a plain closure:

Invariant::make('daily vote bucket matches votes cast today', function (Context $ctx): void {
    // recompute from source data and assert; throw (or fail an assertion) on violation
});

The closure receives the context and should throw — a failed PHPUnit assertion is fine — when the property is violated. Custom and built-in invariants mix freely in the same invariants() array.

Oracles: checking query answers

Invariants check stored state. They can't check what a query endpoint answers: a report, a filtered list, a search, a "due" or "overdue" list. A class of real bugs lives there, with every row correct and the answer wrong. A list that combines its filters with an unbracketed OR, for instance, lets one branch ignore every filter before it.

Check those answers with an oracle: a step that calls the endpoint and compares the answer with one the journey works out itself, from the database or from first principles. Keep the oracle simple and obviously right, even if it's slow. It filters a whole collection in PHP where the endpoint builds a query, so the two can't share a bug.

  • Vary the filters with pick(), so the endpoint's filter combinations are exercised in the orders and states the shuffles produce, not just the one a hand-written test chose.
  • Move the clock for date-bucketed endpoints. Due, overdue, today and this week all change meaning as time passes. A repeatable clock step lets the shuffles ask the question at many points relative to the data.
  • Compare ids, sorted, unless the endpoint promises an order. Then compare in order, as the order is part of the answer.

A worked example: a "due or overdue" list, checked against the dates the journey moves with travel().

public function steps(): array
{
    return [
        Step::make('create task')->repeatable(max: 4)->act(function (Context $ctx): void {
            $ctx->as('owner')->postJson('/tasks', [
                'title' => 'task '.$ctx->timesRan('create task'),
                'project' => $ctx->pick(['inbox', 'work']),
                'due_on' => now()->addDays($ctx->randomInt(-2, 3))->toDateString(),
                'done' => $ctx->pick([true, false]),
            ])->assertCreated();
        }),

        Step::make('next day')->repeatable(max: 3)->act(fn (Context $ctx) => $ctx->travel('+1 day')),

        Step::make('check the due list')->repeatable()->after('create task')->act(function (Context $ctx): void {
            $project = $ctx->pick(['all', 'inbox']);

            $answer = $ctx->as('owner')
                ->getJson('/tasks/due?'.http_build_query(['project' => $project]))
                ->assertOk()
                ->json('data.*.id');

            // The oracle: every task, filtered in PHP from first principles.
            $expected = Task::query()->get()
                ->filter(fn (Task $task): bool => ! $task->done && $task->due_on->lte(today()))
                ->filter(fn (Task $task): bool => $project === 'all' || $task->project === $project)
                ->pluck('id')->sort()->values()->all();

            Assert::assertSame($expected, collect($answer)->sort()->values()->all(), "The due list for \"{$project}\" is wrong.");
        }),
    ];
}

The check is a step, not an invariant, because it makes a request and only means something at certain points. An invariant can make actor requests too, if the answer should hold after every step. Its requests are observations, so they don't replace the steps' lastResponse().