Testing For Dummies — The Axon Test Framework Manual

Read This First

Axon code has no compiler standing between you and a live Haystack database. There's no red squiggly line, no type checker, nothing stopping a typo from quietly corrupting production data at 2am. axonTest exists so you find out your function is wrong on purpose, on your own schedule, in a disposable sandbox — not by accident, in front of your boss, in the real project.

This manual covers everything the framework can do, in the order you'll actually use it: run some tests, write your own, mock the scary parts, graduate to running against a real project, and finally point it at the thing itself. Every command and code sample in here was actually run against a real SkySpark 3.1.12 install. Nothing in this book is theoretical.

Pushing The Big Red Button

No install. No pod build. No running server. Just a Fantom file and a directory of tests:

fan axonTest/fan/AxonTest.fan test/examples

This walks every *.axon file under that directory (recursively), runs each one as a test — or, for a suite, one per test registered inside it — and prints a summary that reads exactly like a crash-test report:

FAIL   suite_before_test_fails.testShouldNeverExecute (50ms)
       beforeTest failed: setup deliberately broken
PASS   suite_hooks_shared_state.testFixtureHasCorrectNum (70ms)
PASS   suite_hooks_shared_state.testFixtureIsFreshEachTime (33ms)
FAIL   suite_hooks_shared_state.testFixtureNumIsWrongOnPurpose (31ms)
       verifyEq failed: 42 != 999
PASS   suite_math.testAddition      (30ms)
FAIL   suite_math.testSubtractionIsWrongOnPurpose (29ms)
       verifyEq failed: 3 != 999
PASS   test_assertions              (28ms)
ERROR  test_expected_error          (28ms)
       Unknown symbol 'thisFunctionDoesNotExist'
FAIL   test_expected_failure        (26ms)
       verifyEq failed: 2 != 3
PASS   test_mockFunc                (26ms)
PASS   test_mockHttp                (82ms)
PASS   test_mockNow                 (30ms)
PASS   test_mockRec_and_readAll     (31ms)
PASS   test_verifyErr               (30ms)
---
14 tests, 9 passed, 4 failed, 1 errored

Three words, three lights: pass fail error. The exit code is 0 if every test passed and 1 otherwise — safe to wire straight into CI with no extra plumbing.

Speaking Machine

The scrolling text report is for humans. For robots, ask for structured output:

fan axonTest/fan/AxonTest.fan test/examples -format=junit -out=report.xml
fan axonTest/fan/AxonTest.fan test/examples -format=json  -out=report.json

The exit code rule from Chapter 1 doesn't change based on -format/-out.

Wiring The Alarm

Ready-to-use workflow files ship for both GitHub Actions (.github/workflows/axon-test.yml) and GitLab CI (.gitlab-ci.yml). Both run the same one-liner and publish the JUnit report — GitLab reads it natively via artifacts.reports.junit; GitHub uses the mikepenz/action-junit-report action.

Assemble Your Dummy

One .axon file is one test. Its name is its path relative to the test directory, minus .axon. Write a do … end block (or a single expression) and call the assertion funcs below — anything the script throws and doesn't catch fails or errors the test.

do
  verifyEq(2+2, 4);
  verifyNotEq(2+2, 5);
  verifyTrue(1 < 2);
  verifyFalse(1 > 2);
  verifyApprox(1.0001, 1.0, 0.01)
end

The Assertions

Fake It Safely

No setup required for isolation: every test already gets its own in-memory database automatically, so nothing below ever leaks between tests or touches a real project.

do
  mockRec({dis:"Pt A", pt:"y", equipRef:"eq1"});
  mockRec({dis:"Pt B", pt:"y", equipRef:"eq1"});
  mockRec({dis:"Not a point"});
  verifyEq(readAll(pt).size, 2)
end

do
  mockFunc("double", (x) => x*2);
  verifyEq(double(21), 42)
end

Calling Outside The Lab

Fun fact, verified the hard way: this installed SkySpark has no outbound-HTTP axon func at all — every installed pod was scanned for an @Axon method with "http" in the name, and the only hit was an unrelated site-URL getter. httpGet/httpPost fill that gap with a real web::WebClient call; mockHttpUrl/mockHttp make them testable against a real local server instead of the actual internet.

do
  mockHttp("/status", "ok", 200);
  mockHttp("/broken", "nope", 500);

  r: httpGet(mockHttpUrl("/status"));
  verifyEq(r->status, 200);
  verifyEq(r->body, "ok");

  r2: httpGet(mockHttpUrl("/broken"));
  verifyEq(r2->status, 500);

  r3: httpGet(mockHttpUrl("/nope-not-registered"));
  verifyEq(r3->status, 404);  // unregistered routes 404, they don't error

  mockHttp("/echo", "", 200, "POST");
  r4: httpPost(mockHttpUrl("/echo"), "posted-body");
  verifyEq(r4->status, 200)
end

httpGet(url, headers := {}) and httpPost(url, body, headers := {}) return {status, body, headers}. They are real, unconditional network calls — nothing about them is test-specific — so mockHttpUrl/mockHttp are what make a test actually hit a server you control. That server is started lazily (only tests that call it pay for it) and stopped automatically after the test, pass, fail, or error.

Both funcs are also registered as real, production @Axon funcs on axonTestLib — a project that adds the lib gets outbound HTTP capability it didn't have before, testable from day one.

Grouping The Dummies

A single .axon file can also be a suite: instead of running assertions directly, its top-level expression evaluates to a Dict that registers several named tests, optionally sharing beforeTest/afterTest hooks — Kotest-style, if you've used that.

{
  tests: {
    testAddition: () => do
      verifyEq(2+2, 4)
    end,
    testSubtractionIsWrongOnPurpose: () => do
      verifyEq(5 - 2, 999)
    end
  }
}

This registers two tests, reported as suite_math.testAddition and suite_math.testSubtractionIsWrongOnPurpose — each running in its own fresh, isolated context, exactly like separate files would.

There's no direct variable-sharing between hooks and a test body (every test gets a fully fresh context, so a var set in beforeTest wouldn't be visible anyway) — share state through the database instead, the same mockRec/readAll idiom used everywhere else:

{
  beforeTest: () => do
    mockRec({dis:"Widget", num:42, testFixture:"y"})
  end,
  afterTest: () => do
    verifyEq(readAll(testFixture).size, 1)
  end,
  tests: {
    testFixtureHasCorrectNum: () => do
      recs: readAll(testFixture);
      verifyEq(recs[0]->num, 42)
    end,
    testFixtureIsFreshEachTime: () => do
      // If isolation were broken and beforeTest ran cumulatively,
      // this would see more than 1 record.
      verifyEq(readAll(testFixture).size, 1)
    end,
    testFixtureNumIsWrongOnPurpose: () => do
      recs: readAll(testFixture);
      verifyEq(recs[0]->num, 999)
    end
  }
}

The Registration Contract

• tests (required) — a Dict of name: () => … closures. A file only counts as a suite if this evaluates to a non-empty Dict; otherwise it's a plain flat test, exactly as before suites existed.
• beforeTest/afterTest (optional) — zero-arg closures run around each test.
• If beforeTest throws, that test is reported failed/errored as "beforeTest failed: …", and neither the body nor afterTest runs.
• If afterTest throws on an otherwise-passing test, the test becomes failed/errored as "afterTest failed: …". If the body had already failed, that original failure stays primary with "(also: afterTest failed: …)" appended — a broken teardown never masks the real bug.
• beforeTest/afterTest defined with no tests key, or an empty tests, reports one error rather than silently passing.

{
  beforeTest: () => do
    fail("setup deliberately broken")
  end,
  afterTest: () => do
    fail("afterTest should never run")
  end,
  tests: {
    testShouldNeverExecute: () => do
      fail("test body should never run")
    end
  }
}

In JUnit output, suite-produced results use classname="{suiteName}"/name="{testName}" so they group properly in CI dashboards; flat/legacy results keep the old fixed classname="axonTest".

Bolting It Into A Real Project

axonTestLib/ packages the same engine as a real installable hx::HxLib, so a running project can call it as an axon func instead of shelling out to the CLI.

cd axonTestLib
fan build.fan compile

BuildPod compiles straight into the environment's pod path — there's no separate install step. From inside a project (shell, Axon Debug window, or a task), add the lib once and call its funcs like any other:

libAdd("axonTestLib")
axonTestRun("test/examples")            // -> Grid: name, status, msg, durMs
axonTestRunJunit("test/examples")       // -> Str: JUnit XML
axonTestRunJson("test/examples")        // -> Str: JSON report
httpGet("https://example.com/status")   // -> Dict: {status, body, headers}
httpPost(url, "some body")              // -> Dict: {status, body, headers}

Taking Off The Training Wheels

Everything so far runs in a fully isolated sandbox — a fresh, empty database per test, blind to your project's real records or funcs. Pass true as a second argument to opt into integration mode, which instead runs test files against your project's real, already-live data and functions:

libAdd("axonTestLib")
axonTestRun("test/integration", true)
axonTestRunJunit("test/integration", true)
axonTestRunJson("test/integration", true)

In this mode, readAll/read see your project's real records, commit performs real writes, and any real project func — a func: record or one registered by another installed lib — is callable by name, exactly like typing it into the axon shell yourself. All seven assertion funcs from Chapter 4 still work — they're registered as real @Axon funcs specifically so this mode has something to assert with.

What's Different From The Sandbox

• No per-test isolation. Every test in one axonTestRun(dir, true) call — including every test in a suite — shares the same live context, so a mutation from one test is visible to the next. There's no way to hand a test its own fresh copy of a real, already-running project's data.
• Mocking funcs are gone. mockRec, mockNow, mockFunc, mockHttpUrl, and mockHttp fail with an honest "Unknown symbol" rather than doing something silently wrong. Mocking is what the sandbox is for; this mode is for the real thing. httpGet/httpPost are unaffected either way.
• beforeTest/afterTest still run, but since there's no fresh database per test, any fixture a hook creates is real and persists — clean it up yourself in afterTest.
• axonTestLib-only. The standalone CLI has no live project/runtime to integrate with, so this mode isn't offered there.
• Namespace collision caveat. Once axonTestLib is added, verifyEq/fail/etc become real, always-callable funcs project-wide — same situation httpGet/httpPost are already in. If your project already has a func by one of these names, it collides.
• It's noticeably faster, as a side effect — skipping the ~30–200ms fresh-context construction per test entirely, since it reuses the already-live one.

The Dummy Tests The Dummy

axonTestLib/test/ is a real Fantom test suite (run via fant, not this framework's own .axon runner — it tests the Fantom code that implements axonTest) covering both deliverables:

fant axonTestLib

This is the suite that caught the stdout-noise bug mentioned back in Chapter 2 — a real regression, found by the framework, in the framework, about the framework. Appropriately recursive for a testing tool.

Read The Warning Labels

• Discovery is filesystem-only. No tagging/filtering, no parallel execution — tests run one at a time, alphabetically by path.
• Every test recompiles defs (~30–200ms overhead per test in isolated mode). Fine for tens of tests; will need attention in the hundreds.
• The mock HTTP server binds real ports in a fixed range starting at 28170 (retrying up to 20 ports on conflict). Only tests that call mockHttpUrl/mockHttp pay for it, and it's stopped after every test — but a firewall or another process squatting that whole range would break it. No env var override yet.
• No beforeAll/afterAll for suites — see Chapter 7 for why.
• The io/ sandbox from Chapter 8 applies only to axonTestLib, never the standalone CLI, which is just a shell argument to a script you're already running with your own OS permissions.
• Axon syntax quirks worth knowing before you write a test:
  • Variable definition inside a do block is name: expr (colon) — not name = expr, and name := expr isn't valid at all.
  • Arithmetic operators tokenize as part of adjacent number literals/units without spaces — 1/0 and 5-2 both throw SyntaxErr; write 1 / 0 and 5 - 2.
  • There's no bare ISO-8601 DateTime literal — construct one with dateTime(date, time, tz).

For The Dummy In A Hurry

Skipped straight to the back of the book? Here's the whole API on one page.

Where Everything Lives

axonTest/fan/AxonTest.fan       standalone CLI (single runnable file, zero install)
axonTestLib/                    real HxLib pod exposing axonTestRun()/etc as axon funcs
  build.fan                     BuildPod - `fan build.fan compile` builds + installs
  lib/lib.trio                  lib def (name, depends, typeName)
  fan/AxonTestCore.fan          same engine as AxonTest.fan, minus the CLI Main
  fan/AxonTestLibLib.fan        HxLib lifecycle class
  fan/AxonTestLibFuncs.fan      @Axon funcs: axonTestRun/axonTestRunJunit/axonTestRunJson
  test/                         `fant axonTestLib` - tests axonTest's own Fantom code
test/examples/                  example tests demonstrating every feature
test/smoke/                     minimal smoke test used during development

The CLI and axonTestLib deliberately duplicate the same engine classes — they serve different use cases (zero-install script vs. an installed project lib) with different dependency needs.