Improve a robot with local trials

Play the same opponent, seed and ruleset before and after a change. Inspect the first decision where behaviour diverges, then repeat across several seeds and both seat orders. Keep a working baseline so a faster run or one win does not hide a control, protocol or startup regression.

For a small standalone project, start with the player kit: clone your language’s starter, then run make match and make replay. It uses the shared @clunkyard/cli npm package and downloads only the common engine and selected SDK. Without Make, use npx @clunkyard/cli match 42 2d classic and npx @clunkyard/cli replay replays/trial-42.json. Use npx @clunkyard/cli for current tooling and record --version with your source commit when comparing trials. The commands below are for contributors working from the full game repository.

JavaScript and TypeScript projects use the npm SDK under the clunkyard import name, which also works in hosted builds. Use the language starter's make match and make replay for its engine and launcher. The npm package also exposes play(robot, lines) for tests driven by recorded protocol messages; a real match exercises the engine, opponent and robot together.

There are two repository routes:

Run the commands below from the repository root. Local CLI programs are trusted programs on your computer. The Docker route uses the hosted supervisor/relay, fixed profile build recipes and isolated runtime.

Start with a real match

This match needs only Node and the checked-in DSL core:

node src/cli.js match \
  --bot @bots/dsl/vane/bot.json --bot @bots/dsl/cadence/bot.json \
  --mode 2d --ruleset classic --seed 42 --ticks 240 \
  --replay replays/trial-42.json
node src/replay-tool.js inspect replays/trial-42.json
node src/cli.js replay replays/trial-42.json

Open the replay URL printed by the last command. Pause and step through contacts, aim, scanner direction, battery and accepted controls. Replaying uses recorded states and does not rerun the robots. See replay inspection and export for archives and video.

A downloaded workbench project can replace either manifest:

node src/cli.js match \
  --bot @/absolute/path/scout/bot.json \
  --bot @/absolute/path/baseline/bot.json \
  --mode 3d --ruleset zero-g --seed 42 --ticks 360 \
  --replay replays/scout-zero-g-42.json

The local runner builds workbench manifests with language and entry for JavaScript, TypeScript, Python, C, Go, Rust, Java, Vane, Cadence and Glyph. It uses the installed SDK at /opt/clunkyard/sdk; set CLUNKYARD_SDK for another prefix. Compiled output goes in .clunkyard/ beside the manifest. Use node src/cli.js roster to see the current House pack.

For another profile, build and launch it using its binding guide. A local manifest can supply explicit build and command arrays; these trusted local commands run with the repository root as their working directory, so use absolute project paths. An already built executable needs only a command. For example:

{
  "name": "Scout native",
  "language": "cpp",
  "protocol": 2,
  "command": ["/absolute/path/scout/bot"],
  "supportedModes": ["2d", "3d"],
  "supportedRulesets": ["classic", "inertial-2d", "gravity-flat", "gravity-hills", "zero-g"]
}

The CLI also accepts --bot 'Scout=/absolute/path/scout/bot' for a quick local command. A saved manifest is easier to repeat and declares supported rulesets explicitly. Hosted builds use the selected language profile's recipe rather than executing a manifest's arbitrary build command.

Exercise a user project through the hosted profiles

Use this route for every hosted language, including adapters, compiled languages, DSLs and ABI 4 guests. Keep the entire project under one directory with bot.json, its entry and any dependencies included as source. Keep generated executables and local build files under .clunkyard/ or build/; the checker omits these directories, .git/, __pycache__/ and node_modules/ from the bundle. Ordinary files still count toward source file/size quotas, so move a loose executable out of the source root before testing. The builder has no network access.

Build the images for your checkout with make images, or select an existing matching release pair:

node scripts/check-starters.js --house /absolute/path/scout \
  --builder r.k3s.ca/slepp/clunkyard-builder:RELEASE_TAG \
  --runtime r.k3s.ca/slepp/clunkyard-runtime:RELEASE_TAG \
  --rulesets all --seed 42

Replace RELEASE_TAG with an immutable release tag present in local Docker. With no image arguments, the script uses the current checkout's Git commit tag, as produced by make images. Both images must already be available. Use matching images so their installed bindings and runtime libraries agree.

Despite its --house name, this option accepts any project directory. It builds the project's actual files through sandbox/worker.js and plays it against the JavaScript starter. --rulesets all exercises every compatible mode/ruleset the manifest declares; without it, the script checks classic 2D and 3D where supported. It does not accept a custom opponent, replay output path or tick limit.

A pass requires successful builds, no match failures, at least one shot in each match and resource use below half the checked limits. A peaceful probe can therefore fail “never fired” while still speaking valid protocol. Read FAIL details and both pod logs to separate robot failures from platform startup failures; a referee-origin startup timeout is an infrastructure failure, not evidence that the tactic exceeded its CPU budget. The script exits non-zero if any project fails.

This check reports CPU and own time for Standard, operations for DSL and fuel for WebAssembly. Wall time is printed for information. The frozen Season 3 rates were calibrated for the arena's hardware; a different local CPU or runtime environment can change headroom. Local Docker checks are useful compatibility and regression tests. Use arena trials to confirm competitive headroom under its current limits.

Repeat a comparison

Keep your baseline and candidate in different directories or at immutable build paths. Choose a seed list before comparing, change one tactic at a time and keep the mode, ruleset, objective and tick limit fixed:

node src/cli.js match --bot @/absolute/path/baseline/bot.json \
  --bot @/absolute/path/candidate/bot.json \
  --mode 2d --ruleset inertial-2d --seed 7 --ticks 360 \
  --replay replays/baseline-first-7.json
node src/cli.js match --bot @/absolute/path/candidate/bot.json \
  --bot @/absolute/path/baseline/bot.json \
  --mode 2d --ruleset inertial-2d --seed 7 --ticks 360 \
  --replay replays/candidate-first-7.json

Repeat with several seeds and opponents. Reverse seat order for each seed because starting positions and robot identity can affect the result. A fixed seed reproduces the engine's scenario; reproducibility also requires the same code, configuration and any tactic randomness. Use the SDK's randomness API where available and control any external randomness yourself.

For a round robin, node src/cli.js tournament --bot @... --bot @... --rounds 3 --seed 1000 --out replays/my-tournament saves a summary and one JSON replay per heat. Tournament seeds advance by heat; this is different from the paired same-seed comparison above. The local spectator's Compare revisions tool runs two installed revisions against a chosen opponent with balanced seat orders and archives their identities, statistics and replays.

A useful edit-run loop is:

1. Record the baseline's failures, shots, hits, damage and result on your chosen seeds. Keep its source and build identity.
2. Change one decision, such as scan width or reserve battery.
3. Run the same fixtures or memory-session tests for protocol/API mistakes, then replay the same real matches. A fixture smoke test is not a match.
4. Inspect the first unwanted turn: was the contact absent, the gun cooling, the shot unreachable, or a control rejected? Log a short status/value or write diagnostics to stderr.
5. Keep changes that improve the intended behaviour across the trial set, then run the hosted-profile check before uploading or rebuilding in the arena.

Default CLI matches use --resources open; they are convenient functional trials and do not reproduce hosted Standard cgroup accounting. --resources trusted is a local resource-reporting mode, not a substitute for the hosted supervisor. The resource guide explains the classes and measurements. Replay outcomes alone do not measure CPU headroom.

These commands are implemented by the CLI, manifest resolution and the hosted-project checker.

Download this page as Markdown