Skip to main content

Suite manifest reference

A suite manifest is a TOML file named suite.toml, snag.toml, or *.snag.toml (any name works when passed explicitly on the command line).

suite.toml
title = "Checkout API"
timeout = "10s"

[variables]
base_url = "https://api.example.com"

[[test]]
id = "cart-create"
name = "POST /carts creates a cart"
tags = ["smoke", "checkout"]
parallel_safe = true
timeout = "20s"
file = "./checkout/cart_create.snag"

[test.variables]
currency = "USD"

Top level

FieldTypeDefaultDescription
titlestring"Untitled suite"Suite label. Appears in reports — JUnit classname, TeamCity suite name, the JSON suite field
timeoutduration stringnoneDefault per-test timeout for this file
variablestable of string → string{}Variables shared by every test in the file
setuphooknoneScript(s) run before every test in the file
teardownhooknoneScript(s) run after every test in the file
testarray of tables[]The tests, written as repeated [[test]] blocks

An empty manifest is valid and contributes zero tests.

[[test]]

FieldTypeRequiredDefaultDescription
idstringyesIdentifier, unique within the file
filepathyesScript path, relative to this manifest
namestringnothe idHuman-readable label used by reporters
tagsarray of stringsno[]Selection tags, matched exactly by -t
timeoutduration stringnothe suite timeoutPer-test timeout
parallel_safeboolnotruefalse pins the test to the serial phase
variablestable of string → stringno{}Overrides of suite variables, for this test only
setuphooknononeScript(s) run before this test, after the suite's setup
teardownhooknononeScript(s) run after this test, before the suite's teardown

[test.variables] attaches to the [[test]] block directly above it — that is ordinary TOML table scoping, and the most common place to get a manifest subtly wrong. snag list --long shows what was actually parsed.

Types

Duration strings

Parsed with humantime. A unit is mandatory.

ValidMeaning
"500ms"500 milliseconds
"30s"30 seconds
"2m"2 minutes
"1m 30s"90 seconds

"10" is not a duration and fails the parse.

Variables

Both [variables] and [test.variables] are flat maps of string to string. Numbers, booleans, arrays, and nested tables are not accepted — quote the value and convert in the script:

[variables]
max_latency_ms = "800"
assert_faster_than(res, max_latency_ms.parse_int());

Hooks

setup and teardown accept a path, a table, or a list of either:

setup = "./login.snag" # one script
setup = ["./login.snag", "./seed.snag"] # in order
teardown = { file = "./reset.snag", always = true } # table form
teardown = [{ file = "./check.snag", always = false }, "./reset.snag"]
FieldTypeDefaultDescription
filepathrequiredScript path, resolved like file on a test
alwaysbooltrueTeardown only: false skips the script when the test failed

Suite hooks wrap test hooks. Setup runs suite-first, teardown unwinds test-first:

suite setup → test setup → the test → test teardown → suite teardown

Hook scripts share the test's scope and variables — a let at the top level of a setup script is readable from the test. See Setup and teardown for the full ordering, including the script-level fn setup / fn teardown and on_teardown(...).

Paths

file is resolved relative to the directory containing the manifest, never relative to the shell's working directory. . components are stripped for display; .. is preserved as written.

tests/auth.snag.toml
file = "./login.snag" # → tests/login.snag
file = "scripts/login.snag" # → tests/scripts/login.snag
file = "../shared/ping.snag" # → tests/../shared/ping.snag

A file that does not exist is not a manifest error: the test runs and fails with script … does not exist, so one broken path does not take down the run.

Validation

Enforced at load time, before anything executes:

RuleFailure
Unknown fields are rejected (deny_unknown_fields)Error: parsing suite <path>, exit 2
id and file must be presentError: parsing suite <path>, exit 2
timeout must be a valid durationError: parsing suite <path>, exit 2
Ids must be unique within a fileduplicate test id 'x' in <path>, exit 2
always may only appear on teardown`always` is only valid on teardown, not setup, exit 2

Common rejected typos: tag (should be tags), script (should be file), parallel-safe (should be parallel_safe), variable (should be variables).

:::note The parse error does not include the TOML detail Today the message is only Error: parsing suite <path> — the underlying reason is not printed. To narrow it down, validate the file with any TOML parser, or comment out [[test]] blocks until snag list succeeds. :::

Precedence

Variables — a test's key wins over the suite's key. Flat, per key, no deep merge:

[variables] → [test.variables]

Timeouts — the most specific setting wins, not the tightest one:

suite timeout → test timeout → --timeout

Each step overrides the one before it whatever its value, so --timeout 60s relaxes a test declaring timeout = "1s".

Parallelismparallel_safe = false always means the serial phase; -j only sizes the pool for the parallel phase.

Resolved test

What the runner sees after a manifest is loaded, one per [[test]]:

PropertyOrigin
idid
namename, else id
tagstags
timeouttest timeout, else suite timeout
parallel_safeparallel_safe, else true
scriptfile, joined to the manifest's directory and normalised
varssuite variables merged with test variables
setupsuite setup followed by test setup, paths resolved
teardowntest teardown followed by suite teardown, paths resolved
suite_titletitle
suite_paththe manifest path as discovered
qualified id<suite_path>::<id>

The qualified id embeds the path as discovered: snag from the repository root produces ./suite.toml::cart-create, while snag suite.toml produces suite.toml::cart-create. Keep one invocation in CI if a dashboard keys on it.

Full example

tests/checkout.snag.toml
title = "Checkout API"
timeout = "10s"

[variables]
base_url = "https://staging.api.example.com"
currency = "EUR"
tenant_id = "tenant-test-001"

# Runs alone, before every parallel test.
[[test]]
id = "reset-fixtures"
name = "fixtures are reset"
tags = ["fixtures"]
parallel_safe = false
timeout = "60s"
file = "./checkout/reset.snag"

[[test]]
id = "cart-create"
name = "POST /carts creates a cart"
tags = ["smoke", "checkout"]
file = "./checkout/cart_create.snag"

[[test]]
id = "cart-create-usd"
name = "POST /carts honours a non-default currency"
tags = ["checkout"]
file = "./checkout/cart_create.snag"

[test.variables]
currency = "USD"

[[test]]
id = "report-generate"
name = "report generation finishes"
tags = ["slow"]
timeout = "120s"
file = "./checkout/report.snag"

Note that cart-create and cart-create-usd share one script and differ only in a variable — the cheapest way to cover a matrix.

See also