Skip to content

Validating specs

The automath check command group validates question specifications without rendering anything. It powers two workflows:

Two subcommands are available: qspec (one spec line) and qblock (a whole questions: block, including multi-line pipe/mix/pool notation).

One level up, a whole script document — structure included, not only its questions — is validated by the /check/document endpoint of automathd. It has no CLI counterpart: on the command line, rendering the script is the more direct way to find out whether it holds together.

automath check qspec

Validate a single question-spec line — the full body that would follow the leading (n) count in a YAML questions block.

automath check qspec "multi_direct 2-9×2-9"

Exit code 0 on a valid spec, 1 otherwise (the error message is printed to stderr).

A trailing # comment is dropped before the spec is read, so a line copied out of a questions: block checks the same way on its own. A spec that is nothing but a comment is an error — there is no spec to check.

Warnings

A spec can be perfectly valid and still be unlikely to give what you meant — typically a constraint that leaves only a handful of drawable values. Those remarks are printed to stderr as Warning: lines and do not change the exit code.

automath check qspec "order_of_2ops 7-19 4-9×4-9 -- variant=plain result=positive"

Warning: order_of_2ops variant 24 (a:b-c): only 2 of the 13 values of the
singleton source are usable with result=positive. Consider a dedicated line
for this variant, or exclude it with variant=^24.

Batch mode

With --batch, one qspec is read per line from stdin (blank lines and comment-only lines ignored) and a JSON array is written to stdout — one object per remaining line, in input order. The command always exits 0 in batch mode, regardless of individual results. This is far cheaper than spawning automath once per qspec.

printf 'multi_direct 2-9×2-9\nbogus_type 1+1\n' | automath check qspec --batch

[{"valid": true}, {"valid": false, "error": "<error message>"}]

A valid entry carries an extra warnings array when there is something to report: {"valid": true, "warnings": ["<message>", "..."]}. The key is omitted when the list would be empty.

automath check qblock

Validate a whole questions: block. Unlike qspec, qblock understands the multi-line pipe/mix/pool notation: it walks the logical structure and validates every identifier and every source mentioned, returning the full list of errors per block.

A block is checked as deeply as qspec checks a single line. Every (identifier, sources, options) pairing a line can resolve to — the cartesian product, since mix notation shuffles its sources before pairing them — goes through the same per-question-type checks: the number of sources the type accepts, the steps option, and the type's own diagnostics. So an arity mistake or an unusable option is reported here and not left to the render.

Only --batch is supported. Pass a JSON array of {name, text} objects on stdin (one per block — for example, one per YAML series in a file) and read a JSON array of {name, errors} results in the same order from stdout. The command always exits 0.

echo '[{"name": "series_a", "text": "questions:\n  - (5) multi_direct 2-9×2-9\n"}]' \
  | automath check qblock --batch

[{"name": "series_a", "errors": [{"line": 2, "message": "..."}]}]

Each error carries a line field (1-based, relative to the start of the block) so an editor can map the message back to the right line of the YAML. Comment lines count: they are emptied, not removed, so the numbers keep matching the text the author is editing.

A block that is valid yet questionable also carries a warnings array of the same shape, present only when there is something to report:

[{"name": "series_a", "errors": [],
  "warnings": [{"line": 1, "message": "..."}]}]

Warnings do not make a block invalid: what they describe renders, just probably not the way its author meant.