Skip to content

Test the application

Tier Command Needs Proves
unit just test-unit nothing entities, rule sets, gates, the parser, adapters against recorded HTTP responses, rendering, keyed views
independent just independent nothing properties written from the requirements alone: a retracted source is never eligible, a species mismatch is never direct, a mismatched span is never admitted, a hypothesis is never verified, an unlisted reviewer never approves, a rendering never adds a claim
integration just test-integration Docker each workflow through the whole service and its HTTP routes, with a scripted model, including restarts mid-process
faults just faults Docker each of the six injected faults is caught at its check and never becomes evidence
living features just features-replay Docker the Gherkin scenarios under features/, through HTTP, against the recorded replies
evaluation just eval Docker the labelled dataset through the service; writes a report under evals/reports/
mutants just mutants Docker disabling each of three safeguards in a copy of the tree makes an independent test fail

just pr runs the feature checker, unit, integration, independent and living-feature tiers, which is what continuous integration runs on a pull request.

One database per run

The integration test kit starts one throwaway Postgres per sbt testOnly invocation, and every suite in that invocation shares it. Suites are written to tolerate each other's data, and the fault suite, which alters fixture records, runs in its own invocation after the integration suites. Run heavy tiers one at a time: two sbt test runs at once compete for the machine and slow the asynchronous parts enough to fail waits that pass on an idle machine.

The scripted model

Integration suites use ankka's TestModelProvider. scriptFromCassettes("plan@v1", ...) loads every recorded reply for those prompt versions as a standing rule keyed by its case marker, so a suite answers the calls it expects and fails on one it does not. A suite that needs a bad reply first scripts it with a rule that fires once.

The living features

The scenarios under features/ are written in the vocabulary of GLOSSARY.md and checked against it by just check-bdd. Their step definitions call the HTTP API only. In the replay tier they run against the cassettes; just features-live runs them against the live model, and a scenario that passes in replay and fails live is reported as drift.

The evaluation

evals/cases holds labelled end-to-end cases, each in one adversarial category such as species mismatch, negation, a retracted paper or genuinely insufficient evidence, with acceptable alternative labels where reviewers might differ. evals/passages holds focused extraction passages with the expected verification label. The report measures each stage separately and lists critical failures first, as a category that blocks regardless of the aggregate scores.