Skip to main content

Execution Reporter

The dev.specbinder:execution-reporter module is a JUnit 5 extension that writes one JSON file per feature to the project's build output directory. You opt a feature in by annotating its marker class with @ExtendWith(SpecBinderReporter.class). Useful for CI dashboards, custom reporting, or any tooling that needs structured spec-execution data.

Source: examples/going-further/example-4 on GitHub.

What this demonstrates​

  • Activated explicitly per feature with @ExtendWith(SpecBinderReporter.class) on the marker class
  • Each feature gets its own JSON report under target/specbinder-reports/...
  • The report mirrors the Gherkin hierarchy: feature → rules → scenarios → outlines → examples
  • Pass / fail / aborted statuses for every node, with full error blocks for failures
  • Non-SpecBinder tests are silently ignored
  • Two generation options shape how rich the report is: descriptionAsAnnotation (opt-in — Gherkin descriptions in the JSON) and emitScenarioHash (on by default — verbatim step text + typed arguments)

Activation​

Add the dependency:

<dependency>
    <groupId>dev.specbinder</groupId>
    <artifactId>execution-reporter</artifactId>
    <version>2026.47.0</version>
    <scope>test</scope>
</dependency>

Then activate the reporter by annotating the marker class with @ExtendWith(SpecBinderReporter.class):

@Gherkin2JUnit("specs/ShoppingCart.feature")
@ExtendWith(SpecBinderReporter.class)
public abstract class ShoppingCartFeature { … }

Because JUnit's @ExtendWith is @Inherited, placing it on the abstract marker means every generated …Scenarios subclass that JUnit runs inherits it — so a single annotation covers the whole feature. Only classes carrying the extension produce a report; every other test is left untouched.

Running it​

cd examples/going-further/example-4
mvn test

After the run, look at:

target/specbinder-reports/specs/ShoppingCart.feature.json

Report schema​

The schema is hierarchical and uses the same displayName field name at every node:

feature
├── scenarios[]      ← plain scenarios + Scenario Outlines under the feature
└── rules[]
    └── scenarios[]  ← scenarios that live under a Rule

The example feature deliberately exercises every code path the listener handles:

  • Several plain scenarios that pass (real assertions against an in-memory cart)
  • One scenario with a wrong expected total → status: "failed", populated error block
  • One scenario whose Given calls Assumptions.abort(...) → status: "aborted"
  • A Rule with two scenarios → routed under the report's rules[] branch
  • A Scenario Outline with three rows → a scenarioOutline node with three examples[] entries

Enriching the report​

Two @Gherkin2JUnitOptions flags decide whether the JSON report carries more than just statuses and timings. One of them is already on for you:

@Gherkin2JUnit("specs/ShoppingCart.feature")
@Gherkin2JUnitOptions(descriptionAsAnnotation = true)
@ExtendWith(SpecBinderReporter.class)
public abstract class ShoppingCartFeature { … }
  • descriptionAsAnnotation = true emits Gherkin description text — under the Feature, Rule, or Scenario — as runtime-retained @Description annotations instead of JavaDoc. The reporter reads them via reflection and adds a description field at each level of the JSON. This feature's Feature: and Rule: blocks both carry a description, so the report shows one at both levels. It defaults to false; the reporter simply omits the description fields until you turn it on.
  • emitScenarioHash — on by default, so there's nothing to switch on. It stamps each generated scenario method with a @ScenarioHash: a content hash of its executable steps. Downstream tooling can compare it against live source to detect when a scenario has drifted since the recorded run.