Skip to main content

Execution Reporter

The dev.specbinder:execution-reporter module is a JUnit 5 extension that writes one JSON file per feature to the project's build output directory. You opt a feature in by annotating its marker class with @ExtendWith(SpecBinderReporter.class). Useful for CI dashboards, custom reporting, or any tooling that needs structured spec-execution data.

Source: examples/going-further/example-4 on GitHub.

What this demonstrates​

  • Activated explicitly per feature with @ExtendWith(SpecBinderReporter.class) on the marker class
  • Each feature gets its own JSON report under target/specbinder-reports/...
  • The report mirrors the Gherkin hierarchy: feature → rules → scenarios → outlines → examples
  • Pass / fail / aborted statuses for every node, with full error blocks for failures
  • Non-SpecBinder tests are silently ignored
  • Two generation options enrich the report: emitScenarioHash (verbatim step text + typed arguments) and descriptionAsAnnotation (Gherkin descriptions in the JSON)

Activation​

Add the dependency:

<dependency>
    <groupId>dev.specbinder</groupId>
    <artifactId>execution-reporter</artifactId>
    <version>2026.38.0</version>
    <scope>test</scope>
</dependency>

Then activate the reporter by annotating the marker class with @ExtendWith(SpecBinderReporter.class):

@Gherkin2JUnit("specs/ShoppingCart.feature")
@ExtendWith(SpecBinderReporter.class)
public abstract class ShoppingCartFeature { … }

Because JUnit's @ExtendWith is @Inherited, placing it on the abstract marker means every generated …Scenarios subclass that JUnit runs inherits it — so a single annotation covers the whole feature. Only classes carrying the extension produce a report; every other test is left untouched.

Running it​

cd examples/going-further/example-4
mvn test

After the run, look at:

target/specbinder-reports/specs/ShoppingCart.feature.json

Report schema​

The schema is hierarchical and uses the same displayName field name at every node:

feature
├── scenarios[]      ← plain scenarios + Scenario Outlines under the feature
└── rules[]
    └── scenarios[]  ← scenarios that live under a Rule

The example feature deliberately exercises every code path the listener handles:

  • Several plain scenarios that pass (real assertions against an in-memory cart)
  • One scenario with a wrong expected total → status: "failed", populated error block
  • One scenario whose Given calls Assumptions.abort(...) → status: "aborted"
  • A Rule with two scenarios → routed under the report's rules[] branch
  • A Scenario Outline with three rows → a scenarioOutline node with three examples[] entries

Enriching the report​

Two @Gherkin2JUnitOptions flags on the marker class make the JSON report carry more than just statuses and timings:

@Gherkin2JUnit("specs/ShoppingCart.feature")
@Gherkin2JUnitOptions(emitScenarioHash = true, descriptionAsAnnotation = true)
@ExtendWith(SpecBinderReporter.class)
public abstract class ShoppingCartFeature { … }
  • emitScenarioHash = true stamps each generated scenario method with a @ScenarioHash — a content hash of its executable steps. Downstream tooling can compare it against live source to detect when a scenario has drifted since the recorded run. It also unlocks the report's verbatim Gherkin step text and typed arguments for each step (both gated on the hash still matching the source .feature).
  • descriptionAsAnnotation = true emits Gherkin description text — under the Feature, Rule, or Scenario — as runtime-retained @Description annotations instead of JavaDoc. The reporter reads them via reflection and adds a description field at each level of the JSON. This feature's Feature: and Rule: blocks both carry a description, so the report shows one at both levels.

Both default to false; the reporter simply omits the corresponding fields when they're off.