Running and debugging

A spec file is never executed itself. At build time SpecBinder turns each spec file into Java test code, and that JUnit code is what runs. The plugin puts the controls for running and debugging the JUnit code in the spec file: Run or Debug beside a feature, a rule, a scenario or one Examples table row runs the corresponding JUnit tests. Results and the debugger stay in the spec file rather than in the generated Java code, so a failing scenario is read in the same words that describe the behaviour.

Run icons in the gutter

While an element has no result yet, the gutter beside its Feature:, Rule: or Scenario: keyword shows the IDE's standard run icon . Click the icon, then select Run or Debug. The IDE runs that element as a JUnit test. Once the element has run, the icon is replaced by the element's last result. The result icon is clickable too, with the same Run and Debug choices, so the element can be run again from the same spot. The standard run icon comes back only when the element is edited after that run.

The Scenario's run icon

Run / Debug, named after the scenario

The Rule's icon opens the same menu

The same gutter control at every level. A Scenario and a Rule each carry a run icon, and each opens the same Run / Debug menu, named after its title in the spec.

Run of a whole Rule

The run icon beside a Rule: keyword runs every scenario inside that Rule, and only those. The results tree comes back grouped the same way as the spec file: the Rule's title, with each scenario beneath it, named by the scenario title from the spec file rather than by a Java method name. The results can therefore be read in the same words as the spec file.

The Rule's run icon

Run, named after the Rule

The tree comes back in the words of the spec file

The Rule's icon now shows the result

Running a Rule from its gutter icon. The results tree comes back in the words of the spec file — the rule title and both scenario titles, not the names of the generated Java methods.

Run of just one Examples row

Each data row of an Examples table has its own run icon in the gutter. Click the icon, then select Run or Debug. The IDE runs the scenario outline one time, with the values of that row.

Every Examples row has its own run icon

Run or Debug just this row

Named after the outline, plus the row's number: [2]

One row ran — the Example the icon sat beside

That row's icon now shows the result; the other rows have not run

One row, one run. The gutter icon on a single data row runs the outline just for that row — the run is named after the outline with the row's number in brackets — the results tree comes back with one node, not three, and only that row's icon changes to show the result.

Running the whole spec file

The Run button at the left end of the spec file's toolbar runs every scenario in the file, wherever the caret is. There is no need to scroll up to the Feature: line and use the gutter icon there.

The button also has a keyboard shortcut, listed under its own SpecBinder group in Settings → Keymap.

Run This Spec File

Nine tests — the whole file

One press runs the whole file: nine tests, one failed and one ignored among them.

Breakpoints on steps

Put a line breakpoint on a step. Start the run in Debug mode, and the run stops at that step. The spec file stays in view, and not the generated Java code. Step over, resume and the Variables view operate as usual. Step into opens the body of the step method.

A breakpoint on a background step stops the run before each scenario. A breakpoint in a scenario outline stops the run one time for each Examples row. If the same step text occurs in more than one scenario, the run stops only at the step that you marked.

A breakpoint on the Then step

Debug, from the scenario's own icon

The run stops in the spec file, on the step you marked

The Variables view opens on the cart the steps filled

Debugging happens in the spec file. The run stops on the step you marked — not in the generated Java — and the Variables view opens on the cart, holding the very number the step is about.

The stopped frame on its own. The highlighted line is the step in the spec file, the spec file never leaves the screen, and subtotal holds the number written in that step.

The test results tree

Double-click a scenario, a rule or the feature in the results tree. The IDE opens the related line in the spec file, and not the generated Java code.

Double-click the failed result

The spec file opens at the failed scenario's line

Double-click a failed result and the editor opens the spec file at that scenario's line — not the generated Java, as the breadcrumb along the bottom keeps showing. A single click only selects the node.

Rerunning only the failed scenarios

Each node still stands for the JUnit test behind it, so Rerun Failed Tests, and running a single node again from the tree, work as they do for any other test. After a run with failures, the Rerun Failed Tests button in the Run tool window's toolbar runs only the scenarios that failed, and the results tree then shows just those.

Rerun Failed Tests

Only the failed scenario runs again

Rerun Failed Tests runs only the scenario that failed, so the results tree then lists that one scenario and nothing else.

Gradle run configurations

In a Gradle module the plugin makes a Gradle configuration with a :test --tests … filter. Thus the test runs with the task graph, the system properties and the classpath of your build. The filter agrees with the element that you clicked: the class, the nested class or one method. In a Maven module the plugin makes an ordinary JUnit configuration.