Investigating failures
When a scenario fails, the spec file itself shows which step failed and why, without opening the generated Java code or searching through the console output. An icon appears in the gutter beside the failed step, and a click on that icon opens a panel directly beneath the step with the details of the failure.
The failure mark on a step
A failed step has an icon on its keyword line, and the icon tells what kind of failure it was:
an assertion failure — the step checked a value, and the value was not the expected one. The
panel beneath the step then shows both the expected and the actual value.
an error — the step stopped with an unexpected error rather than a failed assertion, so no
expected value was compared.
an abort — the scenario was stopped early, before it could pass or fail, because a JUnit
assumption in one of its steps did not hold.
The scenario's own run icon shows the failure too, so a failed scenario is easy to spot while scrolling through the file.
The error panel
Click the failure icon and a panel opens below the step. The panel always starts with the failure message and ends with the stack trace, which starts collapsed — click it to expand. What appears in between depends on the kind of failure. Click the icon again to close the panel.
Assertion failure
The step checked a value and the value did not match. Below the message, the panel shows the Expected value and the Actual value, each on its own row, so the difference is easy to see. A long value shows only its first line until clicked.
With both values present, the panel also has Approve and Show diff buttons — for accepting the actual value into the spec file, or comparing the two side by side. Both are covered on Approval workflow. An assertion that reports no expected and actual values, such as a plain true-or-false check, shows the message alone.
Error
The step stopped with an unexpected error rather than a failed assertion — for example, a service that could not be reached. No value was compared, so the panel has no Expected or Actual rows and no buttons. It shows only the error message and the stack trace.
Abort
The scenario was stopped early, before it could pass or fail. In JUnit this happens when an assumption in a step method does not hold. An assumption states something the test needs before the test can run at all — for example, that a pricing service is reachable:
Assumptions.assumeTrue(pricingService.isAvailable(), "pricing service is unreachable");
When the condition is false, JUnit stops the scenario and marks the scenario as aborted rather than failed. An abort does not mean the behaviour is wrong, only that the scenario could not be tested this time.
The stack trace
Every kind of panel ends with the stack trace. The trace starts collapsed, showing only its first line, which names the error. Click the arrow beside that line to show the whole trace, and click again to hide it.
Each class reference in the trace, such as ShoppingCartTest.java:46, is a link that opens that
Java class in the editor at that line. Lines from the project's own code are styled differently from lines in
libraries, so they stand out; the colour and font style for those lines are set under
Settings → Tools → SpecBinder → Execution results — see Settings.