Category: gherkin

Strictly Good Gherkin with Gwen

Some general rules for writing good Gherkin include:

  • All steps in Scenarios and Backgrounds should satisfy Given-When-Then order
  • Given steps should set context
  • When steps should perform actions
  • Then steps should assert expectations

When it comes to using Gherkin features to drive automated tests, these rules are often broken. Why? Put simply, they are too easy to break when not enforced.

To help, we have introduced a gwen.behavior.rules setting in Gwen, that when set to strict, will report all such violations as errors.

The Bad and the Imperative

To demonstrate, consider the following bad feature specification that breaks most of the above rules and other good practices:

# file: gwen-workspace/features/Todo.feature

Feature: Bad todo

  Scenario: Add and complete items
      Given I navigate to "http://todomvc.com/examples/angularjs"
       Then I should be on the todo page
       When I add a "Walk the dog" item
        And I add a "Get the milk" item
       Then the active item count should be "2"
       When I tick the "Get the milk" item
       Then the active item count should be "1"
       When I tick the "Walk the dog" item
       Then the active item count should be "0"

The automation glue for this feature can be written in a Gwen meta file as follows (custom StepDefs highlighted):

# file: gwen-workspace/features/Todo.meta

Feature: Todo meta

@StepDef
Scenario: I should be on the todo page
    Given the heading can be located by tag name "h1"
     Then the heading should be "todos"
      And the todo field can be located by class name "new-todo"
      And the active item count can be located by css selector ".todo-count > strong"

@StepDef
Scenario: I add a "<todo>" item
     When I enter "$<todo>" in the todo field
     Then the "$<todo>" item can be located by xpath
          """
          //label[contains(.,'$<todo>')]/preceding-sibling::input
          """
      And the "$<todo>" item should not be ticked

Launching Gwen with the default gwen.behavior.rules=lenient setting will successfully execute the Todo.feature file.

Executing the Feature

To install Gwen and run the feature file:

  • Download and unpack this gwen-workspace.zip (Gwen workspace) to a location on your drive
  • Copy the above Todo.feature and Todo.meta files into the features folder in your workspace location
  • Open a command prompt in your workspace directory and run:
    • Linux/Mac/PowerShell:
      •  ./gwen features/Todo.feature -b
    • Windows DOS:
      •  gwen features/Todo.feature -b

Enabling Strict Behavior Checks

Now try running it with strict behavior rules enabled. Set the gwen.behavior.rules property in the gwen.properties file in your workspace to strict.

# behavior rules checking (strict|lenient)
gwen.behavior.rules=strict

Running the feature again will result in the following error:

ERROR - Given-When-Then order not satisfied by steps in Scenario at features/Todo.feature:5

This tells us that the steps in our Scenario do not follow Given-When-Then order. A brute force fix for this could be to change and order the keywords in each step and remove duplicates as follows:

# file: gwen-workspace/features/Todo.feature

Feature: Bad todo

  Scenario: Add and complete items
      Given I navigate to "http://todomvc.com/examples/angularjs"
        And I should be on the todo page
       When I add a "Walk the dog" item
       Then I add a "Get the milk" item
        And the active item count should be "2"
        And I tick the "Get the milk" item
        And the active item count should be "1"
        And I tick the "Walk the dog" item
        And the active item count should be "0"

Running this feature will now result in an new error:

ERROR - Failed step [at line 6]: Given I navigate to "http://todomvc.com/examples/angularjs": Action behavior not permitted where context is expected

You can also see this error in the report generated at target/reports/index.html in your workspace folder.

This error is telling us that we cannot perform an action (the predefined Gwen navigation action in this case) in a Given step which is meant for setting context. When steps are meant for actions but if we change the first step to use When, we will get the previous Given-When-Then order error again.

No matter how we rearrange these steps, the strict validation will always give us an error. So continuing in this manner is futile. We are forced to COMPLETELY rethink what we’re doing if we want to have Good Gherkin!

If we step back a little, it’s easy to see that our scenario has too many steps doing many things. It’s is a mish-mash of imperative operations with no clear behavior.

Let’s do this better!

The Good and the Declarative

The first step is to forget about automation for a moment and rewrite our feature in terms of behavior. Let’s make it about adding and completing items in a todo list with a feature that describes the intended behavior declaratively and conforms to good Gherkin rules like this:

# gwen-workspace/features/Todo.feature

Feature: Good todo

Scenario: Add and complete items
    Given a new todo list
     When the following items are added
          | ITEM          |
          | Get the milk  |
          | Walk the dog  |
      And all items are completed
     Then no active items should remain

Note that Gwen also has a feature mode setting that when set to ‘declarative‘ will prevent you from calling predefined and imperative Gwen DSL steps in feature files and force you to move all I click this and I enter that clutter into meta files where they belong.

All the imperative automation glue for the declarative feature steps can now be confined to our meta file as follows:

# file: features/Todo.meta

Feature: Todo meta

  @StepDef
  Scenario: a new todo list
      Given the heading can be located by tag name "h1"
       When I navigate to "http://todomvc.com/examples/angularjs"
       Then the heading should be "todos"
        And the todo field can be located by class name "new-todo"
        And the active item count can be located by css selector ".todo-count > strong"

  @StepDef
  @DataTable(header="top")
  Scenario: the following items are added
        And I add a todo item for each data record

  @StepDef
  Scenario: I add a todo item
      Given the todo item is "${data[ITEM]}"
       When I enter the todo item in the todo field
       Then the added item can be located by xpath
             """
             //label[contains(.,'${the todo item}')]/preceding-sibling::input
             """
        And the added item should be unticked
        And the active item count should be "${record.number}"

  @StepDef
  Scenario: all items are completed
       When I tick item for each item located by class name "toggle"

  @StepDef
  Scenario: no active items should remain
       Then the active item count should be "0"

The second and third StepDef‘s work together to add each item declared in the data table of the second step in our feature to the list of todo items in the app. For more on data tables, see Gwen data tables.

Custom StepDef Behavior

We’re almost there, but running this feature will result in the following error because the strict validation mechanism in Gwen does not know what type of behavior our step definitions contain.

ERROR - Failed step [at line 6]: Given a new todo list: Missing @Context, @Action, or @Assertion behavior tag on StepDef at features/Todo.meta:6

So we have to annotate each StepDef called by steps in our feature with one of the following tags:

  • @Context : for context setting StepDefs meant for Given steps
  • @Action : for action performing StepDefs meant for When steps
  • @Assertion : for assertion checking StepDefs meant for Then steps

Gwen will then know which StepDef can or cannot be called by Givens, Whens, or Thens and report violations accordingly. This is how behavior types are bound to step definitions.

Note that checking for the presence of behavior tags on StepDefs will only happen in strict behavior mode and Gwen will not perform all the same rules checks on meta files as it does on feature files (since meta files are necessarily imperative automation glue).

Adding behavior tags to all StepDefs called by steps in our feature yields the final and complete meta:

# file: features/Todo.meta

Feature: Todo meta

  @StepDef
  @Context
  Scenario: a new todo list
      Given the heading can be located by tag name "h1"
       When I navigate to "http://todomvc.com/examples/angularjs"
       Then the heading should be "todos"
        And the todo field can be located by class name "new-todo"
        And the active item count can be located by css selector ".todo-count > strong"

  @StepDef
  @Action
  @DataTable(header="top")
  Scenario: the following items are added
        And I add a todo item for each data record

  @StepDef
  Scenario: I add a todo item
      Given the todo item is "${data[ITEM]}"
       When I enter the todo item in the todo field
       Then the added item can be located by xpath
             """
             //label[contains(.,'${the todo item}')]/preceding-sibling::input
             """
        And the added item should be unticked
        And the active item count should be "${record.number}"

  @StepDef
  @Action
  Scenario: all items are completed
       When I tick item for each item located by class name "toggle"

  @StepDef
  @Assertion
  Scenario: no active items should remain
       Then the active item count should be "0"

We’re all Good Now!

Good Gherkin rules will now be satisfied and running the feature again will succeed.

gwen-workspace/target/reports/html/features/Todo/Todo.feature.html

Motivation

Adding behavior checks to Gwen was inspired by various articles and discussions about good Gherkin and current challenges in this space. Some of these include: