Going from Feature to Scenario Level State with Gwen

The Cucumber docs stress that state should not leak across scenarios and that each scenario should be self contained and executable in isolation from others. So Cucumber-like tools that wish to comply with this standard must implement a scenario-level state management mechanism to ensure that each scenario gets a new state and that it is never shared. This means that no global state should exist at the feature level. Following this practice will also enable scenarios to be safely executed in parallel.

Until recently, Gwen only supported feature-level state and parallel feature execution. We have since added scenario-level state management and parallel scenario execution capabilities to support self contained executable scenarios in features. A new setting in Gwen (gwen.state.level=scenario) has been introduced to enable and enforce this approach. In this post, we are going to convert a sample feature that uses feature-level state into one that uses scenario-level state instead.

The first thing you’ll need to do is install Gwen on your machine to get the sample we will be working with. Download this gwen-workspace.zip and unpack it somewhere on your drive and open a command prompt into the unpacked location. If you run into any trouble or would prefer to follow our install guide, you can start here and then come back.

Consider the following sample feature shipped with the Gwen distribution. It utilises feature-level state and has scenarios that depend on each other. This feature can only be executed successfully when the scenarios it contains are executed in the exact same order as they are specified. Attempting to run these scenarios in any other order (or in parallel) would result in a failed execution because each scenario has a dependency on the one before it.

gwen-workspace/samples/todo/feature-level/Todo.feature

 Feature: Create and complete Todo items

Scenario: Add items in my Todo list
    Given I launch the Todo app
     When I add a "Walk the dog" item
      And I add a "Get the milk" item
      And I add a "Feed the cat" item
     Then the number of open items should be "3"

Scenario: Complete second item
    Given I am on the Todo page
     When I complete the "Get the milk" item
     Then the number of open items should be "2"

Scenario: Complete first item
    Given I am on the Todo page
     When I complete the "Walk the dog" item
     Then the number of open items should be "1"

Scenario: Complete third item
    Given I am on the Todo page
     When I complete the "Feed the cat" item
     Then the number of open items should be "0"

Scenario: Clear my completed items
    Given I am on the Todo page
     When I clear all completed items
     Then the number of open items should not be displayed

And for reference, here is the associated Gwen meta that makes it executable:

gwen-workspace/samples/todo/feature-level/Todo.meta

 Feature: Todo meta
  
@StepDef
Scenario: I launch the Todo app
    Given I start a new browser
     When I resize the window to width 800 and height 800
      And I navigate to "http://todomvc.com/examples/angularjs"
     Then I should be on the Todo page

@StepDef
Scenario: I should be on the Todo page
    Given the heading can be located by tag name "h1"
     Then the heading should be "todos"
      And I am on the Todo page
      And the todo field can be located by class name "new-todo"
      And the number of open items can be located by css selector ".todo-count > strong"
      And the clear completed button can be located by class name "clear-completed"
      And the number of open items should not be displayed

@StepDef
Scenario: I add a "<todo>" item
    Given the "$<todo>" item can be located by xpath "//label[contains(.,'$<todo>')]/preceding-sibling::input"
     When I enter "$<todo>" in the todo field
     Then the "$<todo>" item should be unchecked

@StepDef
Scenario: I complete the "<todo>" item
     When I tick the "$<todo>" item
     Then the "$<todo>" item should be ticked

@StepDef
Scenario: I clear all completed items
     When I click the clear completed button

Before continuing, confirm that your current state level is set to feature. Check that the gwen.properties file in your workspace has gwen.state.level=feature defined. If not, then add it in to enable Feature-level state.

gwen-workspace/gwen.properties

# state level (feature|scenario)
gwen.state.level=feature

Then open a command prompt into your gwen-workspace directory and run the following command to execute the feature “as is” to verify that it works:

Linux/Mac/PowerShell:

./gwen -b samples/todo/feature-level

Windows DOS:

gwen -b samples/todo/feature-level

The execution should succeed and you can open the evaluation report generated at /gwen-workspace/target/reports/index.html to view the results.

Next, change the state level to scenario by changing the setting in the gwen.properties file:

# state level (feature|scenario)
gwen.state.level=scenario

Launch Gwen again to execute the same feature and observe the expected failure:

The second step in the “Complete second item” scenario fails with error: Undefined locator binding for the "Get the milk" item: the "Get the milk" item/locator

Scenario: Add items in my Todo list
    Given I launch the Todo app
     When I add a "Walk the dog" item
      And I add a "Get the milk" item
      And I add a "Feed the cat" item
     Then the number of open items should be "3"

The locator binding for the "Get the milk" item is created through meta in the first scenario:

@StepDef
Scenario: I add a "<todo>" item
    Given the "$<todo>" item can be located by xpath "//label[contains(.,'$<todo>')]/preceding-sibling::input"
     When I enter "$<todo>" in the todo field
     Then the "$<todo>" item should be unchecked

That binding is no longer available in the second scenario:

Scenario: Complete second item
    Given I am on the Todo page
     When I complete the "Get the milk" item
     Then the number of open items should be "2"

The is because all the state from the first scenario has been discarded. With scenario-level state management, every scenario gets it’s own new and clean state. Therefore state can no longer be shared between scenarios.

We will now proceed to make the necessary changes to enable this feature to work with scenario-level state. Copy the samples/todo/feature-level folder and the files in it into a new samples/todo/wip folder. We will make our changes to the feature and meta files in this folder now.

The first step in converting the feature is to identify all the common steps and put them into a Background. We can see that the first scenario in the feature performs the common setup required by scenarios that follow. Converting it into a Background will make the common setup repeatable for each scenario (since the Background will be executed before every scenario in the feature).

So let’s make the first Scenario a Background. Edit the gwen-workspace/samples/todo/wip/Todo.feature file by renaming the Scenario keyword in the first Scenario to Background.

gwen-workspace/samples/todo/wip/Todo.feature

Background: Add items in my Todo list
      Given I launch the Todo app
       When I add a "Walk the dog" item
        And I add a "Get the milk" item
        And I add a "Feed the cat" item
       Then the number of open items should be "3"

Now run the following command to execute the modified feature:

Linux/Mac/PowerShell:

./gwen -b samples/todo/wip

Windows DOS:

gwen -b samples/todo/wip

You will notice several things now. Firstly, what was previously the first scenario is now the Background in every scenario and what was previously the second scenario is now the first and so on. Secondly, the “Complete second item” scenario (which is now the first scenario) passes successfully since the "Get the milk" item is now bound into the scenario through the Background which is in scope. The following “Complete first item” scenario however fails on the last step with error: assertion failed: Expected the number of open items to be '1' but got '2'

Opening the attached screenshot reveals that the number of items left open is indeed 2 and not 1.

So what has happened? Well, the Background setup steps now execute at the start of each scenario and in a new browser session (as defined in meta). So every scenario now begins with the same state created by the Background. With this in mind, let’s compare the sequence of steps leading up to this point in both the original and new feature.

In the original case, the logical steps leading up to this point include:

gwen-workspace/samples/todo/feature-level/Todo.feature

    # Steps from original Scenario 1
    Given I launch the Todo app
     When I add a "Walk the dog" item
      And I add a "Get the milk" item
      And I add a "Feed the cat" item
     Then the number of open items should be "3"

    # Steps from original Scenario 2
    Given I am on the Todo page
     When I complete the "Get the milk" item
     Then the number of open items should be "2"

   # Steps from original Scenario 3
    Given I am on the Todo page
     When I complete the "Walk the dog" item
     Then the number of open items should be "1"

So originally, we added 3 items in scenario 1, completed one item in scenario 2 and completed another item in scenario 3. We were then left with 1 open item as expected.

Lets now look at the logical steps leading to the same point in our new feature:

gwen-workspace/samples/todo/wip/Todo.feature

      # Steps from new Background
      Given I launch the Todo app
       When I add a "Walk the dog" item
        And I add a "Get the milk" item
        And I add a "Feed the cat" item
       Then the number of open items should be "3"

      # Steps from new Scenario 2 (which failed)
      Given I am on the Todo page
       When I complete the "Get the milk" item
       Then the number of open items should be "1"

So now we added 3 items in the Background and completed one item in scenario 2. We were then left with 2 open items where previously we had 1.

In the original case, scenario 3 depends on scenario 2 which depends on scenario 1. In the new case, there is no such dependency and therefore the extra “Get the milk” item that was completed by scenario 2 in the original flow is no longer completed in the new flow.

We therefore need to modify our scenario that failed to include completing the additional “Get the milk” item explicitly. While we’re at it, it would make sense to rename this scenario to make it clear that two items are being completed. Let’s do that now.

gwen-workspace/samples/todo/wip/Todo.feature

Scenario: Complete second and first item
    Given I am on the Todo page
     When I complete the "Get the milk" item
      And I complete the "Walk the dog" item
     Then the number of open items should be "1"

Then execute the feature again.

Linux/Mac/PowerShell:

./gwen -b samples/todo/wip

Windows DOS:

gwen -b samples/todo/wip

We run into the same error again for the next scenario, but this time because we haven’t explicitly completed two cumulative items. So we update that scenario to complete them. We also update the last scenario which we know will require all three items to be completed and rename the scenarios accordingly.

Our conversion is now complete and the new feature now looks like this (with all changes highlighted):

   Feature: Create and complete Todo items

Background: Add items to my Todo list
      Given I launch the Todo app
       When I add a "Walk the dog" item
        And I add a "Get the milk" item
        And I add a "Feed the cat" item
       Then the number of open items should be "3"

  Scenario: Complete second item
      Given I am on the Todo page
       When I complete the "Get the milk" item
       Then the number of open items should be "2"

  Scenario: Complete second and first item
      Given I am on the Todo page
       When I complete the "Get the milk" item
        And I complete the "Walk the dog" item
       Then the number of open items should be "1"

  Scenario: Complete second, first, and third item
      Given I am on the Todo page
       When I complete the "Get the milk" item
        And I complete the "Walk the dog" item
        And I complete the "Feed the cat" item
       Then the number of open items should be "0"

  Scenario: Complete and clear all items
      Given I am on the Todo page
       When I complete the "Get the milk" item
        And I complete the "Walk the dog" item
        And I complete the "Feed the cat" item
        And I clear all completed items
       Then the number of open items should not be displayed

Launching the feature now will result in a successful execution. The changes made to support scenario-level state will enable the scenarios to be safely executed in parallel too! To do that, just launch the same command again but this time with the --parallel option.

But Wait!

In this post we converted a test that used feature-level state to one that uses scenario-level state. We did this for the sake of example, but there are a couple of things worth noting.

1) The original feature-level version performed one complete test in one browser session.
2) The resulting scenario-level version performs multiple incremental tests in many browser sessions.

Should we roll up all these scenarios into a single scenario instead? In this case we probably should and we will explore doing that in the next post. Stay tuned!

As promised, here is the follow up post: Refactoring executable Gherkin with Gwen to eliminate redundancy.

Published by

Branko Juric

Imperative by day and functional by night. Co author of the Gwen automation platform.

One thought on “Going from Feature to Scenario Level State with Gwen”

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s