"BDD is not X" and "X is not BDD" and why we need BDA or a similar classification

This post was inspired by a discussion I had on this twitter thread:

I’m proposing a new term such as BDA (Behaviour Driven Automation) or something similar to be introduced to classify the testing and automation part of the BDD process so that discussions and advancements in this space can progress unhindered by “That’s not BDD” commentary.

Say you are building a web application and have adopted a BDD practice and your team has authored some Gherkin features that describe intended behaviour in a declarative manner with steps that conform to all the prescribed BDD rules.

What follows is based on my work on the open source Gwen project. It is not a BDD process by any means but rather what I have been calling BDA (A for Automation). Similar could potentially be done with other tooling.

At some point you will want to use some features “as is” to automate some testing. Gwen enables you to do exactly this by mapping the declarative Gherkin in your features to imperative Gherkin that you define in separate meta files. The Gherkin in these meta files must conform to the prescribed Gwen Web DSL. This meta has nothing to do with BDD but everything to do with web automation and assertion checking which is all necessary imperative behaviour. When you invoke Gwen, it will bind your features to your meta at runtime and execute them for you to achieve automation. So you don’t have to develop any page objects, selenium code or test framework. You just compose some Gherkin meta to describe the automation behaviour.

That’s BDA. Your declarative BDD features are executable and remain untouched.

Automating Declarative Gherkin with Gwen

Writing Gherkin feature specifications with automation in mind? Something like the following perhaps?

Feature: Google search

Scenario: Simple Google search
Given I start a new browser
 When I navigate to "http://www.google.com"
 Then the page title should be "Google"
And the search field can be located by name "q"
When I enter "gwen automation" in the search field
Then the page title should contain "gwen"
And the first result can be located by css selector ".r > a"
When I click the first result
Then the current URL should match regex ".+[G|g]wen.*"

Stop!

The above is what we call an imperative specification. It describes a sequence of steps and checks that a tester (for example) might perform in a robotic fashion when evaluating the Google search page. Gherkin like this is clearly written with implementation and testing in mind.

If you were in a team tasked to develop the Google search page, would you write Gherkin like the above upfront to describe how the search should behave or would you write it like the below instead?

 Feature: Google search

Scenario: Simple Google search
    Given I have Google in my browser
     When I do a search for "Gwen automation"
     Then I should find a Gwen page

This second example is more declarative, unlike the first. It describes the intended behaviour without specifying any implementation or testing detail. Specifications like this can be used to capture requirements, drive development and testing too (what I like to call: first class specifications). Everyone, regardless of their role in a development team can understand this easily.

Now let’s say the team want’s to adopt a test-first practice and have decided to adopt Gwen. They could then just download the gwen-workspace to a location on their drive, save the above feature (the 2nd one) to a GoogleSearch.feature file in the features folder, and then launch Gwen in the workspace directory to evaluate it as follows:

Linux/Mac/PowerShell:

./gwen -b features/GoogleSearch.feature

Windows DOS:

gwen -b features/GoogleSearch.feature

Of course, the test would fail because nothing has been implemented yet and Gwen also doesn’t know how to interpret the first step:

ERROR - Failed step [at line 4]: Given I have Google in my browser: Unsupported or undefined step: Given I have Google in my browser
You can publish your workspace to a git repository and share it with your team.

Now let’s say the developers have implemented the search functionality and now want to automate a test.

Now it’s time to provide Gwen with step definitions for each step in the feature. This is done in Gwen with meta features and predefined Gwen web DSL steps. So your declarative feature will remain intact and unchanged. Create a Google.meta file in the features directory (alongside the GoogleSearch.feature file you created earlier) with the following content:

Feature: Google meta (automation glue)

@StepDef
Scenario: I have Google in my browser
Given I start a new browser
When I navigate to "http://www.google.com"
Then the page title should be "Google"

@StepDef
Scenario: I do a search for "<query>"
Given the search field can be located by name "q"
When I enter "$<query>" in the search field
Then the page title should contain "$<query>"

@StepDef
Scenario: I should find a Gwen page
Given the first result can be located by css selector ".r > a"
When I click the first result
Then the current URL should match regex ".+[G|g]wen.*"

This is how step definitions are defined in Gwen (in Gherkin too). Notice that each Scenario has a @StepDef annotation. This tells Gwen to load these scenarios into memory and only execute them when a step in the feature with the same name is processed. The steps in each StepDef scenario are predefined DSL steps that Gwen already knows how to execute so you don’t have to write any Selenium code.

So now all the imperative steps used to drive automation that we had in our first imperative feature are all confined to a separate meta file instead. Gwen will auto-discover this meta file since we have put it in the same directory as the feature. It will bind the all the imperative steps in the meta to the declarative steps in our feature without us having to modify the feature whatsoever!

With this meta in place, let’s run Gwen again as before to evaluate the feature:

Linux/Mac/PowerShell:

./gwen -b features/GoogleSearch.feature

Windows DOS:

gwen -b features/GoogleSearch.feature

This time, the Google search page will load, a search will be performed, the first result will be clicked and a Gwen page will be displayed. The lucky Google search behaviour is now verified.

See it in action

The evaluation reports will be available at target/reports/index.html under your workpsace location.

Clicking each step in the report will reveal the underlying step definition.

Development teams can use Gwen like this to evaluate implemented features against declarative Gherkin specifications without changing or polluting them with imperative steps or automation clutter. All of that is confined to Gwen meta.

For those who like this approach, we have introduced a new declarative feature mode in Gwen which enforces this practice. There’s been a lot of talk in the Gherkin/Cucumber community about how feature files should describe intended behaviour only and be automation agnostic. Introducing this mode is the first step we have taken to help you to write cleaner and better Gherkin without sacrificing executability.

Refactoring executable Gherkin with Gwen to eliminate redundancy

In the previous post we walked through an example of converting an executable Gherkin specification from feature to scenario level state.

The example we converted was this one:

gwen-workspace/samples/todo/feature-level/Todo.feature

 Feature: Create and complete Todo items

Scenario: Add items in my Todo list
    Given I launch the Todo app
     When I add a "Walk the dog" item
      And I add a "Get the milk" item
      And I add a "Feed the cat" item
     Then the number of open items should be "3"

Scenario: Complete second item
    Given I am on the Todo page
     When I complete the "Get the milk" item
     Then the number of open items should be "2"

Scenario: Complete first item
    Given I am on the Todo page
     When I complete the "Walk the dog" item
     Then the number of open items should be "1"

Scenario: Complete third item
    Given I am on the Todo page
     When I complete the "Feed the cat" item
     Then the number of open items should be "0"

Scenario: Clear my completed items
    Given I am on the Todo page
     When I clear all completed items
     Then the number of open items should not be displayed

In this example, all scenarios execute one after the other in the exact order shown and accumulate feature-level state. Each scenario depends on the one before it and executing them in a different order would result in execution failure. Writing executable Gherkin in this manner in not considered good practice since the scenarios are not self contained and leak state from one scenario to the next. Each scenario does however perform a test (or check) and the specification as a whole executes in one browser session to load three items, complete them individually and then clear the completed list.

The result of the conversion from feature to scenario level state was the following specification:

gwen-workspace/samples/todo/scenario-level/Todo.feature

   Feature: Create and complete Todo items

Background: Add items to my Todo list
      Given I launch the Todo app
       When I add a "Walk the dog" item
        And I add a "Get the milk" item
        And I add a "Feed the cat" item
       Then the number of open items should be "3"

  Scenario: Complete second item
      Given I am on the Todo page
       When I complete the "Get the milk" item
       Then the number of open items should be "2"

  Scenario: Complete second and first item
      Given I am on the Todo page
       When I complete the "Get the milk" item
        And I complete the "Walk the dog" item
       Then the number of open items should be "1"

  Scenario: Complete second, first, and third item
      Given I am on the Todo page
       When I complete the "Get the milk" item
        And I complete the "Walk the dog" item
        And I complete the "Feed the cat" item
       Then the number of open items should be "0"

  Scenario: Complete and clear all items
      Given I am on the Todo page
       When I complete the "Get the milk" item
        And I complete the "Walk the dog" item
        And I complete the "Feed the cat" item
        And I clear all completed items
       Then the number of open items should not be displayed

In this converted example, each scenario gets its own new and clean state. The scenarios are independent of each other and a common background sets up the same precondition for each one. The scenarios can execute in any order and do not leak any state. Each scenario performs a test (or check) in a new browser session.

However, there is a lot of redundancy here. For each scenario, the background starts a new browser session and adds three items to the list. Since there are four scenarios, this happens four times. The first scenario completes one of these items. The second scenario completes the same item that the first one did before completing another one. The third scenario completes the same two items as the last two did before completing the last remaining item. The fourth scenario then does everything all over again before clearing all items from the list. The specification takes longer to execute than the original which used feature-level state. Even if we execute these scenarios in parallel, we will still have lesser performance.

But fear not, we will now refactor this example into the single scenario below that will perform everything just once and eliminate all redundancy!

gwen-workspace/samples/todo/wip/Todo.feature

   Feature: Todo

  Scenario: Create and complete Todo items
      Given I have three open items
       When I complete each open item
        And I clear all completed items
       Then I should have no items left

If you haven’t installed Gwen, now is a good time to do it (just download and unpack the gwen-workspace.zip to a location on your drive).

Create a new samples/todo/wip folder in your Gwen workspace and save the new single scenario feature above in a file called Todo.feature in that directory.

Recall the following Gwen meta from the previous post that was used to make both our feature and scenario level specifications executable. Copy this meta specification to a file called Todo.meta in the newly created wip folder (the feature and meta files should both be there).

gwen-workspace/samples/todo/wip/Todo.meta

 Feature: Todo meta
  
@StepDef
Scenario: I launch the Todo app
    Given I start a new browser
     When I resize the window to width 800 and height 800
      And I navigate to "http://todomvc.com/examples/angularjs"
     Then I should be on the Todo page

@StepDef
Scenario: I should be on the Todo page
    Given the heading can be located by tag name "h1"
     Then the heading should be "todos"
      And I am on the Todo page
      And the todo field can be located by class name "new-todo"
      And the number of open items can be located by css selector ".todo-count > strong"
      And the clear completed button can be located by class name "clear-completed"
      And the number of open items should not be displayed

@StepDef
Scenario: I add a "<todo>" item
    Given the "$<todo>" item can be located by xpath "//label[contains(.,'$<todo>')]/preceding-sibling::input"
     When I enter "$<todo>" in the todo field
     Then the "$<todo>" item should be unchecked

@StepDef
Scenario: I complete the "<todo>" item
     When I tick the "$<todo>" item
     Then the "$<todo>" item should be ticked

@StepDef
Scenario: I clear all completed items
     When I click the clear completed button

We will be adding a few more step definitions to this meta to make our new feature executable.

Lets start with the first step in our Scenario:

Given I have three open items

Our step definition for this step should perform the exact same steps that the background in our previous feature performed. Create that step definition now by adding the following to the end of the gwen-workspace/samples/todo/wip/Todo.meta file:

@StepDef
Scenario: I have three open items
    Given I launch the Todo app
     When I add a "Walk the dog" item
      And I add a "Get the milk" item
      And I add a "Feed the cat" item
     Then the number of open items should be "3"

The @StepDef annotation tells Gwen that this is a step definition. Note that the scenario name matches the expression of the first Given step in our new feature. This is how step definitions work in Gwen.

We will now verify that this step definition works by testing it in the Gwen REPL. Open a command prompt in the root of your gwen-workspace directory and run the following command to load the meta and launch the REPL.

Linux/Mac/PowerShell:

./gwen -m samples/todo/wip/Todo.meta

Windows DOS:

gwen -m samples/todo/wip/Todo.meta
The meta will load and the gwen> will open. Enter our first feature step in the prompt and press enter to evaluate it.
gwen> Given I have three open items

Gwen will perform all the steps defined in our step definition for this step. The browser session will remain open once the step completes and you will see the three items added.

We have verified that our step definition works. Type exit at the gwen prompt to exit the REPL.

We will now move on to the next step in our scenario:

When I complete each open item

We will implement our step definition for this step by completing each of the items one at a time and verify the number of completed items displayed in the bottom left as we go. Add the following step definition to your meta:

@StepDef
Scenario: I complete each open item
    Given I am on the Todo page
     When I complete the "Get the milk" item
     Then the number of open items should be "2"
     When I complete the "Walk the dog" item
     Then the number of open items should be "1"
     When I complete the "Feed the cat" item
     Then the number of open items should be "0"
Launch the REPL again to load the meta as before. Press the up arrow a couple of times to recall the first step we evaluated earlier and press enter on it. Gwen will run the same previous steps as before and load all our items in to the list. Once that is done, type the second scenario step into the prompt and press enter to execute it. Gwen will complete all items as per the step definition we just defined.
gwen> When I complete each open item

Moving onto the third step:

And I clear all completed items

We don’t need to define a new step definition for this one because we already have it defined in the original meta we started with. If you still have the REPL session open, you can type the above step directly into the REPL and it will execute. In case you missed the step definition for this, here it is below:

@StepDef
Scenario: I clear all completed items
     When I click the clear completed button

Now we just need to add the following step definition for the final step in our scenario. The meta entry for that follows:

@StepDef 
Scenario: I should have no items left
     Then the number of open items should not be displayed

We have now completed defining all the step definitions we need. Exit the currently open REPL session if you have it open and run the following command to execute our new feature spec:

Linux/Mac/PowerShell:

./gwen -b samples/todo/wip/Todo.feature

Windows DOS:

gwen -b samples/todo/wip/Todo.feature

That’s it. We’re done!

We have successfully refactored our executable specification and reduced it to a single scenario without all the redundancy we had before. Since this scenario is also self contained, we can execute it in parallel with any other scenarios that we might add to the feature later.

Open the report at gwen-workspace/target/reports to see the results.

Click on each step in the report to drill down into the step definition.

Going from Feature to Scenario Level State with Gwen

The Cucumber docs stress that state should not leak across scenarios and that each scenario should be self contained and executable in isolation from others. So Cucumber-like tools that wish to comply with this standard must implement a scenario-level state management mechanism to ensure that each scenario gets a new state and that it is never shared. This means that no global state should exist at the feature level. Following this practice will also enable scenarios to be safely executed in parallel.

Until recently, Gwen only supported feature-level state and parallel feature execution. We have since added scenario-level state management and parallel scenario execution capabilities to support self contained executable scenarios in features. A new setting in Gwen (gwen.state.level=scenario) has been introduced to enable and enforce this approach. In this post, we are going to convert a sample feature that uses feature-level state into one that uses scenario-level state instead.

The first thing you’ll need to do is install Gwen on your machine to get the sample we will be working with. Download this gwen-workspace.zip and unpack it somewhere on your drive and open a command prompt into the unpacked location. If you run into any trouble or would prefer to follow our install guide, you can start here and then come back.

Consider the following sample feature shipped with the Gwen distribution. It utilises feature-level state and has scenarios that depend on each other. This feature can only be executed successfully when the scenarios it contains are executed in the exact same order as they are specified. Attempting to run these scenarios in any other order (or in parallel) would result in a failed execution because each scenario has a dependency on the one before it.

gwen-workspace/samples/todo/feature-level/Todo.feature

 Feature: Create and complete Todo items

Scenario: Add items in my Todo list
    Given I launch the Todo app
     When I add a "Walk the dog" item
      And I add a "Get the milk" item
      And I add a "Feed the cat" item
     Then the number of open items should be "3"

Scenario: Complete second item
    Given I am on the Todo page
     When I complete the "Get the milk" item
     Then the number of open items should be "2"

Scenario: Complete first item
    Given I am on the Todo page
     When I complete the "Walk the dog" item
     Then the number of open items should be "1"

Scenario: Complete third item
    Given I am on the Todo page
     When I complete the "Feed the cat" item
     Then the number of open items should be "0"

Scenario: Clear my completed items
    Given I am on the Todo page
     When I clear all completed items
     Then the number of open items should not be displayed

And for reference, here is the associated Gwen meta that makes it executable:

gwen-workspace/samples/todo/feature-level/Todo.meta

 Feature: Todo meta
  
@StepDef
Scenario: I launch the Todo app
    Given I start a new browser
     When I resize the window to width 800 and height 800
      And I navigate to "http://todomvc.com/examples/angularjs"
     Then I should be on the Todo page

@StepDef
Scenario: I should be on the Todo page
    Given the heading can be located by tag name "h1"
     Then the heading should be "todos"
      And I am on the Todo page
      And the todo field can be located by class name "new-todo"
      And the number of open items can be located by css selector ".todo-count > strong"
      And the clear completed button can be located by class name "clear-completed"
      And the number of open items should not be displayed

@StepDef
Scenario: I add a "<todo>" item
    Given the "$<todo>" item can be located by xpath "//label[contains(.,'$<todo>')]/preceding-sibling::input"
     When I enter "$<todo>" in the todo field
     Then the "$<todo>" item should be unchecked

@StepDef
Scenario: I complete the "<todo>" item
     When I tick the "$<todo>" item
     Then the "$<todo>" item should be ticked

@StepDef
Scenario: I clear all completed items
     When I click the clear completed button

Before continuing, confirm that your current state level is set to feature. Check that the gwen.properties file in your workspace has gwen.state.level=feature defined. If not, then add it in to enable Feature-level state.

gwen-workspace/gwen.properties

# state level (feature|scenario)
gwen.state.level=feature

Then open a command prompt into your gwen-workspace directory and run the following command to execute the feature “as is” to verify that it works:

Linux/Mac/PowerShell:

./gwen -b samples/todo/feature-level

Windows DOS:

gwen -b samples/todo/feature-level

The execution should succeed and you can open the evaluation report generated at /gwen-workspace/target/reports/index.html to view the results.

Next, change the state level to scenario by changing the setting in the gwen.properties file:

# state level (feature|scenario)
gwen.state.level=scenario

Launch Gwen again to execute the same feature and observe the expected failure:

The second step in the “Complete second item” scenario fails with error: Undefined locator binding for the "Get the milk" item: the "Get the milk" item/locator

Scenario: Add items in my Todo list
    Given I launch the Todo app
     When I add a "Walk the dog" item
      And I add a "Get the milk" item
      And I add a "Feed the cat" item
     Then the number of open items should be "3"

The locator binding for the "Get the milk" item is created through meta in the first scenario:

@StepDef
Scenario: I add a "<todo>" item
    Given the "$<todo>" item can be located by xpath "//label[contains(.,'$<todo>')]/preceding-sibling::input"
     When I enter "$<todo>" in the todo field
     Then the "$<todo>" item should be unchecked

That binding is no longer available in the second scenario:

Scenario: Complete second item
    Given I am on the Todo page
     When I complete the "Get the milk" item
     Then the number of open items should be "2"

The is because all the state from the first scenario has been discarded. With scenario-level state management, every scenario gets it’s own new and clean state. Therefore state can no longer be shared between scenarios.

We will now proceed to make the necessary changes to enable this feature to work with scenario-level state. Copy the samples/todo/feature-level folder and the files in it into a new samples/todo/wip folder. We will make our changes to the feature and meta files in this folder now.

The first step in converting the feature is to identify all the common steps and put them into a Background. We can see that the first scenario in the feature performs the common setup required by scenarios that follow. Converting it into a Background will make the common setup repeatable for each scenario (since the Background will be executed before every scenario in the feature).

So let’s make the first Scenario a Background. Edit the gwen-workspace/samples/todo/wip/Todo.feature file by renaming the Scenario keyword in the first Scenario to Background.

gwen-workspace/samples/todo/wip/Todo.feature

Background: Add items in my Todo list
      Given I launch the Todo app
       When I add a "Walk the dog" item
        And I add a "Get the milk" item
        And I add a "Feed the cat" item
       Then the number of open items should be "3"

Now run the following command to execute the modified feature:

Linux/Mac/PowerShell:

./gwen -b samples/todo/wip

Windows DOS:

gwen -b samples/todo/wip

You will notice several things now. Firstly, what was previously the first scenario is now the Background in every scenario and what was previously the second scenario is now the first and so on. Secondly, the “Complete second item” scenario (which is now the first scenario) passes successfully since the "Get the milk" item is now bound into the scenario through the Background which is in scope. The following “Complete first item” scenario however fails on the last step with error: assertion failed: Expected the number of open items to be '1' but got '2'

Opening the attached screenshot reveals that the number of items left open is indeed 2 and not 1.

So what has happened? Well, the Background setup steps now execute at the start of each scenario and in a new browser session (as defined in meta). So every scenario now begins with the same state created by the Background. With this in mind, let’s compare the sequence of steps leading up to this point in both the original and new feature.

In the original case, the logical steps leading up to this point include:

gwen-workspace/samples/todo/feature-level/Todo.feature

    # Steps from original Scenario 1
    Given I launch the Todo app
     When I add a "Walk the dog" item
      And I add a "Get the milk" item
      And I add a "Feed the cat" item
     Then the number of open items should be "3"

    # Steps from original Scenario 2
    Given I am on the Todo page
     When I complete the "Get the milk" item
     Then the number of open items should be "2"

   # Steps from original Scenario 3
    Given I am on the Todo page
     When I complete the "Walk the dog" item
     Then the number of open items should be "1"

So originally, we added 3 items in scenario 1, completed one item in scenario 2 and completed another item in scenario 3. We were then left with 1 open item as expected.

Lets now look at the logical steps leading to the same point in our new feature:

gwen-workspace/samples/todo/wip/Todo.feature

      # Steps from new Background
      Given I launch the Todo app
       When I add a "Walk the dog" item
        And I add a "Get the milk" item
        And I add a "Feed the cat" item
       Then the number of open items should be "3"

      # Steps from new Scenario 2 (which failed)
      Given I am on the Todo page
       When I complete the "Get the milk" item
       Then the number of open items should be "1"

So now we added 3 items in the Background and completed one item in scenario 2. We were then left with 2 open items where previously we had 1.

In the original case, scenario 3 depends on scenario 2 which depends on scenario 1. In the new case, there is no such dependency and therefore the extra “Get the milk” item that was completed by scenario 2 in the original flow is no longer completed in the new flow.

We therefore need to modify our scenario that failed to include completing the additional “Get the milk” item explicitly. While we’re at it, it would make sense to rename this scenario to make it clear that two items are being completed. Let’s do that now.

gwen-workspace/samples/todo/wip/Todo.feature

Scenario: Complete second and first item
    Given I am on the Todo page
     When I complete the "Get the milk" item
      And I complete the "Walk the dog" item
     Then the number of open items should be "1"

Then execute the feature again.

Linux/Mac/PowerShell:

./gwen -b samples/todo/wip

Windows DOS:

gwen -b samples/todo/wip

We run into the same error again for the next scenario, but this time because we haven’t explicitly completed two cumulative items. So we update that scenario to complete them. We also update the last scenario which we know will require all three items to be completed and rename the scenarios accordingly.

Our conversion is now complete and the new feature now looks like this (with all changes highlighted):

   Feature: Create and complete Todo items

Background: Add items to my Todo list
      Given I launch the Todo app
       When I add a "Walk the dog" item
        And I add a "Get the milk" item
        And I add a "Feed the cat" item
       Then the number of open items should be "3"

  Scenario: Complete second item
      Given I am on the Todo page
       When I complete the "Get the milk" item
       Then the number of open items should be "2"

  Scenario: Complete second and first item
      Given I am on the Todo page
       When I complete the "Get the milk" item
        And I complete the "Walk the dog" item
       Then the number of open items should be "1"

  Scenario: Complete second, first, and third item
      Given I am on the Todo page
       When I complete the "Get the milk" item
        And I complete the "Walk the dog" item
        And I complete the "Feed the cat" item
       Then the number of open items should be "0"

  Scenario: Complete and clear all items
      Given I am on the Todo page
       When I complete the "Get the milk" item
        And I complete the "Walk the dog" item
        And I complete the "Feed the cat" item
        And I clear all completed items
       Then the number of open items should not be displayed

Launching the feature now will result in a successful execution. The changes made to support scenario-level state will enable the scenarios to be safely executed in parallel too! To do that, just launch the same command again but this time with the --parallel option.

But Wait!

In this post we converted a test that used feature-level state to one that uses scenario-level state. We did this for the sake of example, but there are a couple of things worth noting.

1) The original feature-level version performed one complete test in one browser session.
2) The resulting scenario-level version performs multiple incremental tests in many browser sessions.

Should we roll up all these scenarios into a single scenario instead? In this case we probably should and we will explore doing that in the next post. Stay tuned!

As promised, here is the follow up post: Refactoring executable Gherkin with Gwen to eliminate redundancy.

Locating Web Elements with Gwen

Gwen supports locating elements on web pages using all the standard Selenium selectors and additionally by JavaScript too. In this post, we are going to explore them. To make it a little more fun we will use the Gwen REPL which will enable us to verify our locator expressions interactively against a running web application in a browser as we go.

The first thing you’ll need to do is install Gwen somewhere on your machine. It’s simple to do. Just download this gwen-workspace.zip and unpack it somewhere on your drive and open a command prompt into the unpacked location. If you run into any trouble or would prefer to follow our install guide, you can start here and then come back.

Once you’re ready, type the following command at the terminal in the root gwen-workspace directory where you installed Gwen:

Linux/Mac/PowerShell:

./gwen

Windows DOS:

gwen

The first time you launch Gwen it will take just a little time to download the latest version of itself. Subsequent launches won’t have to do that and will start faster.

The REPL will start with a gwen> prompt open:

   __ ___      _____ _ __     _
  / _` \ \ /\ / / _ \ '_ \   { \,"
 | (_| |\ V  V /  __/ | | | {_`/
  \__, | \_/\_/ \___|_| |_|   `
  |___/

Welcome to gwen-web v2.35.1

INFO - Environment context initialised
INFO - gwen-web.sh -r target/reports -p gwen.properties,browsers/chrome.properties,env/local.properties

REPL Console

Enter steps to evaluate or type exit to quit..

gwen> 

In the examples that follow we are borrowing the Selenium Test Page from automationtesting.com to demonstrate locating web elements using all of the available mechanisms in Gwen.

Keep in mind that Gwen is case sensitive. So you will need to use the same casing that is shown in all the examples here.

Enter the following step into the REPL to have Gwen open up a new browser session with the Selenium Test Page all ready and loaded (you can copy and paste the step if you like):

When I navigate to "https://automationintesting.com/selenium/testpage/"

Locating elements with Selenium selectors

We’ll start with ID locators which simply locate elements by their HTML id attribute. The Selenium Test page that we have open in the browser has a First name field in it. If we right click and inspect that field in the browser we will see that it is defined in HTML as follows:

<input id="firstname" type="text" placeholder="Enter your first name">

Enter the following step into Gwen REPL prompt to define a locator named firstNameById for finding this field:

Then firstNameById can be located by id "firstname"

This will instruct Gwen to bind the name firstNameById to the By.id("firstname") Selenium locator under the hood. We can test that it works by entering the following at the REPL prompt:

When I locate firstNameById

Gwen will find and highlight the field in yellow inside the browser session that we opened. If you missed it, rearrange your REPL and browser windows a bit so you can see both side by side and repeat the last command (you can use the up arrow in the REPL to recall the last command).

Demo of locating a web element with Gwen on FriendlyTester‘s Selenium Test Page

Now that we have proven that our firstNameById locator binding works, we can perform an operation on the field by referencing it by name in any one of the predefined Web DSL steps that Gwen supports for web automation. For example, we can type a name into the form by entering the following step into the REPL:

And I type "Gwen" in firstNameById

Gwen will locate and enter the name in the field.

Similarly, we can also locate the form in the page which is defined in the page HTML as:

<form id="contactus" name="contactform">

Let’s locate by name this time. Enter the following step into Gwen REPL prompt to define a locator named formByName for finding this form:

Given formByName can be located by name "contactform"

This will instruct Gwen to bind the name formByName to the By.name("contactform") Selenium locator . We can test it by entering the following at the REPL prompt:

When I locate formByName

Gwen will find and highlight the form in inside the browser. We can then submit the form by entering the following step into the REPL:

And I submit formByName

Gwen will locate and submit the form. Let’s exit the REPL and see how we can use the locators we just defined to execute a Gherkin feature file. Type exit at the gwen> prompt to return to your OS.

exit

Create a subfolder named se-test inside the features subfolder of your Gwen workspace. Then create and save a new text file named SeleniumTestPage.feature in that subfolder with the following content:

 Feature: Selenium Test Page

Scenario: Submit the test form
    Given I am on the Selenium test page
     When I provide some contact details
      And I submit the test form
     Then nothing should happen

Create and save another text file named SeleniumTestPage.meta in the same subfolder with the following content:

 Feature: Selenium Test Page Meta

@StepDef
Scenario: I am on the Selenium test page
     When I navigate to "https://automationintesting.com/selenium/testpage/"
     Then firstNameById can be located by id "firstname"
      And formByName can be located by name "contactform"

@StepDef
Scenario: I provide some contact details
     When I type "Gwen" in firstNameById

@StepDef
Scenario: I submit the test form
     When I submit formByName

@StepDef
Scenario: nothing should happen
     Then the current URL should end with "?"

Type the following command at your OS terminal prompt to have Gwen load the meta file and bind it to the feature file to automate execution:

Linux/Mac/PowerShell:

./gwen features/se-test

Windows DOS:

gwen features/se-test

Gwen will execute the feature by performing all the same actions we performed earlier in the REPL and will leave the REPL session open when done. The locators we defined are now bound in Gwen’s memory and we can continue experimenting with them in the REPL if we wish. Enter the following command to have Gwen submit the form again:

When I submit formByName

Enter the 3rd step in our feature directly into the REPL:

And I submit the test form

Gwen will match it to the 3rd @StepDef annotated Scenario in the meta file by name (this is how step definitions work in Gwen) and execute all the steps that we defined there. Type exit at the Gwen REPL prompt when you are ready to return to your OS.

We just demonstrated how you can locate an element by id with Gwen. You can use the same approach to locate elements using any one of the other standard Selenium locator mechanisms. Borrowing some locator examples from Friendly Tester’s Free Selenium Course, here is how you would define the equivalents in Gwen:

Given genderByClass can be located by class name "gender"

  And firstNameByXPath can be located by xpath "//*[@id="firstname"]"

  And linkByTag can be located by tag name "a"

  And linkByCSS can be located by css selector "div a"

  And linkByLinkText can be located by link text "Our Policy"

  And linkByPartialText can be located by partial link text "Policy"

If you load these locators into the Gwen REPL or define them in your meta file, you will then be a able to perform operations on those elements such as:

When I select "Female" in genderByClass

And I clear firstNameByXPath

And I click linkByTag

Locating elements with JavaScript

We said at the beginning that Gwen can also locate elements by JavaScript. Sometimes this can help us locate dynamically rendered elements more reliably.

If you wanted to, you could define the form locator using a vanilla JavaScript expression like this :

Given formByJS can be located by javascript "document.getElementsByName('contactform')[0]"

Or a form locator using a JQuery expression (if the jquery library is loaded in the browser page) like this:

And formByJQuery can be located by javascript "$('#contactus').get(0)"

Or in extreme cases where the standard locators do not suffice and you have to resort to executing a custom script on a page to find an element, you could do so with an anonymous JavaScript function in a DocString like this:

And elemByJSFunc can be located by javascript
      """
      (function() {
        var elem = ..
          // dynamic logic to find element 
          // based on some fancy rules,
          // conditions or what have you
        return elem;
      })();
      """

To test this last one in the REPL, you would have to use the paste mode since the step spans multiple lines.

That’s it! We hope this has been a good small introduction to locators in Gwen for you.

Visual Testing with Gwen and AppliTools

AppliTools have made visual UI testing extremely simple with their online service. Automating basic visual tests with their Eyes SDK involves just three essential activities:

  1. Starting a new visual test session
  2. Saving one or more checkpoint images
  3. Closing the session and checking the results

Some of our Gwen users that we support have recently enquired about visual testing and we decided to provide it by integrating with AppliTools in gwen-web version 2.32.0.

To perform visual testing with Gwen, you will need:

Set API key in your Environment

Once you have the above, you need to set the APPLITOOLS_API_KEY environment variable on your host (where Gwen will run) to your API key value.

Linux

export APPLITOOLS_API_KEY=your-api-key

Windows

set APPLITOOLS_API_KEY=your-api-key

Start a Visual Test Session

Consider the following executable Gwen feature (and associated meta available here):

Feature:

 Feature: Todo items

Scenario: Add items in my Todo list
    Given I launch the Todo app
     When I add a "Walk the dog" item
      And I add a "Get the milk" item
     Then the number of active items should be "2"

Scenario: Complete one item
     When I tick the "Get the milk" item
     Then the number of active items should be "1"

Scenario: Complete another item
     When I tick the "Walk the dog" item
     Then the number of active items should be "0"

Now let’s say we want to add some visual testing to this.

The first thing we will need to do is instruct Gwen to start a new visual test session and give it a name and viewport size. For example, the following Gwen step will create a new visual test session named “Todo items” and set the viewport size to 600 pixels high and wide:

  • I start visual test as "Todo items" in 600 x 600 viewport (see DSL here)

We can insert this at line 5 in the feature above as follows:

 Feature: Todo items

Scenario: Add items in my Todo list
    Given I launch the Todo app
      And I start visual test as "Todo items" in 600 x 600 viewport
     When ..

Checkpoint Screenshot Images

With the visual test session started, we can now start taking screenshots of the viewport and save them as checkpoints in AppliTools with a name and match level. For example, we can checkpoint the initial state of the Todo UI after the application has loaded and our visual test session has started. At this stage, the Todo UI will have no items in it and we can save the checkpoint of that image under the name “No Todo items” in AppliTools using the STRICT (layout and content) match level as follows:

  • I check viewport visual as "No Todo items" using STRICT match (see DSL here)

We can insert this at line 6 in the feature above as follows:

 Feature: Todo items

Scenario: Add items in my Todo list
    Given I launch the Todo app
      And I start visual test as "Todo items" in 600 x 600 viewport
      And I check viewport visual as "No Todo items" using STRICT match
     When ..

Similarly, we can checkpoint the screenshot after adding two items as follows (line 10 below):

 Feature: Todo items

Scenario: Add items in my Todo list
    Given I launch the Todo app
      And I start visual test as "Todo items" in 600 x 600 viewport
      And I check viewport visual as "No Todo items" using STRICT match
     When I add a "Walk the dog" item
      And I add a "Get the milk" item
     Then the number of active items should be "2"
      And I check viewport visual as "Active Todo items" using STRICT match
      ..

And checkpoint the screenshot after we complete (tick) each item as follows (lines 15 and 20 below):

 Feature: Todo items

Scenario: Add items in my Todo list
    Given I launch the Todo app
      And I start visual test as "Todo items" in 600 x 600 viewport
      And I check viewport visual as "No Todo items" using STRICT match
     When I add a "Walk the dog" item
      And I add a "Get the milk" item
     Then the number of active items should be "2"
      And I check viewport visual as "Active Todo items" using STRICT match

Scenario: Complete one item
     When I tick the "Get the milk" item
     Then the number of active items should be "1"
      And I check viewport visual as "One completed Todo item" using STRICT match

Scenario: Complete another item
     When I tick the "Walk the dog" item
     Then the number of active items should be "0"
      And I check viewport visual as "All completed Todo items" using STRICT match

This gives us a total of four checkpoint images for our visual test.

Close Session and Check Results

The final step is to close the session and check that the visual test has passed. In Gwen we can do this with the following DSL step:

  • the visual test should pass (see DSL here)

We can insert this line at the end of the feature as follows:

 Feature: Todo items

Scenario: Add items in my Todo list
    Given I launch the Todo app
      And I start visual test as "Todo items" in 600 x 600 viewport
      And I check viewport visual as "No Todo items" using STRICT match
     When I add a "Walk the dog" item
      And I add a "Get the milk" item
     Then the number of active items should be "2"
      And I check viewport visual as "Active Todo items" using STRICT match

Scenario: Complete one item
     When I tick the "Get the milk" item
     Then the number of active items should be "1"
      And I check viewport visual as "One completed Todo item" using STRICT match

Scenario: Complete another item
     When I tick the "Walk the dog" item
     Then the number of active items should be "0"
      And I check viewport visual as "All completed Todo items" using STRICT match
      And the visual test should pass

This last step will close the visual test session in AppliTools and return successfully if all checkpoint images pass their respective match level checks. This first time you run a visual test, AppliTools will baseline all checkpoint images and return a successful result. Subsequent runs will compare all respective images with that baseline using the comparison strategy defined by the match level specified at each checkpoint. Any failures will result in an error that Gwen will report.

AppliTools Dasbhoard

Whether all visual checks pass or not, Gwen will provide a link in the console and in the evaluation report to the online AppliTools dashboard containing the detailed results.

Console:

INFO - Evaluating Step: And the visual test should pass
INFO - Visual check status: Passed. Details at: https://applitools/dashboard-url
INFO - [406ms] Passed Step: And the visual test should pass

HTML Report:

Dashboard:

From the AppliTools dashboard, you can inspect and resolve any failures.

Launching the Example

The sample feature described in this post and associated meta are bundled in every Gwen distribution and are available on GitHub here also:

If you install Gwen, you can launch the example in your installation directory on the command line as follows (if you installed a Gwen workspace, you can omit the -r target/reports parameter, don’t forget to set the APPLITOOLS_API_KEY environment variable as described earlier):

Linux

./gwen -b -r target/reports features/visual/todo

Windows

gwen -b -r target/reports features/visual/todo

The Gwen evaluation report will be generated here (relative to your install directory): target/reports/index.html

The example here inlines the visual testing steps in the feature itself, but you could choose to externalise them to Gwen meta to keep your features clean in your own projects if you wish.

More Info

You can also override various AppliTools defaults and runtime aspects in Gwen. For this and more details, see Gwen visual testing on our wiki.

More to Come

Gwen currently supports very basic visual testing with AppliTools (as described above). AppliTools itself has much more features and capabilities which we have not yet fully explored or integrated. We will integrate more capabilities as time goes on and as users start providing us with feedback.

Thanks

We thank the team at AppliTools for helping and supporting us and hope our Gwen users will like this feature and find it very useful.

Gwen in Summary

Simpler web automation.

A Java runtime, a web browser, your favourite text editor, and the Gwen interpreter is all you need to start automating.

No clickety-click drag-drop mouse-pointy UI or dev-centric page-object code-compiling IDE or plugin required.

The interpreter is open source.

The automation is reusable, scalable, and shareable.

Many execution modes including sequential, parallel, dry run, and interactive REPL are provided.

Many reporting formats including JUnit XML, cucumber JSON, and pretty HTML are supported.

JavaScript can be injected in places if needed.

A CLI makes it easy to run locally and integrate with any on-premise or on-cloud build server.

Git ready workspaces make it easy for teams to collaborate.

A package manager downloads and installs native browser drivers for you.

Built on Gherkin and Selenium.

Easy to run on BrowserStack, Sauce Labs, Aerokube Selenoid and other grids.

Matured in industry for over 4 years.

Active user community is growing.

Gwen home page: http://gweninterpreter.org

Share with your friends.

Feedback welcomed.