Gwen Supports Full Gherkin Syntax

With the recent addition of Data Tables and Doc Strings, Gwen now supports the full Gherkin syntax! We’ve also enhanced the REPL console with a paste mode to enable multi-line step evaluation. The latest web engine has been released with all the new capabilities.

This is a major milestone for Gwen. Thank you to all the users who raised issues or requests and to everyone who evaluated the web engine and provided feedback. It really helps us to keep making Gwen better when we hear from you. 🙂

Advertisements

Gwen Automation with Data Tables

Gwen now supports data tables and in this post I’m going to show you some examples of how they can be used with the web engine to automate some repetitive tasks on the TodoMVC web application.

Data tables are handy for when you want to work with lists of items in Gherkin, such as the list of todo items shown in this example below.

File: features/todo/tables/TodoTables.feature


 Feature: Managing todo items with data tables

Scenario: Complete a todo item

    Given I have the following active items
          | Walk the dog |
          | Get the milk |
          | Feed the cat |

     When I complete the "Get the milk" item

     Then the status of my items should be
          | Item         | Status    |
          | Walk the dog | active    |
          | Get the milk | completed |
          | Feed the cat | active    |

Note that this example is a high level (or declarative) feature specification that describes intended behaviour and not a low level (or imperative) list of statements that describe how automation is to be achieved or on what type of system it will be performed on. Those are automation bindings that can be captured in separate meta files that are also expressed in Gherkin when using Gwen. As it turns out, we already have a meta file that specifies some step definitions that call out to predefined steps in the web engine to interact with the TodoMVC app. One of the capabilities of Gwen is the ability to reuse this meta, so we will reuse it here to help compose some step definitions for our data tables above. You can have a look at that Todo.meta file here if you like.

Data Table Bindings

Looking at the first Given step in our feature, we can see that it contains three todo items in a table. The step implies that we have those three items in our todo list. So what we want to do is load those three items into our list in the todo app when running this step. To do that, we need to create a step definition for it that will launch the app, load the items, and verify that they are all active. We can do that with some new meta as follows:

File: features/todo/tables/TodoTables.meta

 Feature: TodoMVC tables meta

@StepDef
@DataTable(horizontal="Item")
Scenario: I have the following active items
    Given I launch the Todo app
     When I add a "${data[1][Item]}" item
      And I add a "${data[2][Item]}" item
      And I add a "${data[3][Item]}" item
     Then the "${data[Item]}" item should be active for each data record

This meta is also a Gherkin specification with a tagged scenario containing all the steps that will perform the desired operations on the data table.

  • This first @StepDef tag tells Gwen to load this scenario to memory and only execute its steps when a step in a feature references it by name. Note that the name of this scenario is the same expression used in the Given step in the feature. This is how step definitions work in Gwen.
  • The second @DataTable tag contains an attribute that declares the table to contain horizontal data records and that each one of those contains just one data cell that we will call “Item”. We need to do this since the table in the feature does not include a header record at the top and we have to assign each cell a name so we can access its data by that name.
  • The steps from lines 7 to 9 load the items one at a time by explicitly referencing the data in each record
  • The last step on line 10 checks that each item is active by iterating over each record of the table

With this in place and the latest web engine installed, we can now invoke the feature with Gwen as follows.

gwen -b features/todo/tables/TodoTable.feature

Gwen will automatically discover and load the automation bindings in all meta files that exist in the path of the feature file which in our case includes both the Todo.meta we want to reuse and the TodoTables.meta we just created. The Given step in the feature will execute to load the table items and verify that they are all active, and the When step will complete the second item. The Then step in the feature will fail as expected because we haven’t defined a step definition for it yet. The web browser output is shown below:

Similarly, we now add a step definition for the last Then step in our feature to our meta to verify the status of all items and complete all our binding work.

    • The table in this step includes a header record, so we specify header="top" in the @DataTable tag to link the names in that header to the cells in each data record.

    File: features/todo/tables/TodoTables.meta

     Feature: TodoMVC tables meta
    
    @StepDef
    @DataTable(horizontal="Item")
    Scenario: I have the following active items
        Given I launch the Todo app
         When I add a "${data[1][Item]}" item
          And I add a "${data[2][Item]}" item
          And I add a "${data[3][Item]}" item
         Then the "${data[Item]}" item should be active for each data record
    
    @StepDef
    @DataTable(header="top")
    Scenario: the status of my items should be
         Then the "${data[Item]}" item should be ${data[Status]} for each data record
    

    Now the entire feature is executable!

    Simplifying it Further

    For the sake of demonstrating explicit table data access, the steps from lines 7 to 9 in our meta process each record one at a time. Imagine if we had a lot more records? There would be a lot of redundancy wouldn’t there? We can remove this by reducing these steps into a single step that loops through each record in the same way that the steps on line 10 and 15 do.

    File: features/todo/tables/TodoTables.meta

     Feature: TodoMVC tables meta
    
    @StepDef
    @DataTable(horizontal="Item")
    Scenario: I have the following active items
        Given I launch the Todo app
         When I add a "${data[Item]}" item for each data record
         Then the "${data[Item]}" item should be active for each data record
    
    @StepDef
    @DataTable(header="top")
    Scenario: the status of my items should be
         Then the "${data[Item]}" item should be ${data[Status]} for each data record
    

    Now there is no need to explicitly specify the index for each record when referencing each data cell. But for cases where you might need to cross reference different records or really want to be explicit in how you access each data cell, the earlier approach of not iterating will give you the flexibility you need. It really depends on your particular use case and you should always choose the approach that works the best in the given circumstance.

    The feature and meta files created here are available in our samples features/todo/tables folder on Github and are also bundled with the Gwen-web binary distribution. If you download and install it, you can execute the feature in the same way that we did above in your installation directory.

    And, that’s it! Our feature with tables is executable and all redundancy has been removed from the bindings. We have mapped declarative feature steps to imperative steps in meta and reused some existing meta too! 🙂

Evaluating Gwen with Modern JavaScript Web Apps

Gwen-web was designed to take the development pain out of web automation and make it easier to achieve with all types of web applications regardless of the underlying technologies or frameworks they were built with. Our goal was to make Gwen work consistently with web pages built on any kind of server or client side framework. All web pages are just HTML documents when they hit the browser after all. But this HTML is not always static and with the recent trend of modern JavaScript framework adoption, web pages are getting more and more dynamic. A lot more things happen in web pages nowadays. Lots of JavaScript code gets loaded and all types of browser events trigger functions and Ajax requests at various times resulting in all sorts of dynamic and asynchronous rendering that makes for a very rich user experience. This might make things more pleasant for human users but it is often a hard challenge for automation programs (or robots).

The Evaluation Test

So I thought I’d try Gwen out on some web pages built on the popular and modern JS frameworks of today to see how well it performs. To do this, I wrote a feature suite that mimics these Serenity tests and ran it over the various JS implementations of the publicly available TodoMVC web application (this is a project which offers the same Todo web application implemented on different JavaScript frameworks).

It is important to note that Gwen uses the Java implementation of the Selenium web driver and is not a JS framework itself but rather a Java executable that reads plain text Gherkin features and dynamically interprets them into automated web page interactions. The evaluation performed here provides us with an indication of how well Gwen interacts with web applications built on different types of JS frameworks.

The Evaluation Results

The suite consists of 26 scenarios which I ran over 33 different JS implementations of the Todo app (for a total of 858 scenarios). I ran it in a Chrome browser on a Mac. It took about 50 minutes to run every scenario in sequence. I also ran the tests in parallel (on a dual CPU quad core machine) which took about half the time and produced consistent results. Gwen was executed with the gwen.feature.failfast setting overriden to false to force all scenarios in a feature to execute even if one or more prior ones fail.

94% of Scenarios Passed 🙂

All tests completed successfully and passed for the following frameworks.

  • JavaScript:
    • Backbone.js
    • AngularJS
    • Ember.js
    • KnockoutJS
    • Dojo
    • Knockback.js
    • CanJS
    • Polymer
    • React
    • Mithril
    • Ampersand
    • Vue.js
    • Marionette.js
    • Vanilla JS
    • Vanilla ES6
    • jQuery
  • Compile-to-JS:
    • Spine
    • Dart
    • GWT
    • TypeScript + Backbone.js
    • TypeScript + AngularJS
    • TypeScript + React
    • Serenade.js
    • Reagent
    • Scala.js + React
    • Scala.js + Binding.scala
    • js_of_ocaml
    • Humble + GopherJS

6% of Scenarios Failed

Some tests failed for the following frameworks. I have not investigated these in detail but have included my initial findings below.

  • JavaScript:
    • Flight
      • 15 scenarios failed, 11 passed
      • Intermittent element hits and misses
      • Entered data leaks or is lost
    • TroopJS + RequireJS
      • 15 scenarios failed, 11 passed
      • Unresponsive to enter key being sent to field
  • Compile-to-JS:
    • Closure
      • 5 scenarios failed, 21 passed
      • Unresponsive to checkbox being clicked
    • Elm
      • 8 scenarios failed, 18 passed
      • Intermittently not sending some characters to field
    • AngularDart
      • 3 scenarios failed, 23 passed
      • Intermittently not rendering conditionally visible buttons

Conclusion


Gwen interacted successfully with a large majority of the popular JS implementations. All tests passed for 28 of the 33 implementations evaluated. More than 94% of all scenarios passed in total.

Try it Yourself

All the todoMVC features used in this evaluation are also bundled in the gwen-web distribution as of release 2.3.3. If you install the latest gwen-web distribution, you can run it by calling the following command in the directory where you install Gwen (to run in parallel mode, just add the --parallel switch):

  • Windows:
    • gwen features/todoMVC
  • Linux:
    • ./gwen features/todoMVC

Gwen Gets Scenario Outlines

After some consideration, full support for scenario outlines has been added to gwen-web as of version 2.3.0 to give users more power and better support BDD.

Standard Execution

The following example shows a meta file that defines a step definition for joining two strings together and a feature file containing a scenario outline that exercises it with different string values.

Meta file:

Feature: Join Strings Meta

  @StepDef
  Scenario: I join the two strings
      Given the result is "${string 1}${string 2}"

Feature file:

Feature: Join Strings

  Scenario Outline: Joining <string 1> and <string 2> should yield <result>

    This scenario is evaluated at the point where the outline is declared.
    Joining <string 1> and <string 2> should yield <result>

    Given string 1 is "<string 1>"
      And string 2 is "<string 2>"
     When I join the two strings
     Then the result should be "<result>"

    Examples: Basic string concatenation

      The header row contains the placeholder names. The body rows that
      follow contain the data that is bound to each scenario that is evaluated.

      | string 1 | string 2 | result   |
      | howdy    | doo      | howdydoo |
      | any      | thing    | anything |

The Gwen interpreter will expand and evaluate each record in the body of the Examples table at the point where the scenario outline is declared in the feature. This example contains just one Examples clause in the outline but many can be specified.

Evaluated scenario outlines appear in HTML reports as follows.

Each Examples table record in the report (excluding the header) is rendered as a hyperlink that when hovered over and clicked will reveal the expanded scenario.

Outlines as StepDefs for Deferred Execution

For cases where you may want to reuse a scenario outline across many scenarios or features, you can make it a step definition in the meta and then call it by name in the feature where you want it to execute.

Meta file:

Feature: Join Strings Meta

  @StepDef
  Scenario: I join the two strings
      Given the result is "${string 1}${string 2}"

  @StepDef
  Scenario Outline: Joining <string 1> and <string 2> should yield <result>

    This outline is loaded into memory and execution is deferred until a call is made.
    Joining <string 1> and <string 2> should yield <result>

    Given string 1 is "<string 1>"
      And string 2 is "<string 2>"
     When I join the two strings
     Then the result should be "<result>"

    Examples: Basic string concatenation

      The header row contains the placeholder names. The body rows that
      follow contain the data that is bound to each scenario that is evaluated.

      | string 1 | string 2 | result   |
      | howdy    | doo      | howdydoo |
      | any      | thing    | anything |

Feature file:

Feature: Join Strings

  Scenario: Join different pairs of strings and compare their results
      Given Joining <string 1> and <string 2> should yield <result>

The Gwen interpreter will load the outline in the meta to memory and execute it when it is called by the scenario in the feature. Any number of scenarios in any features can call the outline in this way as long as the meta is in scope.

Calls to scenario outlines are rendered as steps in HTML reports (just like all calls to step definitions are).

When the step is hovered over and clicked, the called outline is revealed as a StepDef.

Each Examples table record in the report (excluding the header) is rendered as a hyperlink that when hovered over and clicked will reveal the expanded scenario.

Notes about outlines as StepDefs

  • They are reusable
  • They can be called from anywhere, including backgrounds and the REPL console too!
  • If the name of the outline contains data placeholders, then their bound table values are substituted into the expanded scenario name prefixes at evaluation time. Specifying placeholders in the name is handy in this way but it is also optional. We could have named the outline ‘joining two strings should yield a compound string‘ instead and called it by that name if we wanted to. In this case the expanded scenarios would all have this same name prefix. Either way, the expanded scenarios will always be given unique names that are suffixed with the example record number (see report snapshots above). If you are using placeholders in the name though, then your call to the step definition should also include the same placeholder name literals (as shown in this example) to prevent potential name clashes with any other step definitions you may have with a similar name. If a name clash does occur, Gwen will report it and you will need to rename.

No Page Objects – There’s no long way to go. We’re already there!

Specifications driven web automation with no page objects or selenium coding ~ We’ve done it!

Page Objects

I recently discovered a Page Objects Refactored article which suggests that ‘flaky’ web tests are often misattributed to Selenium web driver issues and that the problem instead is poorly coded page objects that were not written with solid OO principles in mind. Programming languages allow programmers to achieve the same or similar results in more than one way. For this reason, design patterns were introduced to help developers implement known solutions to known problems. But even design patterns require discipline and cannot be strictly enforced to any useful degree without introducing a specialised framework. Frameworks also are geared at software developers and not software users.

What is needed is a pre-programmed automation tool that frees users from development concerns.

No Page Objects

One of the primary goals of the gwen-web project is to give users a tool for automating web pages without needing to develop any page objects or compile any code. Gwen-web is an interpreter that accepts plain text Gherkin features as input and produces executing web driver instructions as output. All interactions with the web driver API happen in a precompiled and embedded web engine through a prescribed DSL. This web engine also handles many of the known gotchas and pitfalls that programmers typically face when working directly with the web driver. Users also have the flexibility to compose custom DSLs (their own step definitions) which are also expressed in Gherkin. With this approach, developing page objects and adopting design patterns is no longer required. Furthermore, programmer errors are eliminated and ‘flakiness’ is removed. The ability to compose specifications in plain Gherkin text is all that is necessary.

An example

Consider the following Gherkin feature:

 Feature: Google search

Scenario: Perform a google search
    Given I do a google search for "gwen-web"
     When I click the first result
     Then the page title should contain "gwen-interpreter/gwen-web"
      And the current URL should be "https://github.com/gwen-interpreter/gwen-web"

With Gwen, we can take a plain text feature like this one and execute it “as is”.

If you install gwen-web, you will find the above feature file located at features\google\GoogleSearch.feature relative to your root install directory (the location where you unpacked the distribution). If you open a command prompt to this location, you can execute this feature as follows:

On Windows:

gwen -b features\google\GoogleSearch.feature

On a Mac:

./gwen -b features/google/GoogleSearch.feature

When this is launched, gwen-web will:

  • Open a new browser session
  • Navigate to the google home page
  • Submit a Google search
  • Click the first result that comes back
  • Verify that the title of the resulting page contains some expected text
  • Check the URL of the resulting page

How does it work?

Alongside the GoogleSearch.feature file, you will also find a Google.meta file. This file is also a Gherkin feature file, but with a .meta file extension. It defines the step definition for the first step in our feature. The remaining steps are all predefined in the web engine. We don’t need to define step definitions for those since the web engine already knows how to execute them.

Here are the contents of the Google.meta file (minus the comments):

 Feature: Google search

@StepDef
Scenario: I do a google search for "<query>"
    Given I navigate to "http://www.google.com"
      And the search field can be located by name "q"
     When I enter "$<query>" in the search field
     Then the page title should start with "$<query>"
      And the first match can be located by class name "r"
      And the first result can be located by tag name "a" in the first match

This meta defines an annotated scenario named: I do a google search for "<query>". This is how you define a step definition with Gwen. It’s simply just a @StepDef annotated scenario that in this case accepts a <query> string as a parameter followed by a sequence of steps that define its operations. When this meta file is processed by Gwen, the step definition is loaded and made available to any executing feature that is passed in. In our example, Gwen automatically discovers this meta file (because it is in the same directory as the feature itself) and preloads the step definition defined in it to memory. When the first step in our feature executes, Gwen matches it to the step definition by name and binds the passed in “gwen-web” string literal to the <query> parameter which is then used by the steps therein to perform the Google search.

Comparing the feature step with the step definition reveals the match that Gwen is able to detect:

I do a google search for "gwen-web"
I do a google search for "<query>"

Another example

You could also express this Serenity example as a feature specification like this:

features\todo\CompleteATodo.feature

   Feature: Complete a Todo

Background: Open a new browser
      Given I start a browser for James

  Scenario: I should be able to complete a todo
      Given I browse to the application home page
       When I add a "Walk the dog" item
        And I add a "Put out the garbage" item
        And I complete the "Walk the dog" item
       Then the "Walk the dog" item should be completed

  Scenario: I should see the number of todos decrease when an item is completed
      Given I browse to the application home page
       When I add a "Walk the dog" item
        And I add a "Put out the garbage" item
        And I complete the "Put out the garbage" item
       Then the number of items left should be "1"

And make it executable with a meta specification like this:

features\todo\Todo.meta

 Feature: Todo meta

@StepDef
Scenario: I browse to the application home page
    Given I navigate to "http://todomvc.com/examples/angularjs/#/"
     Then the todo field can be located by id "new-todo"
      And the number of items left can be located by javascript "document.getElementById('todo-count').children[0]"
      And I locate the todo field

@StepDef
Scenario: I add a "<todo>" item
    Given I enter "$<todo>" in the todo field
      And the "$<todo>" item can be located by xpath "//*[@class='view' and contains(.,'$<todo>')]//label"
      And the "$<todo>" item checkbox can be located by xpath "//*[@class='view' and contains(.,'$<todo>')]//input[@type='checkbox']"

@StepDef
Scenario: I complete the "<todo>" item
    Given I click the "$<todo>" item checkbox

@StepDef
Scenario: the "<todo>" item should be completed
    Given the "$<todo>" item checkbox should be checked

And then launch it like this (assuming you have created the above two files):

Windows:

gwen -b features\todo\CompleteATodo.feature

Mac:

./gwen -b features/todo/CompleteATodo.feature

Composable automation

With Gwen we don’t have to do any heavy development work. We only need to compose specifications.

Notice that all the technical automation glue (JavaScript expressions, XPath locators etc..) are confined to meta. The features themselves are clean, readable, self documenting, and easily understood. You could easily have a BA or tester write the feature files upfront and then have a developer plug in some meta behind it. Or you could have tech savvy testers with enough HTML and JavaScript skills to write the meta themselves. Teams can choose to be as declarative or as imperative as they like in how they write their features. Tailored meta can always be crafted to suit.

So there you have it ~ web automation with #NoPageObjects and no selenium coding!

If you would like to see a more detailed example, then see our Automation by Meta blog post. See also our Wiki and FAQ pages for more information about Gwen.

Automation By Meta

All that is needed to drive automation from specifications is at least one more specification.

In the previous post we used the REPL console in gwen-web to interactively complete the floodio challenge one step at a time. In this post we will use a lot of the same steps again but will automate the entire challenge through a feature file instead. Like before, there will be no need to compile any code. But unlike before, we will introduce some custom steps and compose step definitions for those in a meta file. Before proceeding, be sure to install gwen-web if you have not done so already and open a command prompt to your installation directory.

Writing the Feature

Features should be self documenting.

We start by writing a FloodIO.feature file containing the content shown below. For convenience, create this file in the root of your gwen-web installation. You can use any plain text editor you like. This will be the feature specification that we will use to automate the challenge.

   Feature: Complete the floodio challenge
       As a gwen user
       I want to automate the floodio challenge
       So that I can verify that it works

  Scenario: Launch the challenge
      Given I launch the floodio challenge
       Then I should be on the start page

  Scenario: Complete step 1
      Given I am on the start page
       When I click the Start button
       Then I should be on the step 2 page

  Scenario: Complete step 2
      Given I am on the step 2 page
       When I select "21" in the how old are you dropdown
        And I click the next button
       Then I should be on the step 3 page
       
  Scenario: Complete step 3
      Given I am on the step 3 page
       When I select and enter the largest order value
        And I click the next button
       Then I should be on the step 4 page
   
  Scenario: Complete step 4
      Given I am on the step 4 page
       When I click the next button
       Then I should be on the step 5 page

  Scenario: Complete step 5
      Given I am on the step 5 page
       When I enter the one time token
        And I click the next button
       Then I should be on the challenge completed page

The above specification consists of a feature declaration followed by several scenarios. The feature declaration is merely documentation. It has a name followed by a narrative. Each scenario has a name and a sequence of steps. If you look carefully you will discover that the steps in the feature are not the same set of steps we previously used to complete the challenge in the REPL. There are several reasons for this:

  1. We are writing the feature first and want it to read well.
  2. We are grouping steps into scenarios.
  3. We expect that all steps will execute one after the other in rapid succession when sourced from a feature file. In the REPL, pages had plenty of time to load between manually entering steps. But in batch mode we will not get any free page loading time between steps. We therefore have to introduce additional steps that will explicitly wait for pages to load before subsequent steps interact with those pages.

Writing the Meta

We are now ready to identify and compose the custom steps we have introduced. The easiest way to do this is to launch the feature and define a step definition for every step that fails to execute because it is undefined, and then repeat the process until there are no more undefined steps. We start using that approach now and issue the following command in the root of our gwen-web installation to launch the feature in batch mode (using the -b option). Batch mode forces the Gwen session to exit as soon as the feature exits.

Windows:

gwen -b FloodIO.feature

Mac:

./gwen -b FloodIO.feature

automation-by-meta-1

The output shows that the first step fails:

ERROR - Unsupported step: Given I launch the floodio challenge

This error reports that an unsupported step has been found and identifies which one it was. We can be sure now that this is a custom step and so immediately compose a step definition for it. To do this, create a new file called FloodIO.meta in the root of your gwen-web installation (alongside the feature file) and update it with the following content. Again, use any plain text editor you like.

   Feature: floodio meta
  
  @StepDef
  Scenario: I launch the floodio challenge
      Given I navigate to "https://challengers.flood.io/start"

Notice that we composed this new step definition by simply declaring a Scenario that:

  • Is annotated with the @StepDef tag
  • Has the name ‘I launch the floodio challenge‘. Note that the ‘Given’ keyword prefix is omitted from this name.
  • Contains one step that launches the browser and navigates to the challenge url.

The above step definition contains only a single step. This effectively makes ‘I launch the floodio challenge‘ an alias for ‘I navigate to https://challengers.flood.io/start‘. This may not seem very useful but what it does do in this instance is externalise the URL to the meta file. The URL is considered configuration data and therefore should be in the meta file and not the feature file.

Now when this meta is loaded, the bound navigation step will execute whenever ‘I launch the floodio challenge‘ is referenced by a step running in the interpreter.

The ‘When’, ‘Then’, ‘And’, or ‘But’ keywords would also work in place of the ‘Given’ keyword in the calling step. Which keyword you use is up to you, but you should consider both fluency and context when authoring your own features.

Next, we launch the feature again to confirm that the step now works. But this time we do it in interactive mode (without using the -b option) so that the browser remains open and the REPL mode starts after the feature exits.

Windows:

gwen FloodIO.feature

Mac:

./gwen FloodIO.feature

automation-by-meta

Gwen will automatically find the meta file if it is in the same directory as (or somewhere in the path of) the feature file. If you saved the meta somewhere else, then you can specify its location using the -m option.

The step we just composed now works and its output looks good. The start page has loaded in the browser and the console has reported a ‘Passed’ status for it. But now we get an undefined error on the next step:

ERROR - Failed step [at line 28]: Then I should be on the start page:
Unsupported or undefined step: Then I should be on the start page

So we proceed to define this step in our meta file as follows:

   
   Feature: floodio meta
        
  @StepDef
  Scenario: I launch the floodio challenge
      Given I navigate to "https://challengers.flood.io/start"
      
  @StepDef 
  Scenario: I should be on the start page
      Given I wait until "$('h1').text().trim() == 'Welcome to our Script Challenge'"
       Then I am on the start page
        And the heading can be located by tag name "h1"
        And the heading should be "Welcome to our Script Challenge"
        And the Start button can be located by name "commit"

As before, we defined this step definition as a Scenario and annotated it with the @StepDef tag, but this time we named it ‘I should be on the start page‘ and bound five steps to it:

  • The first step waits for the javascript predicate “$('h1').text().trim() == 'Welcome to our Script Challenge'” to return true. We consider the start page to be loaded when the heading is rendered in the browser. Simply checking the heading is sufficient in this case, but other pages on other sites could require more sophisticated checks. It’s up to you to discover what constitutes a page load and tailor a suitable predicate to match.
  • The second step puts the start page in scope
  • The third step defines the locator binding for the heading
  • The fourth step checks that the rendered heading content is what we expected. Although the predicate in the first step already does this, it is good practice to always check the heading (or some other text) on every page after it loads regardless. It just so happens in this instance that the predicate checks the same heading. But this will not always be so.
  • The fifth step defines the locator binding for the Start button

We now type exit in the previous Gwen session to close it and launch the feature again to confirm that the above works.

Windows:

gwen FloodIO.feature

Mac:

./gwen FloodIO.feature

automation-by-meta

We observe from the output that the custom step does work and that we have progressed beyond the first page and have made it to the second page of the challenge in the browser. We also observe that the following two steps executed successfully:

Given I am on the start page
 When I click the Start button

This is because these are not custom steps, but rather predefined steps in gwen-web. The embedded web engine in the interpreter knows how to execute them without us having to do anything. But we do again fail with a similar error to the one we got last time on the custom step that follows.

ERROR - Failed step [at line 33]: Then I should be on the step 2 page:
Unsupported or undefined step: Then I should be on the step 2 page

The Completed Meta

Repeating the process for the remaining custom steps (and doing some refactoring as we go to eliminate redundancies) yields the complete meta specification below. Most of the steps in this meta are borrowed from the previous post. We simply bound them here to step definitions and mapped them by name to the custom steps in our feature.

   Feature: floodio meta
  
  Scenario: Initialise
      Given the heading can be located by tag name "h2"
        And the next button can be located by class name "btn"
       
  @StepDef
  Scenario: I launch the floodio challenge
      Given I navigate to "https://challengers.flood.io/start"
  
  @StepDef 
  Scenario: I should be on the start page
      Given I wait until "$('h1').text().trim() == 'Welcome to our Script Challenge'"
       Then I am on the start page
        And the heading can be located by tag name "h1"
        And the heading should be "Welcome to our Script Challenge"
        And the Start button can be located by name "commit"

  @StepDef
  Scenario: I should be on the step 2 page
      Given I wait until "$('h2').text().trim() == 'Step 2'"
       Then I am on the step 2 page
        And the heading should be "Step 2"
        And the how old are you dropdown can be located by id "challenger_age"

 @StepDef
  Scenario: I should be on the step 3 page
      Given I wait until "$('h2').text().trim() == 'Step 3'"
       Then I am on the step 3 page
        And the heading should be "Step 3"
        And the largest order value is defined by javascript "Math.max.apply(Math, $.map($('.radio'), function(x) { return parseInt($(x).text()); }))"
        And the largest order input field can be located by id "challenger_largest_order"
        And the largest order radio button can be located by javascript "$('.radio:contains(${the largest order value}) input').get(0);"
      
  @StepDef  
  Scenario: I should be on the step 4 page
      Given I wait until "$('h2').text().trim() == 'Step 4'"
       Then I am on the step 4 page
        And the heading should be "Step 4"
        
  @StepDef
  Scenario: I should be on the step 5 page
      Given I wait until "$('h2').text().trim() == 'Step 5'"
       Then I am on the step 5 page
        And the heading should be "Step 5"
        And the one time token can be located by css selector ".token"
        And the one time token field can be located by id "challenger_one_time_token"
        
  @StepDef
  Scenario: I should be on the challenge completed page
      Given I wait until "$('h2').text().trim() == 'You're Done!'"
       Then I am on the challenge completed page
        And the heading should be "You're Done!"
     
  @StepDef
  Scenario: I select and enter the largest order value
      Given I type the largest order value in the largest order input field
        And I click the largest order radio button
  
  @StepDef
  Scenario: I enter the one time token
      Given I wait for the one time token text
        And I type the one time token in the one time token field

Conclusion

With the meta complete and all custom steps resolved, we now have a fully executable feature file for automating the floodio challenge. We did this by writing the feature specification first and composing the meta specification second to make the feature executable “as is” (without modifying it whatsoever). Launching the feature should now always result in success, unless something has changed in the application or it suddenly goes offline.

Windows:

gwen -b FloodIO.feature

Mac:

./gwen -b FloodIO.feature

automation-by-meta-3

Reports

If you want to also generate reports:

Windows:

gwen -b -r target\reports FloodIO.feature

Mac:

./gwen -b -r target/reports FloodIO.feature

A HTML report will be available at target/reports/index.html.

Tips

The approach presented here can be used to automate any web application and works consistently well across the four major browsers. The core tips for doing it right include:

  • Composing a custom step that will take you to the entry point of the application. The step definition for this should include all the necessary steps to get you there.
  • Composing custom wait steps for each page. For any given page, you must first wait for that page to load through an appropriately tailored JavaScript predicate before putting the page in scope, binding its locators, and performing actions and assertions on it.
  • Creating all conditions and asserting all expectations for completeness.

Next Time

In the next post we will look at properties and see how they can be used to bind environment and user data.

Update – 16 July 2015
See configurable user properties with gwen-web.