Gwen Gets Scenario Outlines

After some consideration, full support for scenario outlines has been added to gwen-web as of version 2.3.0 to give users more power and better support BDD.

Standard Execution

The following example shows a meta file that defines a step definition for joining two strings together and a feature file containing a scenario outline that exercises it with different string values.

Meta file:

Feature: Join Strings Meta

  @StepDef
  Scenario: I join the two strings
      Given the result is "${string 1}${string 2}"

Feature file:

Feature: Join Strings

  Scenario Outline: Joining <string 1> and <string 2> should yield <result>

    This scenario is evaluated at the point where the outline is declared.
    Joining <string 1> and <string 2> should yield <result>

    Given string 1 is "<string 1>"
      And string 2 is "<string 2>"
     When I join the two strings
     Then the result should be "<result>"

    Examples: Basic string concatenation

      The header row contains the placeholder names. The body rows that
      follow contain the data that is bound to each scenario that is evaluated.

      | string 1 | string 2 | result   |
      | howdy    | doo      | howdydoo |
      | any      | thing    | anything |

The Gwen interpreter will expand and evaluate each record in the body of the Examples table at the point where the scenario outline is declared in the feature. This example contains just one Examples clause in the outline but many can be specified.

Evaluated scenario outlines appear in HTML reports as follows.

Each Examples table record in the report (excluding the header) is rendered as a hyperlink that when hovered over and clicked will reveal the expanded scenario.

Outlines as StepDefs for Deferred Execution

For cases where you may want to reuse a scenario outline across many scenarios or features, you can make it a step definition in the meta and then call it by name in the feature where you want it to execute.

Meta file:

Feature: Join Strings Meta

  @StepDef
  Scenario: I join the two strings
      Given the result is "${string 1}${string 2}"

  @StepDef
  Scenario Outline: Joining <string 1> and <string 2> should yield <result>

    This outline is loaded into memory and execution is deferred until a call is made.
    Joining <string 1> and <string 2> should yield <result>

    Given string 1 is "<string 1>"
      And string 2 is "<string 2>"
     When I join the two strings
     Then the result should be "<result>"

    Examples: Basic string concatenation

      The header row contains the placeholder names. The body rows that
      follow contain the data that is bound to each scenario that is evaluated.

      | string 1 | string 2 | result   |
      | howdy    | doo      | howdydoo |
      | any      | thing    | anything |

Feature file:

Feature: Join Strings

  Scenario: Join different pairs of strings and compare their results
      Given Joining <string 1> and <string 2> should yield <result>

The Gwen interpreter will load the outline in the meta to memory and execute it when it is called by the scenario in the feature. Any number of scenarios in any features can call the outline in this way as long as the meta is in scope.

Calls to scenario outlines are rendered as steps in HTML reports (just like all calls to step definitions are).

When the step is hovered over and clicked, the called outline is revealed as a StepDef.

Each Examples table record in the report (excluding the header) is rendered as a hyperlink that when hovered over and clicked will reveal the expanded scenario.

Notes about outlines as StepDefs

  • They are reusable
  • They can be called from anywhere, including backgrounds and the REPL console too!
  • If the name of the outline contains data placeholders, then their bound table values are substituted into the expanded scenario name prefixes at evaluation time. Specifying placeholders in the name is handy in this way but it is also optional. We could have named the outline ‘joining two strings should yield a compound string‘ instead and called it by that name if we wanted to. In this case the expanded scenarios would all have this same name prefix. Either way, the expanded scenarios will always be given unique names that are suffixed with the example record number (see report snapshots above). If you are using placeholders in the name though, then your call to the step definition should also include the same placeholder name literals (as shown in this example) to prevent potential name clashes with any other step definitions you may have with a similar name. If a name clash does occur, Gwen will report it and you will need to rename.
Advertisements

Scenario Outlines Vs Parameterised StepDefs

Several users have enquired about scenario outlines for Gwen. So I recently created a pull request to add support for them but am a little hesitant to merge for the following reasons:

Consider the following example taken from the pull request above.

Feature: Join Strings

   Scenario Outline: Join two strings together

     Given string 1 is "<string 1>"
       And string 2 is "<string 2>"
      When I join the two strings
      Then the result should be "<result>"

     Examples:
     | string 1 | string 2 | result   |
     | howdy    | doo      | howdydoo |
     | any      | thing    | anything |

Parameterised StepDef Approach

The above can be reduced to a single Scenario containing two steps for each Example record:

Scenario: Join two strings together
    Given "howdy" joined with "doo" should yield "howdydoo"
      And "any" joined with "thing" should yield "anything"

These steps can be bound to a step definition in Meta as follows:

  @StepDef
  Scenario: "<string 1>" joined with "<string 2>" should yield "<result>"
      Given string 1 is "$<string 1>"
        And string 2 is "$<string 2>"
       When I join the two strings
       Then the result should be "$<result>"

With this approach:

  • We achieve the same effect with just two steps (one for each example record)
  • Each step is executable in the REPL console

    CSV Data Feed Approach

    The same can also be achieved by feeding a CSV file containing header and data rows to a feature.

    CSV file:

    string 1,string 2,result
    howdy,doo,howdydoo
    any,thing,anything
    

    Feature file:

    Scenario: I join two strings together
    
        Given string 1 is "${string 1}"
          And string 2 is "${string 2}"
         When I join the two strings
         Then the result should be "${result}"
    

    A Possible Compromise

    Alternatively we could add support for scenario outlines but mandate that they be confined to Meta as step definitions.

    For example, we could declare the outline to be a StepDef in Meta as follows:

       @StepDef
       Scenario Outline: I join two strings together
    
         Given string 1 is "<string 1>"
           And string 2 is "<string 2>"
          When I join the two strings
          Then the result should be "<result>"
    
         Examples:
           | string 1 | string 2 | result   |
           | howdy    | doo      | howdydoo |
           | any      | thing    | anything |
    

    And then we could call it in features as follows:

    Given I join two strings together
    

    But declaring rows of data in meta is also not ideal.

    Do we need Scenario Outlines?

    I’ve created a pull request to add support for scenario outlines in feature files but cannot support them in the REPL. I’ve shown here that we can achieve the same effect using parameterised StepDefs or CSV data feeds. I’ve also proposed a compromised alternative.

    I’d like to know your thoughts.

  • SQL Data Bindings for Gwen

    SQL Data Bindings have been introduced to Gwen v2.1.0 to enable easy access to data stored in databases. This is handy for cases where you need to fetch data from a database and do some work with that data. For example, checking that the value of a field in a database contains some expected value or assigning a value stored in (or derived from) a database to an attribute and doing something with it.

    The DSL for binding the result of an SQL query to an attribute is defined as follows:

    <attribute> <is|will be> defined by sql "<selectStmt>" in the <dbName> database
    

    Where:

    • attribute = the name of the attribute to bind the fetched data to
    • selectStmt = the SQL select statement used to fetch the data
    • dbName = a unique name assigned to the database connection

    DB Connection Setup

    Setting up a database connection is a two step once-off process.

    1. Configure the database connection by defining the following Gwen settings. Replace dbName  with the name you want to assign to your connection.
      • gwen.db.dbName.driver = the name of the JDBC driver class
      • gwen.db.dbName.url = the URL to the database containing all connection properties including credentials
    2. Point your GWEN_CLASSPATH environment variable to the location of the JDBC driver JAR. You could choose to set this in a wrapper script (bat or shell) or in the environment settings of your local machine.

    For more details, see SQL Data Bindings on our wiki.

    Any number of database connections can be configured. You just need to assign a unique dbName to each connection and configure the driver and url settings using that name and add any required JDBC driver JARs to your GWEN_CLASSPATH variable (multiple JAR entires can be delimited using the standard Java classpath separator for the platform you are on).

    Simple Usage

    Assuming you have a database named ‘feedback‘ configured as per the MySQL example here, you can then bind an SQL query that fetches an email address (for example) to an attributed named ‘my email‘ as shown below. Ideally you should define this binding in a Gwen Meta file.

    Given my email is defined by sql "select email from subscribers where name='gweninterpreter'" in the feedback database
    

    In this example:

    • attribute = my email
    • selectStmt = select email from subscribers where name=’gweninterpreter’
    • dbName = feedback

    With this binding in place, you can then reference the ‘my email‘ attribute anywhere in your features and Gwen will execute the SQL query to fetch the email value from the database and bind it in place.

    For example, if you perform an assertion as follows, Gwen will fetch and bind the ‘my email‘ value before doing the comparison.

    # compare email in database with expected literal
    Then my email should be "gweninterpreter@gmail.com"
    

    As another example, you can also enter the fetched email value into a field on a web page:

    # web page field binding (in Meta)
    Given the email field can be located by name "email"
    
    # enter email value fetched from database into field on web page
    When I enter my email in the email field
    

    Advanced Usage

    You can also use string interpolation to soft code the ‘name‘ parameter in the above query to another attribute called ‘my name‘ by redefining the SQL in your binding as follows instead:

    select email from subscribers where name='${my name}'
    

    Now whatever value is bound to the ‘my name‘ attribute in memory will be substituted for the ‘${my name}‘ placeholder in the SQL string before Gwen executes it. This value could be the captured value of a field on a web page, a property value, an assigned literal, or any other type of Gwen binding (another SQL binding even).

    Gwen 2 released

    Gwen 2 has been released with the latest technologies including Java 8 and Selenium WebDriver 3. All binary dependencies have also been updated and the project is now built using Scala 2.12.

    There is one impact to Firefox users but nothing else has changed. With the upgrade to Selenium 3, all browser vendors (including Mozilla) are responsible for developing and providing the native drivers for their own browsers. To use Firefox, you will now need to download a native Firefox driver and set the following in your gwen.properties file:

    webdriver.gecko.driver=/your-downloaded-driver-location/geckodriver

    The settings page on our Gwen wiki has also been updated.

    Gwen 2.x will only run on Java 8 environments. Gwen 1.x will still run on Java 7 and above as before, but will no longer have new features added to it.

    The latest Gwen release is available here. See also install instructions.

    Integrating Gwen with Maven

    Although you can download and install Gwen locally to run automated tests, it is sometimes useful to integrate it with a build tool such as Maven and have it download and install it for you and run tests as part of the release verification process. Integrating with Maven can also make it easier for you to run Gwen on a continuous integration build server too.

    Gwen does not ship with a Maven plugin, but you can still integrate it with Maven using a POM file like this:

    <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    
      <modelVersion>4.0.0</modelVersion>
      <groupId>org.gweninterpreter</groupId>
      <artifactId>gwen-web-maven</artifactId>
      <name>gwen-web-maven</name>
      <version>4.0.0</version>
      <packaging>pom</packaging>
    
      <profiles>
        <profile>
          <id>gwen</id>
          <activation>
            <property>
              <name>gwen.args</name>
            </property>
          </activation>
          <properties>
            <gwen.web.version>???</gwen.web.version>
            <!--selenium.version>???</selenium.version-->
            <!--selenium.dir>target/selenium-${selenium.version}</selenium.dir-->
            <gwen.dir>target/gwen-web-${gwen.web.version}</gwen.dir>
            <gwen.classpath>${selenium.dir}/*${path.separator}${gwen.dir}/lib/*</gwen.classpath>
          </properties>
          <dependencies>
            <!--dependency>
              <groupId>org.seleniumhq.selenium</groupId>
              <artifactId>selenium-java</artifactId>
              <version>${selenium.version}</version>
            </dependency-->
            <dependency>
              <groupId>org.gweninterpreter</groupId>
              <artifactId>gwen-web</artifactId>
              <version>${gwen.web.version}</version>
              <type>zip</type>
              <scope>provided</scope>
            </dependency>
          </dependencies>
          <build>
            <plugins>
              <plugin>
                <artifactId>maven-dependency-plugin</artifactId>
                <version>2.9</version>
                <executions>
                  <execution>
                    <id>install-gwen</id>
                    <phase>integration-test</phase>
                    <goals>
                      <goal>unpack-dependencies</goal>
                    </goals>
                    <configuration>
                      <includeGroupIds>org.gweninterpreter</includeGroupIds>
                      <includeArtifactIds>gwen-web</includeArtifactIds>
                      <includeTypes>zip</includeTypes>
                      <outputDirectory>target</outputDirectory>
                      <stripClassifier>true</stripClassifier>
                      <stripVersion>true</stripVersion>
                    </configuration>
                  </execution>
                  <!--execution>
                    <id>install-selenium</id>
                    <phase>integration-test</phase>
                    <goals>
                      <goal>copy-dependencies</goal>
                    </goals>
                    <configuration>
                      <excludeScope>provided</excludeScope>
                      <outputDirectory>${selenium.dir}</outputDirectory>
                    </configuration>
                  </execution-->
                </executions>
              </plugin>
              <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>exec-maven-plugin</artifactId>
                <version>1.3.2</version>
                <executions>
                  <execution>
                    <id>launch-gwen</id>
                    <phase>verify</phase>
                    <goals>
                      <goal>exec</goal>
                    </goals>
                  </execution>
                </executions>
                <configuration>
                  <executable>java</executable>
                  <commandlineArgs>-cp ${gwen.classpath} gwen.web.WebInterpreter -b ${gwen.args}</commandlineArgs>
                  <successCodes>
                    <successCode>0</successCode>
                  </successCodes>
                </configuration>
              </plugin>
            </plugins>
          </build>
        </profile>
      </profiles>
      
    </project>
    

    Be sure to change the Gwen version by updating the gwen.web.version property at line 20 with the latest version or another version you would like to use.

    With this POM and your Gwen settings in place you can have Maven download, install, and launch the latest version of Gwen with the following command (where <args> is a space separated list of arguments that you want to pass to Gwen):

    mvn verify -Dgwen.args="<args>"

    So if your executable tests are in a folder named ‘features’ relative to your POM file, then you can execute them with a Maven command like this:

    mvn verify -Dgwen.args="features"

    And to also generate html and junit style reports in the target/reports folder..

    mvn verify -Dgwen.args="-r target/reports -f html,junit features"

    Similarly, any arguments can be passed to Gwen in this manner through the gwen.args -D command line argument. Note that the -b batch mode switch is implicitly passed to Gwen and therefore you do not need to explicitly specify it as an argument. The gwen profile will only be activated if gwen.args is specified in the call to Maven.

    By default, Gwen will use the latest version of Selenium bundled in its distribution but you can change it if you need to by performing the following. You would normally not need to do this unless you are targeting a legacy or beta browser or need to run a specific version of selenium.

    • Uncomment lines 21, 22, 27 to 31, and 61 to 71.
    • Update the selenium.version property (at line 21) with the release number you would like to use

    No Page Objects – There’s no long way to go. We’re already there!

    Specifications driven web automation with no page objects or selenium coding ~ We’ve done it!

    Page Objects

    I recently discovered a Page Objects Refactored article which suggests that ‘flaky’ web tests are often misattributed to Selenium web driver issues and that the problem instead is poorly coded page objects that were not written with solid OO principles in mind. Programming languages allow programmers to achieve the same or similar results in more than one way. For this reason, design patterns were introduced to help developers implement known solutions to known problems. But even design patterns require discipline and cannot be strictly enforced to any useful degree without introducing a specialised framework. Frameworks also are geared at software developers and not software users.

    What is needed is a pre-programmed automation tool that frees users from development concerns.

    No Page Objects

    One of the primary goals of the gwen-web project is to give users a tool for automating web pages without needing to develop any page objects or compile any code. Gwen-web is an interpreter that accepts plain text Gherkin features as input and produces executing web driver instructions as output. All interactions with the web driver API happen in a precompiled and embedded web engine through a prescribed DSL. This web engine also handles many of the known gotchas and pitfalls that programmers typically face when working directly with the web driver. Users also have the flexibility to compose custom DSLs (their own step definitions) which are also expressed in Gherkin. With this approach, developing page objects and adopting design patterns is no longer required. Furthermore, programmer errors are eliminated and ‘flakiness’ is removed. The ability to compose specifications in plain Gherkin text is all that is necessary.

    An example

    Consider the following Gherkin feature:

     Feature: Google search
    
    Scenario: Perform a google search
        Given I do a google search for "gwen-web"
         When I click the first result
         Then the page title should contain "gwen-interpreter/gwen-web"
          And the current URL should be "https://github.com/gwen-interpreter/gwen-web"
    

    With Gwen, we can take a plain text feature like this one and execute it “as is”.

    If you install gwen-web, you will find the above feature file located at features\google\GoogleSearch.feature relative to your root install directory (the location where you unpacked the distribution). If you open a command prompt to this location, you can execute this feature as follows:

    On Windows:

    gwen -b features\google\GoogleSearch.feature

    On a Mac:

    ./gwen -b features/google/GoogleSearch.feature

    When this is launched, gwen-web will:

    • Open a new browser session
    • Navigate to the google home page
    • Submit a Google search
    • Click the first result that comes back
    • Verify that the title of the resulting page contains some expected text
    • Check the URL of the resulting page

    How does it work?

    Alongside the GoogleSearch.feature file, you will also find a Google.meta file. This file is also a Gherkin feature file, but with a .meta file extension. It defines the step definition for the first step in our feature. The remaining steps are all predefined in the web engine. We don’t need to define step definitions for those since the web engine already knows how to execute them.

    Here are the contents of the Google.meta file (minus the comments):

     Feature: Google search
    
    @StepDef
    Scenario: I do a google search for "<query>"
        Given I navigate to "http://www.google.com"
          And the search field can be located by name "q"
         When I enter "$<query>" in the search field
         Then the page title should start with "$<query>"
          And the first match can be located by class name "r"
          And the first result can be located by tag name "a" in the first match
    

    This meta defines an annotated scenario named: I do a google search for "<query>". This is how you define a step definition with Gwen. It’s simply just a @StepDef annotated scenario that in this case accepts a <query> string as a parameter followed by a sequence of steps that define its operations. When this meta file is processed by Gwen, the step definition is loaded and made available to any executing feature that is passed in. In our example, Gwen automatically discovers this meta file (because it is in the same directory as the feature itself) and preloads the step definition defined in it to memory. When the first step in our feature executes, Gwen matches it to the step definition by name and binds the passed in “gwen-web” string literal to the <query> parameter which is then used by the steps therein to perform the Google search.

    Comparing the feature step with the step definition reveals the match that Gwen is able to detect:

    I do a google search for "gwen-web"
    I do a google search for "<query>"
    

    Another example

    You could also express this Serenity example as a feature specification like this:

    features\todo\CompleteATodo.feature

       Feature: Complete a Todo
    
    Background: Open a new browser
          Given I start a browser for James
    
      Scenario: I should be able to complete a todo
          Given I browse to the application home page
           When I add a "Walk the dog" item
            And I add a "Put out the garbage" item
            And I complete the "Walk the dog" item
           Then the "Walk the dog" item should be completed
    
      Scenario: I should see the number of todos decrease when an item is completed
          Given I browse to the application home page
           When I add a "Walk the dog" item
            And I add a "Put out the garbage" item
            And I complete the "Put out the garbage" item
           Then the number of items left should be "1"
    

    And make it executable with a meta specification like this:

    features\todo\Todo.meta

     Feature: Todo meta
    
    @StepDef
    Scenario: I browse to the application home page
        Given I navigate to "http://todomvc.com/examples/angularjs/#/"
         Then the todo field can be located by id "new-todo"
          And the number of items left can be located by javascript "document.getElementById('todo-count').children[0]"
          And I locate the todo field
    
    @StepDef
    Scenario: I add a "<todo>" item
        Given I enter "$<todo>" in the todo field
          And the "$<todo>" item can be located by xpath "//*[@class='view' and contains(.,'$<todo>')]//label"
          And the "$<todo>" item checkbox can be located by xpath "//*[@class='view' and contains(.,'$<todo>')]//input[@type='checkbox']"
    
    @StepDef
    Scenario: I complete the "<todo>" item
        Given I click the "$<todo>" item checkbox
    
    @StepDef
    Scenario: the "<todo>" item should be completed
        Given the "$<todo>" item checkbox should be checked
    

    And then launch it like this (assuming you have created the above two files):

    Windows:

    gwen -b features\todo\CompleteATodo.feature

    Mac:

    ./gwen -b features/todo/CompleteATodo.feature

    Composable automation

    With Gwen we don’t have to do any heavy development work. We only need to compose specifications.

    Notice that all the technical automation glue (JavaScript expressions, XPath locators etc..) are confined to meta. The features themselves are clean, readable, self documenting, and easily understood. You could easily have a BA or tester write the feature files upfront and then have a developer plug in some meta behind it. Or you could have tech savvy testers with enough HTML and JavaScript skills to write the meta themselves. Teams can choose to be as declarative or as imperative as they like in how they write their features. Tailored meta can always be crafted to suit.

    So there you have it ~ web automation with #NoPageObjects and no selenium coding!

    If you would like to see a more detailed example, then see our Automation by Meta blog post. See also our Wiki and FAQ pages for more information about Gwen.