Locating Web Elements with Gwen

Gwen supports locating elements on web pages using all the standard Selenium selectors and additionally by JavaScript too. In this post, we are going to explore them. To make it a little more fun we will use the Gwen REPL which will enable us to verify our locator expressions interactively against a running web application in a browser as we go.

The first thing you’ll need to do is install Gwen somewhere on your machine. It’s simple to do. Just download this gwen-workspace.zip and unpack it somewhere on your drive and open a command prompt into the unpacked location. If you run into any trouble or would prefer to follow our install guide, you can start here and then come back.

Once you’re ready, type the following command at the terminal in the root gwen-workspace directory where you installed Gwen:

Linux/Mac/PowerShell:

./gwen

Windows DOS:

gwen

The first time you launch Gwen it will take just a little time to download the latest version of itself. Subsequent launches won’t have to do that and will start faster.

The REPL will start with a gwen> prompt open:

   __ ___      _____ _ __     _
  / _` \ \ /\ / / _ \ '_ \   { \,"
 | (_| |\ V  V /  __/ | | | {_`/
  \__, | \_/\_/ \___|_| |_|   `
  |___/

Welcome to gwen-web v2.35.1

INFO - Environment context initialised
INFO - gwen-web.sh -r target/reports -p gwen.properties,browsers/chrome.properties,env/local.properties

REPL Console

Enter steps to evaluate or type exit to quit..

gwen> 

In the examples that follow we are borrowing the Selenium Test Page from automationtesting.com to demonstrate locating web elements using all of the available mechanisms in Gwen.

Keep in mind that Gwen is case sensitive. So you will need to use the same casing that is shown in all the examples here.

Enter the following step into the REPL to have Gwen open up a new browser session with the Selenium Test Page all ready and loaded (you can copy and paste the step if you like):

When I navigate to "https://automationintesting.com/selenium/testpage/"

Locating elements with Selenium selectors

We’ll start with ID locators which simply locate elements by their HTML id attribute. The Selenium Test page that we have open in the browser has a First name field in it. If we right click and inspect that field in the browser we will see that it is defined in HTML as follows:

<input id="firstname" type="text" placeholder="Enter your first name">

Enter the following step into Gwen REPL prompt to define a locator named firstNameById for finding this field:

Then firstNameById can be located by id "firstname"

This will instruct Gwen to bind the name firstNameById to the By.id("firstname") Selenium locator under the hood. We can test that it works by entering the following at the REPL prompt:

When I locate firstNameById

Gwen will find and highlight the field in yellow inside the browser session that we opened. If you missed it, rearrange your REPL and browser windows a bit so you can see both side by side and repeat the last command (you can use the up arrow in the REPL to recall the last command).

Demo of locating a web element with Gwen on FriendlyTester‘s Selenium Test Page

Now that we have proven that our firstNameById locator binding works, we can perform an operation on the field by referencing it by name in any one of the predefined Web DSL steps that Gwen supports for web automation. For example, we can type a name into the form by entering the following step into the REPL:

And I type "Gwen" in firstNameById

Gwen will locate and enter the name in the field.

Similarly, we can also locate the form in the page which is defined in the page HTML as:

<form id="contactus" name="contactform">

Let’s locate by name this time. Enter the following step into Gwen REPL prompt to define a locator named formByName for finding this form:

Given formByName can be located by name "contactform"

This will instruct Gwen to bind the name formByName to the By.name("contactform") Selenium locator . We can test it by entering the following at the REPL prompt:

When I locate formByName

Gwen will find and highlight the form in inside the browser. We can then submit the form by entering the following step into the REPL:

And I submit formByName

Gwen will locate and submit the form. Let’s exit the REPL and see how we can use the locators we just defined to execute a Gherkin feature file. Type exit at the gwen> prompt to return to your OS.

exit

Create a subfolder named se-test inside the features subfolder of your Gwen workspace. Then create and save a new text file named SeleniumTestPage.feature in that subfolder with the following content:

 Feature: Selenium Test Page

Scenario: Submit the test form
    Given I am on the Selenium test page
     When I provide some contact details
      And I submit the test form
     Then nothing should happen

Create and save another text file named SeleniumTestPage.meta in the same subfolder with the following content:

 Feature: Selenium Test Page Meta

@StepDef
Scenario: I am on the Selenium test page
     When I navigate to "https://automationintesting.com/selenium/testpage/"
     Then firstNameById can be located by id "firstname"
      And formByName can be located by name "contactform"

@StepDef
Scenario: I provide some contact details
     When I type "Gwen" in firstNameById

@StepDef
Scenario: I submit the test form
     When I submit formByName

@StepDef
Scenario: nothing should happen
     Then the current URL should end with "?"

Type the following command at your OS terminal prompt to have Gwen load the meta file and bind it to the feature file to automate execution:

Linux/Mac/PowerShell:

./gwen features/se-test

Windows DOS:

gwen features/se-test

Gwen will execute the feature by performing all the same actions we performed earlier in the REPL and will leave the REPL session open when done. The locators we defined are now bound in Gwen’s memory and we can continue experimenting with them in the REPL if we wish. Enter the following command to have Gwen submit the form again:

When I submit formByName

Enter the 3rd step in our feature directly into the REPL:

And I submit the test form

Gwen will match it to the 3rd @StepDef annotated Scenario in the meta file by name (this is how step definitions work in Gwen) and execute all the steps that we defined there. Type exit at the Gwen REPL prompt when you are ready to return to your OS.

We just demonstrated how you can locate an element by id with Gwen. You can use the same approach to locate elements using any one of the other standard Selenium locator mechanisms. Borrowing some locator examples from Friendly Tester’s Free Selenium Course, here is how you would define the equivalents in Gwen:

Given genderByClass can be located by class name "gender"

  And firstNameByXPath can be located by xpath "//*[@id="firstname"]"

  And linkByTag can be located by tag name "a"

  And linkByCSS can be located by css selector "div a"

  And linkByLinkText can be located by link text "Our Policy"

  And linkByPartialText can be located by partial link text "Policy"

If you load these locators into the Gwen REPL or define them in your meta file, you will then be a able to perform operations on those elements such as:

When I select "Female" in genderByClass

And I clear firstNameByXPath

And I click linkByTag

Locating elements with JavaScript

We said at the beginning that Gwen can also locate elements by JavaScript. Sometimes this can help us locate dynamically rendered elements more reliably.

If you wanted to, you could define the form locator using a vanilla JavaScript expression like this :

Given formByJS can be located by javascript "document.getElementsByName('contactform')[0]"

Or a form locator using a JQuery expression (if the jquery library is loaded in the browser page) like this:

And formByJQuery can be located by javascript "$('#contactus').get(0)"

Or in extreme cases where the standard locators do not suffice and you have to resort to executing a custom script on a page to find an element, you could do so with an anonymous JavaScript function in a DocString like this:

And elemByJSFunc can be located by javascript
      """
      (function() {
        var elem = ..
          // dynamic logic to find element 
          // based on some fancy rules,
          // conditions or what have you
        return elem;
      })();
      """

To test this last one in the REPL, you would have to use the paste mode since the step spans multiple lines.

That’s it! We hope this has been a good small introduction to locators in Gwen for you.

Advertisements

Running Gwen on BrowserStack

BrowserStack allows you to run automated Selenium tests on a variety of platforms, browsers, and devices on the cloud. Since Gwen is built on Selenium and also supports remote web driver, you can easily run your Gwen tests “as is” on BrowserStack too. As of gwen-web version 2.25.1, all you need to do is provide some runtime properties.

Remote URL

If you sign up for a BrowserStack account, you will receive a user name and access key. Add the following settings to your gwen.properties file to configure your connection to BrowserStack:

file: ~/gwen.properties

bs.user = your-browserstack-username
bs.key = your-browsetstack-access-key
gwen.web.remote.url = https://${bs.user}:${bs.key}@hub-cloud.browserstack.com/wd/hub

Target Capabilities

BrowserStack provides a neat capabilities page that can generate capabilities settings for you. Simply navigate to that page and select your desired target platform and browser. The capabilities settings will be generated for you and displayed as shown below:

This example shows the capabilities settings for a Chrome browser on the Mac OS X High Sierra Platform. To configure Gwen to use these settings, create a new properties file containing each of the settings above as follows:

file: chrome-mac.properties

gwen.web.capability.os = OS X
gwen.web.capability.os_version = High Sierra
gwen.web.capability.browser = Chrome
gwen.web.capability.browser_version = 67.0
gwen.web.capability.browserstack.local = false
gwen.web.capability.browserstack.selenium_version = 3.11.0

You can save this file anywhere you like. For example, you could save it in your current working directory as chrome-mac.properties.

You can create a separate properties file in the same manner for each other platform, browser, or device you wish to target.

Execution

Now to run your Gwen features on BrowserStack, simply invoke Gwen passing in the above properties file (through the -p option) as follows. This example will run all features in the features/google folder on BrowserStack targeting the Chrome browser on the Mac.

./gwen -b -p chrome-mac.properties features/google

To target a different platform or browser, just pass in the properties file containing those capabilities instead.

If you logon to browserstack.com, you will see the results of all the remotely executed web driver steps and a video recording of the feature execution too.

That’s it!

Gwen Workspaces

Setting up Gwen for teams just got a whole lot easier!

Manually setting up and installing Gwen on multiple machines or build servers in a team environment can be tedious and can also result in inconsistent configurations across workstations. One of the reasons why we created gwen-gpm was to provide consistent installation across machines and platforms. But a team needs more than that. A team needs a consistent and seamless way of getting Gwen configured and running on any user workstation or build server too.

Introducing Gwen workspaces

Gwen workspaces solve this problem by defining a standard project structure on the file system complete with settings files and wrapper scripts that can easily be committed to Git and checked out on any machine. Any team member can then just checkout the workspace and start using Gwen straight away. Gwen will be automatically downloaded and installed on any user workstation or build server is not already present (through an embedded Gwen package manager in the workspace).

It is assumed that the target browser is already installed on the system. If not, you will need to manually install it.

The structure of this workspace is defined as follows:

gwen-workspace
|--/env : Put environment properties here
|--/features : Put your Gherkin feature and common meta files here
| gwen : Gwen launcher/wrapper script for linux
| gwen.bat : Gwen launcher/wrapper script for windows
| gwen.properties : Common Gwen properties
| log4j.properties : Gwen log settings
| gwen-gpm.jar : Gwen package manager
| .gitignore : Git ignore file

Create and Commit a Workspace to Git

To create a Gwen workspace for your team, perform the following (only one person in the team needs to do this):

  • Download and extract the workspace.zip to a location on your computer
  • Tweak or add team wide settings to the gwen.properties file in the workspace root. You can also tailor the wrapper scripts if necessary (for example, if you want to change the default Gwen launch options or add some new ones).
  • Verify that it all works by launching gwen (on windows) or ./gwen (on linux). Type exit when done to quit the REPL session.
  • Commit and push the workspace to your remote Git repository (see online Git help if you are not sure how to do this)

Checkout Workspace from Git and Go..

To use the workspace on any machine, perform the following (all team members need to do this):

  • Ensure that the target browser is installed on the system
  • Checkout the workspace from Git
  • Open a command prompt to the root workspace location
  • Type..
    • gwen (on windows) or ./gwen (on linux) followed by the options you require
    • Gwen will execute using the settings in the workspace
      Note: this same command can be used on any build server that checks out the workspace too
    • Gwen and native web drivers will self install on the first call
    • Type exit when done to quit the REPL session (if you started Gwen in REPL mode).

The team can now manage all their Gwen settings, environment configurations, Gherkin feature files, and Gwen meta files in a single workspace that can easily be pulled down and executed on any machine.

Help

For help, open a command prompt in the workspace root and type:

  • gwen help (on windows platforms)
  • ./gwen help (on linux platforms)

Each folder in the workspace also includes a README.txt file that can help to guide you.

Gwen and Selenoid reduced test execution from 50 minutes to 5

Thats 1000% (10 times) quicker than executing features in sequence!!

Gwen has always had the capability to make use of selenium grid which is particularly useful when running in parallel, and after discovering selenoid by the guys over at Aerokube I had to give it a shot.

Installing selenoid was a breeze with the configuration manager docker setup.  Once installed, selenoid downloads the last two driver versions of chrome, firefox and opera, and starts a grid ready to run tests.

Gwen comes with 235 features out of the box, ready for anyone to try.  These features cover a number of different scenarios including a flood.io challenge, navigating etsy, completing a todo, navigating blogs, and searching in google.    The features are also written using both imperative and declarative styles declarative-vs-imperative-gherkin.

Typically running 235 features takes 49 minutes and 33 seconds when running the features sequentially, however running these in parallel with selenoid took only 4 minutes and 41 seconds.  Both samples were run using a 34 core machine.

 

Gwen running 235 features sequentially
 

Gwen running 235 features in parallel
 

I have also included a screenshot of the running docker images, taken at a point in time where parallel was running full swing.

Thats all for now, hope you enjoyed a brief look into Selenoid and Gwen.

Evaluating Gwen with Modern JavaScript Web Apps

Gwen-web was designed to take the development pain out of web automation and make it easier to achieve with all types of web applications regardless of the underlying technologies or frameworks they were built with. Our goal was to make Gwen work consistently with web pages built on any kind of server or client side framework. All web pages are just HTML documents when they hit the browser after all. But this HTML is not always static and with the recent trend of modern JavaScript framework adoption, web pages are getting more and more dynamic. A lot more things happen in web pages nowadays. Lots of JavaScript code gets loaded and all types of browser events trigger functions and Ajax requests at various times resulting in all sorts of dynamic and asynchronous rendering that makes for a very rich user experience. This might make things more pleasant for human users but it is often a hard challenge for automation programs (or robots).

The Evaluation Test

So I thought I’d try Gwen out on some web pages built on the popular and modern JS frameworks of today to see how well it performs. To do this, I wrote a feature suite that mimics these Serenity tests and ran it over the various JS implementations of the publicly available TodoMVC web application (this is a project which offers the same Todo web application implemented on different JavaScript frameworks).

It is important to note that Gwen uses the Java implementation of the Selenium web driver and is not a JS framework itself but rather a Java executable that reads plain text Gherkin features and dynamically interprets them into automated web page interactions. The evaluation performed here provides us with an indication of how well Gwen interacts with web applications built on different types of JS frameworks.

The Evaluation Results

The suite consists of 26 scenarios which I ran over 33 different JS implementations of the Todo app (for a total of 858 scenarios). I ran it in a Chrome browser on a Mac. It took about 50 minutes to run every scenario in sequence. I also ran the tests in parallel (on a dual CPU quad core machine) which took about half the time and produced consistent results. Gwen was executed with the gwen.feature.failfast setting overriden to false to force all scenarios in a feature to execute even if one or more prior ones fail.

94% of Scenarios Passed :)

All tests completed successfully and passed for the following frameworks.

  • JavaScript:
    • Backbone.js
    • AngularJS
    • Ember.js
    • KnockoutJS
    • Dojo
    • Knockback.js
    • CanJS
    • Polymer
    • React
    • Mithril
    • Ampersand
    • Vue.js
    • Marionette.js
    • Vanilla JS
    • Vanilla ES6
    • jQuery
  • Compile-to-JS:
    • Spine
    • Dart
    • GWT
    • TypeScript + Backbone.js
    • TypeScript + AngularJS
    • TypeScript + React
    • Serenade.js
    • Reagent
    • Scala.js + React
    • Scala.js + Binding.scala
    • js_of_ocaml
    • Humble + GopherJS

6% of Scenarios Failed

Some tests failed for the following frameworks. I have not investigated these in detail but have included my initial findings below.

  • JavaScript:
    • Flight
      • 15 scenarios failed, 11 passed
      • Intermittent element hits and misses
      • Entered data leaks or is lost
    • TroopJS + RequireJS
      • 15 scenarios failed, 11 passed
      • Unresponsive to enter key being sent to field
  • Compile-to-JS:
    • Closure
      • 5 scenarios failed, 21 passed
      • Unresponsive to checkbox being clicked
    • Elm
      • 8 scenarios failed, 18 passed
      • Intermittently not sending some characters to field
    • AngularDart
      • 3 scenarios failed, 23 passed
      • Intermittently not rendering conditionally visible buttons

Conclusion


Gwen interacted successfully with a large majority of the popular JS implementations. All tests passed for 28 of the 33 implementations evaluated. More than 94% of all scenarios passed in total.

Try it Yourself

All the todoMVC features used in this evaluation are also bundled in the gwen-web distribution as of release 2.3.3. If you install the latest gwen-web distribution, you can run it by calling the following command in the directory where you install Gwen (to run in parallel mode, just add the --parallel switch):

  • Windows:
    • gwen features/todoMVC
  • Linux:
    • ./gwen features/todoMVC

Gwen 2 released

Gwen 2 has been released with the latest technologies including Java 8 and Selenium WebDriver 3. All binary dependencies have also been updated and the project is now built using Scala 2.12.

There is one impact to Firefox users but nothing else has changed. With the upgrade to Selenium 3, all browser vendors (including Mozilla) are responsible for developing and providing the native drivers for their own browsers. To use Firefox, you will now need to download a native Firefox driver and set the following in your gwen.properties file:

webdriver.gecko.driver=/your-downloaded-driver-location/geckodriver

The settings page on our Gwen wiki has also been updated.

Gwen 2.x will only run on Java 8 environments. Gwen 1.x will still run on Java 7 and above as before, but will no longer have new features added to it.

The latest Gwen release is available here. See also install instructions.

Configurable User Properties in gwen-web

One of the really cool features of gwen-web is the ability to load in different parameters at run time.  This allows you to run gwen-web against multiple environments without changing either the feature or the meta files.

Lets take one of the examples with gwen-web and flood.io   Below are two examples of how one could set the value.  The first example sets “the how old are you dropdown” to 21, the second example uses a property, in this case different files for two different environments to set the value.

Lets define how the dropdown can be found in the meta first.

FloodIO.meta

And the heading should be “Step 2”
And the how old are you dropdown can be located by id “challenger_age”

FloodIO.feature

When I select “21” in the how old are you dropdown

Now with the properties approach. Lets create two files, DEV.properties and SIT.properties. The contents of the two files are below:

DEV.properties
user.age=44

SIT.properties
user.age=21

In addition we need to make a small modification to both the feature and the meta files

FloodIO.meta

And the heading should be “Step 2”
And my age is defined by property “user.age”
And the how old are you dropdown can be located by id “challenger_age”

Now replace the strikethrough line with the new line in the feature

FloodIO.feature

When I select “21” in the how old are you dropdown
When I select my age in the how old are you dropdown

Editting is complete. To run gwen-web with the environment specific age, simply type “gwen-web features/floodio/ -p dev.properties -b -r devReport”, or alternatively you can run “gwen-web features/floodio/ -p sit.properties -b -r sitReport”.