Testing JavaScript applications with Cucumber and Selenium WebDriver is very common when the API is written in Rails. After all, you write it in Ruby and have direct access to the database, stubs on global classes and other goodies. But you might want to stop and think about that decision. Over time we’ve been using Selenium WebDriver. As a result, stale element exceptions were making our test suites very unpredictable. But you’ve probably been there, haven’t you? Re-running specs and praying for the random failures to magically align and succeed all at once ;-)
Enter WebdriverIO. WebDriverIO actually has a solution for stale elements. Since v4, the team behind it introduced internal handling of stale elements with a retry mechanism. There’s lot more on that topic in their guides. In this blog post, we’re going to explore how to move all your Cucumber specs to be compatible with WebdriverIO. In order to reuse your workflow, you will have to move your specs to JavaScript. A slight frown on your face tells me that you’re not very happy about this ;-)
But hold on for a moment and let’s try this out by exploring our actual setup at INTUO.
Creating directory structure
First, let’s create a directory structure. Nothing new here – it’s a completely identical structure as any other cucumber directory structure in a Rails or Ruby project.
...
features/
# Directory for your step definitions
step_definitions/
example_steps.js
authentication_steps.js
# Support directory containing helpers and Before and After hooks
support/
# Your feature files
example.feature
authentication.feature
package.json
# Setup scripts and such ...
..
Mysterious note “Setup scripts and such …” is required in our setup, since we are running multiple applications written in different languages communicating with each other as microservices. Therefore, our acceptance testing suite is also an independent application, which offers a lot more flexibility and decouples dependencies. More on that later.
Porting Cucumber to JavaScript
Porting your cucumber features into JavaScript is pretty easy. Just copy them. To actually run them, you need to install cucumber.js. It’s a JavaScript implementation for running cucumber specs on Node.js.
Let’s install it.
~ $ npm install cucumber --save-dev
It’s a good idea to define default settings for Cucumber.js CLI. You can do it by creating your cucumber.js
file in top-level directory structure. Basically the place where your package.json
file and node_modules
directory live.
// cucumber.js
common = '--strict --require features --format pretty --tags ~@skip';
module.exports = {
build: common + ' --format progress',
'default': common,
'es5': '--tags ~@es6'
};
And now you can run your cukes with Cucumber.js CLI as node_modules/.bin/cucumber.js
. Feel free to explore more options in the Cucumber.js CLI Reference.
Running Selenium Standalone
In order to test with WebDriverIO, you need to install and run Selenium Standalone. There are two ways. Download Selenium Standalone and run it as
~ $ java -jar selenium-standalone-x-y-z.jar
Note that you should have Firefox installed for this option. Or if you’re a hacker, just explore Selenium Standalone options and set your preferred browser.
But using a Selenium Standalone Docker image with pre-installed Chrome is very much preferred.
~ $ docker run -p 127.0.0.1:4444:4444 selenium/standalone-chrome:latest
And here it comes – by using the Docker image, you can run Selenium Standalone with an actual Chrome instance in your CI. Pretty neat, huh?
Setting up WebDriverIO
You have two options to set up WebDriverIO. By using their runner wdio
or standalone. We’re going to explore the standalone option in this guide, but feel free to try out wdio
as it might suit your needs more.
Let’s configure WebDriverIO in support/webdriverio.js
.
// support/webdriverio.js
"use strict";
let WebDriverIO = require('webdriverio');
let browser = WebDriverIO.remote({
baseUrl: 'https://my-awesome-app.com', // Or other url, e.g. localhost:3000
host: 'localhost', // Or any other IP for Selenium Standalone
port: 4444,
waitforTimeout: 120 * 1000,
logLevel: 'silent',
screenshotPath: `${__dirname}/../../screenshots/`,
desiredCapabilities: {
browserName: process.env.SELENIUM_BROWSER || 'chrome',
},
});
global.browser = browser;
module.exports = function() {
this.registerHandler('BeforeFeatures', function(event, done) {
browser.init().call(done);
});
this.registerHandler('AfterFeatures', function(event, done) {
browser.end().call(done);
});
};
And that’s it. Standalone WebDriverIO is configured. If you’re interested in more configuration options, explore them at WebDriverIO Developer Guides.
Setting up Chai.js as assertion library
Let’s install it!
~ $ npm install chai --save-dev
Define Chai.js assertion helpers globally.
// support/chai.js
"use strict";
let chai = require('chai');
global.expect = chai.expect;
global.assert = chai.assert;
And that’s it.
Writing Cucumber.js steps
Porting your examples might be a bit tricky. You have to rewrite them to JavaScript. The structure for writing the steps is the same, only the language and test helpers changed. As an example, take a simple authentication feature with Given
, When
and Then
statements.
# features/authentication.feature
Feature: Authentication
Scenario: User logs in with correct credentials
Given I open up the application
When I fill in login as "smolnar" and password as "password123"
Then I should be logged in as "smolnar"
For the above feature, the step definitions look as follows:
// features/step_definitions/authentication_steps.js
module.exports = function() {
this.Given('I open up the application', function(done) {
browser
.url('/')
.call(done);
});
// Note that this is a shorthand for regular expression
// as /^I fill in login as "([^"]*)" and password as "([^"]*)"$/.
// So don't worry, you don't have to rewrite your step matchers to strings ;-)
this.When('I fill in login as "$string" and password as "$string"', function(login, password, done) {
browser
.waitForExist('#login')
.setValue('#login', login)
.setValue('#password', password)
.click('#login-button')
.call(done)
});
this.Then('I should be logged in as "$string"', function(login, done) {
browser
.waitForExist('#logged-in-user')
.getText('#logged-in-user').then(function(text) {
expect(login).to.eql(text);
})
.call(done);
});
};
Full documentation for the browser API is available at the official WebDriverIO documentation. The API probably seems a bit strange to you, since it’s very different and lot more tedious if you’re used to Capybara. It’s just a different approach to testing. It makes heavy use of the promise API and follows the JSON Wire Protocol by Selenium. But that’s just a question of wrapping it and making it more friendly. The most important advantage of this approach is that it’s very stable when testing single page applications (such as Angular, React or Ember.js), making your acceptance test suite a lot more reliable.
Run this scenario as
node_modules/.bin/cucumber.js features/authentication.feature
And that’s it. Now you have Cucumber.js, WebDriverIO and Chai all set up.
Acceptance Test Suite with Continuous Integration
And lastly, let’s explore an example how it all stacks up.
As advised earlier, our acceptance test suite at INTUO is an independent repository with a build scripts and other goodies to build the entire application from a bunch of small microservices. It’s a different approach than a monolithic structure with a main application, like e.g. Rails. Having multiple small applications co-operating with each other helps us to scale the infrastructure and decouple dependencies.
As a first step of our CI setup, automated scripts fetch our microservices from their respective repositories and build them. To start up the entire infrastructure on CI, we fire up a tmux session. It’s possible to start everything in the background and omit tmux completely, but this approach turned out to be very helpful when debugging errors directly on CI. And to set up the database with testing data, we use special data seed that we keep up to date as we develop new features.
Our browser setup in CI is backed by a Selenium Standalone Docker image with Chrome as a browser. This way we are running our tests in a real browser, so the expected behaviour is consistent and we don’t run into any problems with missing browser API calls or strange behaviour with some headless browsers, e.g. Phantom.js. This approach is also extensible for other browsers and can provide a very good strategy for making sure that your application works on any browser you support.
That’s our acceptance test suite in a nutshell. We hope this approach helps you to improve your acceptance testing workflow as well.
Caveats
But hold on just for a bit …
WebDriverIO has a couple of caveats we found while working with it for last couple weeks. One of them is waitForExist
. If you’re using waitForExist
, you should know it actually checks only if the element is present in DOM. You might want to use waitForVisible
in conjunction with waitForExist
, since by clicking on the element, your test will fail if something is overlapping the element – waitForVisible
will correctly wait for the temporary overlapping element to disappear.
Let us know in the comments section below what you think about this workflow. We’re looking forward to hear your comments and opinions!