Testing JavaScript applications with Cucumber and Selenium WebDriver is very common when the API is written in Rails. After all, you write it in Ruby and have direct access to the database, stubs on global classes and other goodies. But you might want to stop and think about that decision. Over time we’ve been using Selenium WebDriver. As a result, stale element exceptions were making our test suites very unpredictable. But you’ve probably been there, haven’t you? Re-running specs and praying for the random failures to magically align and succeed all at once ;-)

Enter WebdriverIO. WebDriverIO actually has a solution for stale elements. Since v4, the team behind it introduced internal handling of stale elements with a retry mechanism. There’s lot more on that topic in their guides. In this blog post, we’re going to explore how to move all your Cucumber specs to be compatible with WebdriverIO. In order to reuse your workflow, you will have to move your specs to JavaScript. A slight frown on your face tells me that you’re not very happy about this ;-)

But hold on for a moment and let’s try this out by exploring our actual setup at INTUO.

Creating directory structure

First, let’s create a directory structure. Nothing new here – it’s a completely identical structure as any other cucumber directory structure in a Rails or Ruby project.

...

features/
  # Directory for your step definitions
  step_definitions/
    example_steps.js
    authentication_steps.js

  # Support directory containing helpers and Before and After hooks
  support/

  # Your feature files
  example.feature
  authentication.feature

package.json

# Setup scripts and such ...
..

Mysterious note “Setup scripts and such …” is required in our setup, since we are running multiple applications written in different languages communicating with each other as microservices. Therefore, our acceptance testing suite is also an independent application, which offers a lot more flexibility and decouples dependencies. More on that later.

Porting Cucumber to JavaScript

Porting your cucumber features into JavaScript is pretty easy. Just copy them. To actually run them, you need to install cucumber.js. It’s a JavaScript implementation for running cucumber specs on Node.js.

Let’s install it.

~ $ npm install cucumber --save-dev

It’s a good idea to define default settings for Cucumber.js CLI. You can do it by creating your cucumber.js file in top-level directory structure. Basically the place where your package.json file and node_modules directory live.

// cucumber.js

common = '--strict --require features --format pretty --tags ~@skip';

module.exports = {
  build: common + ' --format progress',
  'default': common,
  'es5': '--tags ~@es6'
};

And now you can run your cukes with Cucumber.js CLI as node_modules/.bin/cucumber.js. Feel free to explore more options in the Cucumber.js CLI Reference.

Running Selenium Standalone

In order to test with WebDriverIO, you need to install and run Selenium Standalone. There are two ways. Download Selenium Standalone and run it as

~ $ java -jar selenium-standalone-x-y-z.jar

Note that you should have Firefox installed for this option. Or if you’re a hacker, just explore Selenium Standalone options and set your preferred browser.

But using a Selenium Standalone Docker image with pre-installed Chrome is very much preferred.

~ $ docker run -p 127.0.0.1:4444:4444 selenium/standalone-chrome:latest

And here it comes – by using the Docker image, you can run Selenium Standalone with an actual Chrome instance in your CI. Pretty neat, huh?

Setting up WebDriverIO

You have two options to set up WebDriverIO. By using their runner wdio or standalone. We’re going to explore the standalone option in this guide, but feel free to try out wdio as it might suit your needs more.

Let’s configure WebDriverIO in support/webdriverio.js.

// support/webdriverio.js

"use strict";

let WebDriverIO = require('webdriverio');
let browser = WebDriverIO.remote({
  baseUrl: 'https://my-awesome-app.com', // Or other url, e.g. localhost:3000
  host: 'localhost', // Or any other IP for Selenium Standalone
  port: 4444,
  waitforTimeout: 120 * 1000,
  logLevel: 'silent',
  screenshotPath: `${__dirname}/../../screenshots/`,
  desiredCapabilities: {
    browserName: process.env.SELENIUM_BROWSER || 'chrome',
  },
});

global.browser = browser;

module.exports = function() {
  this.registerHandler('BeforeFeatures', function(event, done) {
    browser.init().call(done);
  });

  this.registerHandler('AfterFeatures', function(event, done) {
    browser.end().call(done);
  });
};

And that’s it. Standalone WebDriverIO is configured. If you’re interested in more configuration options, explore them at WebDriverIO Developer Guides.

Setting up Chai.js as assertion library

Let’s install it!

~ $ npm install chai --save-dev

Define Chai.js assertion helpers globally.

// support/chai.js

"use strict";

let chai = require('chai');

global.expect = chai.expect;
global.assert = chai.assert;

And that’s it.

Writing Cucumber.js steps

Porting your examples might be a bit tricky. You have to rewrite them to JavaScript. The structure for writing the steps is the same, only the language and test helpers changed. As an example, take a simple authentication feature with Given, When and Then statements.

# features/authentication.feature

Feature: Authentication
  Scenario: User logs in with correct credentials
    Given I open up the application
    When I fill in login as "smolnar" and password as "password123"
    Then I should be logged in as "smolnar"

For the above feature, the step definitions look as follows:

// features/step_definitions/authentication_steps.js

module.exports = function() {
  this.Given('I open up the application', function(done) {
    browser
      .url('/')
      .call(done);
  });

  // Note that this is a shorthand for regular expression
  // as /^I fill in login as "([^"]*)" and password as "([^"]*)"$/.
  // So don't worry, you don't have to rewrite your step matchers to strings ;-)
  this.When('I fill in login as "$string" and password as "$string"', function(login, password, done) {
    browser
      .waitForExist('#login')
      .setValue('#login', login)
      .setValue('#password', password)
      .click('#login-button')
      .call(done)
  });

  this.Then('I should be logged in as "$string"', function(login, done) {
    browser
      .waitForExist('#logged-in-user')
      .getText('#logged-in-user').then(function(text) {
        expect(login).to.eql(text);
      })
      .call(done);
  });
};

Full documentation for the browser API is available at the official WebDriverIO documentation. The API probably seems a bit strange to you, since it’s very different and lot more tedious if you’re used to Capybara. It’s just a different approach to testing. It makes heavy use of the promise API and follows the JSON Wire Protocol by Selenium. But that’s just a question of wrapping it and making it more friendly. The most important advantage of this approach is that it’s very stable when testing single page applications (such as Angular, React or Ember.js), making your acceptance test suite a lot more reliable.

Run this scenario as

node_modules/.bin/cucumber.js features/authentication.feature

And that’s it. Now you have Cucumber.js, WebDriverIO and Chai all set up.

Acceptance Test Suite with Continuous Integration

And lastly, let’s explore an example how it all stacks up.

As advised earlier, our acceptance test suite at INTUO is an independent repository with a build scripts and other goodies to build the entire application from a bunch of small microservices. It’s a different approach than a monolithic structure with a main application, like e.g. Rails. Having multiple small applications co-operating with each other helps us to scale the infrastructure and decouple dependencies.

As a first step of our CI setup, automated scripts fetch our microservices from their respective repositories and build them. To start up the entire infrastructure on CI, we fire up a tmux session. It’s possible to start everything in the background and omit tmux completely, but this approach turned out to be very helpful when debugging errors directly on CI. And to set up the database with testing data, we use special data seed that we keep up to date as we develop new features.

Our browser setup in CI is backed by a Selenium Standalone Docker image with Chrome as a browser. This way we are running our tests in a real browser, so the expected behaviour is consistent and we don’t run into any problems with missing browser API calls or strange behaviour with some headless browsers, e.g. Phantom.js. This approach is also extensible for other browsers and can provide a very good strategy for making sure that your application works on any browser you support.

That’s our acceptance test suite in a nutshell. We hope this approach helps you to improve your acceptance testing workflow as well.

Caveats

But hold on just for a bit …

WebDriverIO has a couple of caveats we found while working with it for last couple weeks. One of them is waitForExist. If you’re using waitForExist, you should know it actually checks only if the element is present in DOM. You might want to use waitForVisible in conjunction with waitForExist, since by clicking on the element, your test will fail if something is overlapping the element – waitForVisible will correctly wait for the temporary overlapping element to disappear.

Let us know in the comments section below what you think about this workflow. We’re looking forward to hear your comments and opinions!