Automated UI testing with Nightmare

At Scaledrone we put a heavy emphasis on testing. While our backend is well tested, the website has been lacking in testing for a while.

In past projects, we have used Capybara for UI testing. As our dashboard is written in Node.js, we wanted to browse around to see if any Node.js UI testing frameworks have caught up with the all-mighty rodent.

Introducing Nightmare

We decided to give Nightmare a go. Nightmare is a high-level browser automation library from Segment written in JavaScript. Rushing ahead we can say that Nightmare is a step in the right direction for Node.js UI testing frameworks but still needs some time to catch up with all of the features of Capybara.

Without further ado let's get started with the code.

Nightmare itself is just used to interact with the UI, to actually write tests with it a testing library needs to be used. We decided to use Mocha, but other libraries such as Jest or Jasmine would also work.

Installing dependencies

npm install mocha --save-dev
npm install chai --save-dev
npm install nightmare --save-dev

We'll also be installing some generator flow libraries.

npm install co --save-dev
npm install co-mocha --save-dev

Add this part to your package.json file.

"scripts": {
  "test": "mocha --timeout 10000"
}

Put the test files into the test/ directory and run them with:

npm test

Taking a look at the Nightmare code example

The Nightmare GitHub repository basic example looks like this.

import Nightmare from 'nightmare';
import {expect} from 'chai';
 
describe('test duckduckgo search results', () => {
  it('should find the nightmare github link first', (done) => {
    const nightmare = Nightmare()
    nightmare
      .goto('https://duckduckgo.com')
      .type('#search_form_input_homepage', 'github nightmare')
      .click('#search_button_homepage')
      .wait('#zero_click_wrapper .c-info__title a')
      .evaluate(() =>
        document.querySelector('#zero_click_wrapper .c-info__title a').href
      )
      .end()
      .then((link) => {
        expect(link).to.equal('https://github.com/segmentio/nightmare');
        done();
      })
  });
});

While this example works fine, it's not an ideal approach for larger projects. Some things that we could improve are:

Use generators and the yield syntax instead of promises. In our opinion, it's much easier to write tests in a blocking style. Also, generators are much easier to use inside loops and more complex testing cases.
Generate helper functions to get rid of the evaluate() calls. One drawback of Nightmare is that it has a pretty limited API. The only way to parse text or hrefs from the page is to use evaluate.

A better approach

Having created an Nightmare.prototype.href() function and used generators, the code is much more concise.

require('co-mocha');
const Nightmare = require('nightmare');
const {expect} = require('chai');
 
Nightmare.prototype.href = function*(selector) {
  yield this.wait(selector);
  const href = yield this.evaluate(selector =>
    document.querySelector(selector).href
  , selector);
  return href.trim();
};
 
describe('test duckduckgo search results', function() {
  it('should find the nightmare github link first', function*() {
    const t = Nightmare()
    yield t.goto('https://duckduckgo.com')
    yield t.type('#search_form_input_homepage', 'github nightmare')
    yield t.click('#search_button_homepage')
    const link = yield t.href('#zero_click_wrapper .c-info__title a')
    expect(link).to.equal('https://github.com/segmentio/nightmare');
    yield t.end()
  });
});

Now that we have a good base for the tests let's see how we could set up a more real-world UI testing project.

Tips & tricks

Getting the text content of an element

A typical pattern is to check if a particular element or page contains some text. Use this helper function to do this with one line instead of four.

Nightmare.prototype.innerText = function*(selector) {
  yield this.wait(selector);
  const text = yield this.evaluate(selector =>
    document.querySelector(selector).innerText
  , selector);
  return text.trim();
};
 
// usage
const emptyStateText = yield nightmare.innerText('.empty-state');
expect(emptyStateText).to.equal('Get started');

Reusable Nightmare instance initialization

When starting a new Nightmare session, there are common activities that need to be run for each test. Extract them into a function.

function startTest = function*() {
  const t = Nightmare({
    show: true,
    typeInterval: 3,
    pollInterval: 50,
  });
  yield t.viewport(1000, 800);
  yield t.goto('http://localhost:3333/');
  yield t.wait('.auth-page__title');
  return {t}; // nightmare instance needs to be wrapped into an object for this to work
};
 
// usage
it('tests something', function*() {
  const {t} = yield startTest();
  yield t.type('.foo', 'bar');
  .
  .

Fixing wait

For the wait() function to work we had to set the pollInterval to 50 instead of the default 250ms.

const t = Nightmare({
  pollInterval: 50,
});

Conclusion

Nightmare does a pretty good job at what it provides. We noticed a minimal amount of random fails and most error messages gave proper context. It lacks functions for getting the content and parameters of elements; it also lacks the possibility to search for elements by their text content. Luckily it's a breeze to extend the library and add your own functions (which you should be doing).