Perceptual Difference Test using protractor

tl:dr npm script that compares two screenshots and validates its similarity in a jasmine test using yahoo’s blink-diff.

The term perceptual difference describes the difference of two images. This concept can be used to find regressions in your user interface, that are usually difficult to identify by conventional end-to-end tests, because their focus more on the DOM and only basic CSS.

Comparing two images is not trivial, but luckily there are a couple of libraries that address that issue. One of them is yahoo’s blink-diff ported as an npm package.

The following script takes the two arguments from the command line expectedUrl and actualUrl, that define two websites to compare. Protractor’s browser.takeScreenshot() will take the screenshots.

/**
 * Compares the perceptual difference of two urls by using a screenshot of those sites. If the difference is larger than `THRESHOLD` the test will fail.
 *
 * Usage: npm run test:pdiff -- --expectedUrl= --actualUrl=
 *
 * Without npm scripts it looks like this:
 * $ ./node_modules/.bin/protractor --specs /path/to/project/test/pdiff.js --expectedUrl=http://stable.myurl.com --actualUrl=http://deployment1.myurl.com
 */
const fs = require('fs');
const rimraf = require('rimraf');

const EC = protractor.ExpectedConditions;
const REPORTS_DIR = './build/reports/pdiff';
const THRESHOLD = 1; // in percent

// --

// cleanup report dir
rimraf(REPORTS_DIR, () => {
 fs.mkdirSync(REPORTS_DIR);
});

const expectedUrl = argv.expectedUrl;
const actualUrl = argv.actualUrl;

console.log(`comparing \`${expectedUrl}\` with \`${actualUrl}\``);

describe(`Perceptual-differences of ${actualUrl}`, () => {
 const getScreenshot = url =>
   new Promise((resolve, reject) => {
     browser.get(url);
     browser.waitForAngular();
     // wait for the spinner to disappear
     browser.wait(EC.not(() => videoCmp.spinner().isDisplayed()));

     browser.takeScreenshot()
      .then((base64) => {
        resolve(new Buffer(base64, 'base64'));
      })
     .catch(reject);
  });

  let imageExpected;
  let imageActual;

  beforeEach((done) => {
    getScreenshot(expectedUrl).then((screenshot) => {
      imageExpected = screenshot;
      done();
    });
  });

  beforeEach((done) => {
    getScreenshot(actualUrl).then((screenshot) => {
      imageActual = screenshot;
      done();
    });
  });

  it(`are less than ${THRESHOLD}%`, (done) => {
    const actualUrlFn = actualUrl.replace(/\/|\.|:/g, '_'); // replace / and . by _
    const diff = new BlinkDiff({
      imageA: imageExpected,
      imageB: imageActual,

      thresholdType: BlinkDiff.THRESHOLD_PERCENT,
      threshold: THRESHOLD / 100,

      // export the the images highlighting the difference
      imageOutputPath: `${REPORTS_DIR}/${actualUrlFn}.png`
    });

    diff.run((error, result) => {
      if (error) {
        throw error;
      } else {
        console.log(`Found ${result.differences} differences.`);
        expect(diff.hasPassed(result.code)).toBe(true);
        done();
      }
    });
 });
});

For simplicity you define an npm script for this:

"test:pdiff": "./node_modules/.bin/protractor --specs /path/to/project/test/pdiff.js",
Advertisements
Perceptual Difference Test using protractor

One thought on “Perceptual Difference Test using protractor

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s