Writing your own Karma adapter

September 9th, 2013 by Misha Reyzlin

Background

When we started to work on the new version of our mobile web app, we knew we wanted to run unit tests on a wide variety of clients, mobile devices, PhantomJS, and on Chrome when running locally. Because we practice continuous integration, we knew we also wanted Git hooks and proper results formatting.

We chose Karma runner, which is a project from the Angular JS team that provides developers with a “productive testing environment”. One of the advantages that Karma runner offers over other similar projects is its ability to use any testing framework. At SoundCloud, we aim to have the same toolset across various JavaScript projects, and our unit test framework of choice is Tyrtle.

Using Tyrtle

You can write your own Karma adapter by using the Tyrtle example that follows. The idea is to tie your tests to the Karma API. The pieces of information that you need are the number of tests, test suites or modules, the results of each test (with possible assertion or execution errors, or both), and a hook to let Karma know that the runner ran all of the tests.

You also need to provide a start function that configures the unit test framework, loads the test files, and starts the tests.

The basic template for an adapter is as follows:

(function (win) {
/**
 * Returned function is used to kick off tests
 */
function createStartFn(karma) {
  return function () {
  };
}

/**
 * Returned function is used for logging by Karma
 */
function createDumpFn(karma, serialize) {
  // inside you could use a custom `serialize` function
  // to modify or attach messages or hook into logging
  return function () {
    karma.info({ dump: [].slice.call(arguments) });
  };
}

win.__karma__.start = createStartFn(window.__karma__);
win.dump = createDumpFn(win.__karma__, function (value) {
  return value;
});
})(window);

Next, create a renderer/reporter for the unit test framework that will pass the data to Karma. Tyrtle has a renderer that can render HTML, XML for CI, or print to any other type of output.

To pass the data to Karma, implement the methods that follow:

/**
 * Tyrtle renderer
 * @interface
 */
function Renderer () {}
Renderer.prototype.beforeRun  = function (tyrtle) {};
Renderer.prototype.afterRun   = function (tyrtle) {};
Renderer.prototype.afterTest  = function (test, module) {};

The createStartFn function creates a renderer object, with a Karma runner instance available within the start-function’s scope.

Create a parameter named karma:

function TyrtleKarmaRenderer (karma) {
  this.karma = karma;
}

Tell karma what the total number of tests is:

/**
 * Invoked before all tests are run; reports complete number of tests
 * @param  {Object} tyrtle Instance of Tyrtle unit tests runner
 */
TyrtleKarmaRenderer.prototype.beforeRun = function (tyrtle) {
  this.karma.info({
    // count number of tests in each of the modules
    total: tyrtle.modules.reduce(function(memo, currentModule) {
      return memo + currentModule.tests.length;
    }, 0)
  });
};

After each test, pass the resulting data to Karma:

/**
 * Invoked after each test, used to provide Karma with feedback
 * for each of the tests
 * @param  {Object} test current test object
 * @param  {Object} module instance of Tyrtle module
 *                  to which this test belongs
 */
TyrtleKarmaRenderer.prototype.afterTest = function (test, module) {
  this.karma.result({
    description: test.name,
    suite: [module.name + "#"] || [],
    success: test.status === Tyrtle.PASS,
    log: [test.statusMessage] || [],
    time: test.runTime
  });
};

Next, inform Karma that the tests have all finished running:

/**
 * Invoked after all the tests are finished running
 * with unit tests runner as a first parameter.
 * `window.__coverage__` is provided by Karma.
 * This function notifies Karma that the unit tests runner is done.
 */
TyrtleKarmaRenderer.prototype.afterRun = function (/* tyrtle */) {
  this.karma.complete({
    coverage: window.__coverage__
  });
};

You now have a renderer constructor. Next, turn your attention to the createStartFn function. You need to configure and initialize the unit test framework that returns a function, which requires a list of test files that are served from the Karma server and starts the actual runner.

Karma serves the files that are required for testing from a path that Karma creates and timestamps the files to avoid caching issues in browsers. Karma makes each path available as a key in a hash named __karma__.files. This makes Karma a bit tricky to configure when using an AMD-loader such as require.js. To understand how to use AMD with Karma, go to:

http://karma-runner.github.io/0.8/plus/RequireJS.html

Here is the final createStartFn function:

/**
 * Creates instance of Tyrtle to run the tests.
 *
 * Returned start function is invoked by Karma runner when Karma is
 * ready (connected with a browser and loaded all the required files)
 *
 * When invoked, the start function will AMD require the list of test
 * files (saved by Karma in window.__karma__.files) and set them
 * as test modules for Tyrtle and then invoke Tyrtle runner to kick
 * off tests.
 *
 * @param  {Object} karma Karma runner instance
 * @return {Function}     start function
 */
function createStartFn(karma) {
  var runner = new Tyrtle({});

  Tyrtle.setRenderer(new TyrtleKarmaRenderer(karma));
  return function () {
    var testFiles = Object.keys(window.__karma__.files)
      .filter(function (file) {
        return (/-test\.js$/).test(file);
      })
      .map(function (testFile) {
        return testFile.replace('/base/public/', '').replace('.js', '');
      });
    require(testFiles, function (testModules) {
      // test files can return a single module, or an array of them.
      testFiles.forEach(function (testFile) {
        var testModule = require(testFile);
        if (!Array.isArray(testModule)) {
          testModule = [testModule];
        }
        testModule.forEach(function (aModule, index) {
          aModule.setAMDName(testFile, index);
          runner.module(aModule);
        });
      });
      runner.run();
    });
  };
}

To find more examples of how this all fits together, see the scripts test-main.js (the RequireJS configuration to work with Karma) and karma.conf.js. Also, there are many adapter implementations such as Mocha, NodeUnit, and QUnit on the Karma GitHub page.

Ursula Kallio contributed to the writing of this post.