DevOps Zone is brought to you in partnership with:

Ariya is a passionate engineer interested in bleeding-edge technologies. He has been involved in various large projects, from KDE to WebKit. These days, his focus is mostly on software craftsmanship around web technologies. His (little) spare time is spent running the projects PhantomJS (headless WebKit) and Esprima (JavaScript parser). Ariya is a DZone MVB and is not an employee of DZone and has posted 59 posts at DZone. You can read more from them at their website. View Full User Profile

Code Coverage of Jasmine Tests using Istanbul and Karma

10.08.2013
| 5346 views |
  • submit to reddit

For modern web application development, having dozens of unit tests is not enough anymore. The actual code coverage of those tests would reveal if the application is thoroughly stressed or not. For tests written using the famous Jasmine test library, an easy way to have the coverage report is via Istanbul and Karma.

For this example, let’s assume that we have a simple library sqrt.js which contains an alternative implementation of Math.sqrt. Note also how it will throw an exception instead of returning NaN for an invalid input.

var My = {
  sqrt: function(x) {
    if (x < 0) throw new Error("sqrt can't work on negative number");
      return Math.exp(Math.log(x)/2);
  }
};

Using Jasmine placed under test/lib/jasmine-1.3.1, we can craft a test runner that includes the following spec:

describe("sqrt", function() {
  it("should compute the square root of 4 as 2", function() {
    expect(My.sqrt(4)).toEqual(2);
  });
});

Opening the spec runner in a web browser will give the expected outcome:

jasmine_runner

So far so good. Now let's see how the code coverage of our test setup can be measured.

The first order of business is to install Karma. If you are not familiar with Karma, it is basically a test runner which can launch and connect to a specific set of web browsers, run your tests, and then gather the report. Using Node.js, what we need to do is:

npm install karma karma-coverage

Before launching Karma, we need to specify its configuration. It could be as simple as the following my.conf.js (most entries are self-explained). Note that the tests are executed using PhantomJS for simplicity, it is however quite trivial to add other web browsers such as Chrome and Firefox.

module.exports = function(config) {
  config.set({
    basePath: '',
    frameworks: ['jasmine'],
    files: [
      '*.js',
      'test/spec/*.js'
    ],
    browsers: ['PhantomJS'],
    singleRun: true,
    reporters: ['progress', 'coverage'],
    preprocessors: { '*.js': ['coverage'] }
  });
};

Running the tests, as well as performing code coverage at the same time, can be triggered via:

node_modules/.bin/karma start my.conf.js

which will dump the output like:

INFO [karma]: Karma v0.10.2 server started at http://localhost:9876/
INFO [launcher]: Starting browser PhantomJS
INFO [PhantomJS 1.9.2 (Linux)]: Connected on socket N9nDnhJ0Np92NTSPGx-X
PhantomJS 1.9.2 (Linux): Executed 1 of 1 SUCCESS (0.029 secs / 0.003 secs)

As expected (from the previous manual invocation of the spec runner), the test passed just fine. However, the most particular interesting piece here is the code coverage report, it is stored (in the default location) under the subdirectory coverage. Open the report in your favorite browser and there you'll find the coverage analysis report.

branch_uncovered

Behind the scene, Karma is using Istanbul, a comprehensive JavaScript code coverage tool (read also my previous blog post on JavaScript Code Coverage with Istanbul). Istanbul parses the source file, in this example sqrt.js, using Esprima and then adds some extra instrumentation which will be used to gather the execution statistics. The above report that you see is one of the possible outputs, Istanbul can also generate LCOV report which is suitable for many continuous integration systems (Jenkins, TeamCity, etc). An extensive analysis of the coverage data should also prevent any future coverage regression, check out my other post Hard Thresholds on JavaScript Code Coverage.

One important thing about code coverage is branch coverage. If you pay attention carefully, our test above is still not exercising the situation where the input to My.sqrt is negative. There is a big "I" marking in the third-line of the code, this is Istanbul telling us that the if branch is not taken at all (for the else branch, it will be an "E" marker). Once this missing branch is noticed, improving the situation is as easy as adding one more test to the spec:

it("should throw an exception if given a negative number", function() {
  expect(function(){ My.sqrt(-1); }).
    toThrow(new Error("sqrt can't work on negative number"));
});

Once the test is executed again, the code coverage report looks way better and everyone is happy.

full_coverage

If you have some difficulties following the above step-by-step instructions, take a look at a Git repository I have prepared: github.com/ariya/coverage-jasmine-istanbul-karma. Feel free to play with it and customize it to suit your workflow!



Published at DZone with permission of Ariya Hidayat, author and DZone MVB. (source)

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)