One of the core ideas behind Facebook’s React library is that there should be no need to learn a new API for things that you already know how to do in vanilla JavaScript. Why bother memorizing Angular’s ng-repeat syntax when you can just use good-old Array.map()? That’s a good idea, and it’s a big part of what made the project so appealing to developers in the first place. So then why should Jest–the same company’s JavaScript testing framework, and a popular choice among React developers–encourage you to learn a new assertion API, and to write code like

expect(result).toEqual(expect.not.stringContaining(unexpectedSubstring));

when you already know how to assert the same thing using vanilla JavaScript?

assert(!result.includes(unexpectedSubstring));

The sort of assertion patterns encouraged by Jest’s API seem fundamentally antithetical to the philosophy behind React.

This definitely isn’t a Jest-specific complaint (and Jest is also notably much more than an assertion library). It applies to the majority of popular testing assertion libraries in JavaScript, such as chai, must.js, and should.js. Testing in the JavaScript world tends to be heavily oriented towards BDD-style assertion libraries with verbose and complex assertion APIs in general. That’s fine if you’re into that kind of thing, but it’s a lot of extra syntax to memorize when you already know other ways to perform the same assertions in a simpler and more concise way.

Unsurprisingly, there are some good reasons for why these assertion libraries tend to work the way they do. If you simply use Node’s built in assert module to make assertions, then your error messages provide you with fairly limited context about what’s being tested. The vanilla JavaScript version of our earlier “unexpected substring” test will fail with the same error message regardless of what the strings contain.

AssertionError [ERR_ASSERTION]: false == true

That tells you that the test failed, but it would be useful to see what the values of the result and unexpectedSubstring variables are so that you can determine why the test failed.

Here’s the output when the Jest test fails for comparison.

expect(received).toEqual(expected)

Expected value to equal:
  StringNotContaining "World"
Received:
  "Hello World"

This immediately lets us know that the string “Hello World” unexpectedly contained the substring “World.” We could of course write your own more informative assertion error messages using the vanilla approach, but this would requires additional work for each test. It’s not too bad for this substring case, something like this would suffice.

assert(
  !result.includes(unexpectedSubstring),
  `"${result}" unexpectedly included "${unexpectedSubstring}".`,
);

This approach really doesn’t scale well to more complicated tests however, especially when dealing with arrays and nested objects. With Jest et al., you get useful and informative error messages out of the box.

For a long time, I thought that you had to choose between using a verbose assertion API or writing your own assertion messages. Well, it turns out that that isn’t the case! There’s a library out there called Power Assert that makes it possible to use the standard assert module while still getting highly informative error messages. It’s the best of both worlds.

The basic premise behind Power Assert is that there’s no need to use a complex API in order to give an assertion library context about what’s being tested; that information can instead be deduced from the code itself. That of course means that Power Assert needs to have access to the code itself, but, for better or worse, JavaScript tends to make it’s way through a transformation or two before the code runs these days anyway. Using tools like Babel, WebPack, and Gulp is the norm rather than the exception, and Power Assert has plugins available for each of these to provide more informative assertion messages.

Returning to our “unexpected substring” from earlier, Power Assert will transform the assertion so that it produces an error message like the following.

assert(!result.includes(unexpectedSubstring))
        ||      |        |
        ||      true     "World"
        |"Hello World"
        false

     + expected - actual

     -false
     +true

This message allows us to immediately see what the values of result and unexpectedSubstring are, and each intermediate value during evaluation. The code that was run here is exactly identical to what we ran earlier. The only thing that was changed was that I enabled the Babel Power Assert preset before running the test a second time. That’s literally all it takes to start getting more useful error messages if you’re already using Node’s assertion module. You don’t need to learn any new assertion methods, you can just use the existing assert API.

In fact, the slogan of Power Assert is “no API is the best API.” That’s a well-put sentiment that really resonates with me as a developer. It’s my favorite thing about React in comparison to earlier web frameworks, and it’s also one of the main ideas behind our browser automation framework Remote Browser. We developed the library to have as minimal of an API as possible while making it extremely easy for developers to use vanilla JavaScript, HTML browser contexts, and the Web Extensions API to accomplish complex testing and web scraping tasks (you can check out the Remote Browser Interactive Tour to see what I mean by that). We didn’t come up with as snazzy of a slogan, but I feel like it would apply nearly as well to Remote Browser as to Power Assert.

That resonance inspired me to reach out to Takuto Wada, the primary author of the Power Assert library. Long story short, we ended up doing a brief interview about the project. We’ll take a look at what he had to say in the next section, and then we’ll move on to a quick tutorial about how Power Assert can be integrated into a JavaScript project. Once it’s been added to an example project, we’ll construct a handful of tests as examples to see the sort of error messages that you can expect assertions to produce with Power Assert.

Mini-Interview with Takuto Wada

These questions and answers are from a series of emails which Takuto Wada and I exchanged. Takuto Wada–in addition to being the primary author of Power Assert–has over 20 years of experience in programming, is well-known as a “TDD evangelist” in Japan, and has translated Kent Beck’s Test-Driven Development into Japanese. Let’s see what he has to say about other testing frameworks, the origin of Power Assert, and the philosophy behind the project.

1. There are some similar frameworks available in other languages which also use the “Power Assert” name, but it looks like your version came first based on their commit histories. Is that true? If so, did you coin the term “Power Assert,” and how did you initially get the idea for Power Assert?

   Not true. The origin of “Power Assert” is Spock framework written in groovy. When I first saw groovy’s Power Assert, I was so surprised and attracted.

   On Jan 08, 2013, I’ve decided to implement power-assert as POC, using Mozilla SpiderMonkey Parser API, a predecessor of the ESTree Spec.

2. Since you first started working on Power Assert, the popularity of major assertion libraries has shifted around a bit. One notable shift is that Jest has risen from being relatively obscure to being as popular as Chai. Do you have any thoughts on whether Jest provides a better assertion interface than Chai, or is it just more of the same?

   Just more of the same. Jest is a fork of Jasmine, kind of “BDD style” dialect of testing frameworks. Jest is more modest than chai, however it’s basically the same, tons of matchers, tons of APIs to learn.

   Kent Beck once said, “There is a clash between constraints. Easy to write vs. Easy to learn to write.” Chai, Jest, and its ancestor RSpec aims to “Easy to write”. Power-assert aims to “Easy to learn to write” = simple.

3. Are there any utility libraries that you think couple particularly well with Power Assert? Informative error messages are half of the equation, but some assertions can require a bit of boilerplate to construct from scratch. Libraries like Lodash provide a wide range of methods that facilitate complex interactions with objects and arrays, and which can be used to write some tests in a less verbose way. Would you consider Lodash to be a good complement to Power Assert, or is it merely trading the complexity of one API for another?

   Functions or methods that simply returns boolean (e.g. lodash) works well with power-assert. Boolean functions tend to be avoided due to the less informative behavior on test failure, but with power-assert, you can gain informative failure message. Simplicity wins.

4. The philosophy behind Power Assert seems to be quite minimalist and in-line with Keep It Simple Stupid (KISS). I’m curious whether that’s a preference that carries over to your other development tooling. What does your development hardware/software setup look like, and do you consider it be oriented towards minimalist tools?

   I’m an Unix citizen for 20 years so my development stack consists of small and beautiful FLOSS tools.

   • Macbook Pro
   • Mainly Emacs, sometimes WebStorm, Atom, VSC.
   • zsh / nodebrew / ghq / peco

   As you guessed, philosophy behind my product is KISS principle. Simplicity matters most. My programming style is strongly influenced by Unix philosophy and also philosophy of Rich Hickey, the creator Clojure language.

   • The UNIX Philosophy
   • The Art of Unix Programming
   • Rich Hickey fanclub

5. Have your contributions towards Power Assert been purely on your own time, or have you been able to work on it as part of your employment?

   Unfortunately, purely on my own time.

6. I’m going to go out on a limb here and guess that you believe that chainable semantic APIs in BDD assertion styles are needlessly complex.

   Yes.

   If that’s true, then do you feel the same way about chainable semantic APIs in general, or are there some cases where it’s a good choice?

   I saw many Chainable semantic APIs in BDD assertion styles gradually become complex and complicated. They are “easy to write” but not “easy to learn to write”. Chainable semantic APIs (a.k.a Fluent interface sometimes works well, especially in statically typed languages with help of compiler and IDE’s method completion (e.g. SQL Query Builder written in Java).

   Is there a library that you know of that uses a chainable API that you think pulls it off really well?

   “Scopes” in Rails is well designed and works well.

7. I’m writing this article because I’m a big fan of Power Assert, and I think that there are probably a lot of people who would love to use it but haven’t heard of it yet. What are a few open-source projects that you’re not affiliated with, but which you think more people should know about?

   I’d like to introduce ghq and peco. They made my OSS life really enjoyable.

8. Is it OK if we steal the phrase “no API is the best API,” and use it when talking about Remote Browser :-)?

   Of course, yes :)

Setting Up Power Assert with Mocha and Babel

Let me start off by saying that there are a lot of plugins, presets, tasks, and whatnot for integrating Power Assert with the various JavaScript development toolsets. I’m going to focus here on using Babel in conjunction with Mocha because that’s the setup that I like to use for my own tests. You can find instructions relevant to many other scenarios in the main Power Assert repository if you prefer a different testing setup. The actual error messages and testing strategies that we’ll look at should be independent of the tooling however, so it will hopefully be useful to see Power Assert in action regardless.

I’ll also mention that the final project configuration and the tests that we’ll be writing are available in the intoli-article-materials repository. If you’re trying to use this code as a template for adding Power Assert to your own project, then it may be useful to head over there where you can see the finished product. Oh, and star the repository while you’re over there! We put up supplementary materials for most of our articles over there, and starring the repository is a great way to find out about new and upcoming articles.

Now that that’s all out of the way, let’s get started on the project setup. As always with JavaScript projects, you’ll want to create a package.json file in an otherwise clean working directory. Adding the following contents to the package.json file will specify all of the development dependencies that you need for running tests with Power Assert enabled.

{
  "scripts": {
    "test": "NODE_ENV=testing mocha --exit --require babel-register"
  },
  "devDependencies": {
    "babel": "^6.23.0",
    "babel-core": "^6.26.3",
    "babel-preset-env": "^1.7.0",
    "babel-preset-power-assert": "^2.0.0",
    "babel-register": "^6.26.0",
    "mocha": "^5.2.0",
    "power-assert": "^1.5.0"
  }
}

Running npm install or yarn install will then install the dependencies into a local node_modules/ subdirectory. Only two of these dependencies are Power Assert specific here… the ones with “power-assert” in their names. These are the main power-assert package and the babel-preset-power-assert Babel preset that makes Power Assert work with Babel. The other dependencies cover the Mocha testing framework and a minimal set of Babel packages.

One other thing to note from the package.json file is that we defined a test script command. This isn’t specific to Power Assert, but we’re doing two things that are important here: setting the NODE_ENV environment variable to testing, and using the --require babel-register command-line argument when running Mocha. The --require flag simply tells Mocha that it should require a specific package before running the tests.

In this case, the flag tells Mocha to evaluate require('babel-require') before the tests. That seems fairly innocuous, but it has a profound impact on how our test code will be run. The babel-register package is a special part of Babel which, when required, will bind itself to Node’s require() method and cause any further required packages to be transpiled by Babel on the fly.

The behavior of the transpilation can be configured by creating a .babelrc file. A basic .babelrc configuration which includes support for Power Assert might look something like the following.

{
  "env": {
    "testing": {
      "presets": [
        "power-assert"
      ]
    }
  },
  "presets": [["env", {
    "targets": {
      "node": "6.10"
    }
  }]]
}

The only part of this that’s actually specific to Power Assert is that we’ve told Babel to add the Power Assert preset when NODE_ENV is testing (i.e. when we’re run our test script). This testing configuration will be merged with the default configuration. In this case, the default configuration just specifies that the Babel env preset should be used to dynamically determine which plugins are necessary to target Node 6.10.

After adding our .babelrc file, everything should be ready to work with Babel and Power Assert. Now just create a subdirectory called test, the default search directory for Mocha, and create a JavaScript file called test/test-assertion-errors.js with the following contents.

import assert from 'assert';


// Note that all of these tests are designed to fail, so that we can see the error messages!
describe('Power Assert Testing Examples', () => {
  it('check that an unexpected substring is not found', () => {
    const result = 'Hello World';
    const unexpectedSubstring = 'World';

    // Jest Equivalent: expect(result).toEqual(expect.not.stringContaining(unexpectedSubstring));
    assert(!result.includes(unexpectedSubstring));
  });
});

We’re able to use the ECMAScript 2015 import syntax here because our tests are being transpiled by Babel. Any other configuration that you add to your .babelrc file will also apply to your tests, so you can use fun stuff like transform-object-rest-spread and transform-optional-changing if you’re into that sort of thing. The only part that really matters for Power Assert, however, is that we specified the Power Assert preset as part of the configuration.

You can now run the test with either yarn test or npm run test, and you’ll see the same informative error message that we looked at in the introduction.

assert(!result.includes(unexpectedSubstring))
        ||      |        |
        ||      true     "World"
        |"Hello World"
        false

     + expected - actual

     -false
     +true

Any other tests that you write using the assert module will also include similarly helpful error messages. We’ll take a look at some practical examples in the next section!

Power Assert in Action

Now that we have everything configured, let’s take a quick look at a few common real-world testing patterns and the assertion messages that Power Assert produces in them. We’ll write each assertion using vanilla JavaScript and the Node assert module, but we’ll also include commented-out Jest assertions for comparison. Note that only the individual tests definitions will be shown for brevity. Each test is intended to be placed into the describe() block inside of test/test-assertion-errors.js, so you can copy and paste them there if you would like to run the tests yourself (with yarn test).

Check that no members of an array are included in another array

In this first test, we’ll have an array called result and we’ll need to check that it doesn’t contain any of the elements from a second array called unexpectedMembers. There are several different logically equivalent ways that you could write an assertion for this test. You could loop through result and assert that each member isn’t part of unexpectedMembers, you could loop through unexpectedMembers and assert that each member isn’t part of result, you could compute the intersection between the two arrays considered as sets and assert that it’s the empty set, etc. I personally think that each of these approaches imply slightly different intentions, and that the most appropriate choice in any given situation depends on the underlying context and purpose of the test.

When I wrote this particular test, I was thinking of result as a meaningful sequence of words and unexpectedMembers as a collection of unrelated words. Given that context, it makes sense to break the overall assertion up into a series of separate assertions where result is checked for each unexpected member in sequence. We can use Array.forEach() to loop over unexpectedMembers, and then Array.include() to check whether result includes one of the unexpected members.

it('check that no members of an array are included in another array', () => {
  const result = ['Hello', 'World'];
  const unexpectedMembers = ['Evan', 'World'];
  // Jest Equivalent: expect(result).toEqual(expect.not.arrayContaining(unexpectedMembers));
  unexpectedMembers.forEach(member =>
    assert(!result.includes(member))
  );
});

Running this test will fail and produce the following error message.

assert(!result.includes(member))
       ||      |        |
       ||      true     "World"
       |["Hello","World"]
       false

    + expected - actual

    -false
    +true

This message shows us the full value of result, the specific member of unexpectedMembers that the test is failing for, the fact that result.includes(member) evaluates to true, and that we’re explicitly negating that value to false.

Check that a regular expression matches a string

This one is pretty simple: we have a string called result and we want to check that a regular expression called regex matches it. JavaScript has great support for regular expressions, and we can just use RegExp.test() to determine whether a regular expression matches the result string.

it('check that a regular expression matches a string', () => {
  const regex = /^Hello World!/;
  const result = 'Hello World';
  // Jest Equivalent: expect(result).toEqual(expect.stringMatching(regex));
  assert(regex.test(result));
});

This test will fail due to the stray exclamation point in the regular expression, and the produced error message will look like this.

assert(regex.test(result))
       |     |    |
       |     |    "Hello World"
       |     false
       /^Hello World!/

    + expected - actual

    -false
    +true

We can easily see from this what the value of the regular expression is, the value of the result string, and that the regular expression check failed.

Check that an array contains at least one number

In this example, we’ll want to check that an array called result includes at least one number. The Array.some() method allows us to check if a function evaluates to true for any elements in an array. We can use this in conjunction with an arrow function that checks the type of each member in order to construct this test.

it('check that an array contains at least one number', () => {
  const result = ['Hello', 'World'];
  // Jest Equivalent: expect(result).toContainEqual(expect.any(Number));
  assert(result.some(member => typeof member === 'number'));
});

This test will also fail of course, and its output will look something like this.

assert(result.some(member => typeof member === 'number'))
       |      |
       |      false
       ["Hello","World"]

    + expected - actual

    -false
    +true

This lets us clearly see how we defined the arrow function that checks each member, the value of the result array, and the fact that the arrow function evaluated to false for all of the members.

Check for deep equality between two objects

We’ve exclusively been making bare assert() calls so far, but I should point out that the assert module does have an API that extends beyond this. This includes methods like assert.deepStrictEqual()/assert.notDeepStrictEqual() for checking deep equality, assert.throws() for checking that an error is thrown, and assert.rejects() for checking that a promise is rejected. Much of this API becomes superfluous when using Power Assert for the same reasons that apply to third-party assertion libraries. For example, there’s really little benefit to making an assert.strictEqual(a, b) call over assert(a === b) with Power Assert, and similar statements apply to most of the rest of the API.

I consider the deep equality comparisons to be a notable exception to this. Although it’s quite possible to check this manually using recursion, this is one case where I don’t think it makes sense to code it yourself. The other examples that we’ve looked at so far really haven’t necessitated sacrificing any brevity of code or clarity of intention in order to use bare assert() calls, but we would need to sacrifice both of these to manually check for deep equality. So we’ll use assert.deepStrictEqual() instead, and the assertion’s error message will still get transformed by Power Assert.

  it('check for deep equality between two objects', () => {
    const expectedResult = { 'a': [1, 2], 'b': [1, 2] }
    const result = { 'a': [1, 2], 'b': [1, 2, 3] }
    // Jest Equivalent: expect(result).toEqual(expectedResult);
    assert.deepStrictEqual(result, expectedResult);
  });

Running this test will output the following.

assert.deepStrictEqual(result, expectedResult)
                       |       |
                       |       Object{a:#Array#,b:#Array#}
                       Object{a:#Array#,b:#Array#}

    + expected - actual

       ]
       "b": [
         1
         2
    -    3
       ]
     }

This tells us that both result and expectedResult are objects that both contain array properties called a and b. We can also tell that the result.b array includes a third element with a value of 3 that isn’t expected to be there.

The error message here is fairly useful, but it’s worth noting that you can customize the behavior of Power Assert to show the actual values of each array rather than #Array#. There’s a maxDepth option which determines how many levels of nesting there can be before objects are replaced with generic identifiers. Customizing the behavior requires adding a bit of Power-Assert-specific code to our tests, but we can gracefully fall back to the standard assert module when Power Assert isn’t being used with an initialization like this.

import uncustomizedAssert from 'assert';


const assert = !uncustomizedAssert.customize ? uncustomizedAssert : (
  uncustomizedAssert.customize({
    output: {
        maxDepth: 5,
    }
  })
);

This customization allows us to see the actual value of the arrays inside of each object.

assert.deepEqual(result, expectedResult)
                 |       |
                 |       Object{a:[1,2],b:[1,2]}
                 Object{a:[1,2],b:[1,2,3]}

    + expected - actual

       ]
       "b": [
         1
         2
    -    3
       ]
     }

Conclusion

Well, I hope that this was able to inspire a few people to give Power Assert a try! The assertion messages that it produces won’t always be as specialized as those that you get from Jest or other libraries, but I’ve found them to be more than adequate for daily usage. It’s generally helpful to drop into a debugger anyway when you’re dealing with extremely complicated assertions, and the error messages become secondary at that point.

I used Mocha as the test runner for the examples here, but I would also like to give a quick shout-out to AVA. It’s a testing framework which uses both Babel and Power Assert out of the box, and it additionally includes some of Jest’s more interesting features (e.g. support for snapshot testing). It’s highly opinionated, but it’s definitely worth checking out if there are things you like about both Jest and Power Assert.