Yesterday marked the end of a several week sprint to improve tests around Johnny-Five. My goal started with fixing an inconstantly failing test, innocent enough. Today I'm proud to say we have cleaner code, a faster suite runtime, and a further decoupling of objects and libraries within the project. I learned a lot about testing in javascript, and I'm finally able to publicly share code I've written with regards to testing. A lot of this I'd love to find

Mock Firmata

The first step towards a cleaner test suite was to stop testing firmata. This required decoupling firmata from our Johny-Five `Board` object. Traditionally during construction `Board` would find the correct serial port (unless you gave it one) and create a `firmata` object with it. When the firmata object was connected board would then emit a ready event. Super useful and makes working with Johnny-Five a lot easier, however, it's tough to test.

Since we're able to specify the serial port, we could pass a `MockSerial` object that would save the last write. Comparing the writes to expected values would give us a good idea of what happened. Changing our constructor to accept an already connected firmata object allows us to pass a `MockFirmata` object that doesn't do anything except let us spy on it. This passing of objects to override the creation of objects internally is called dependency injection.

if (opts.firmata) {
  // If you already have a connected firmata instance
  this.firmata = opts.firmata;
  this.ready = true;
  this.pins = Board.Pins(this);
  this.emit("connected", null);
  this.emit("ready", null);
}

Testing Our Libraries

Johnny-Five loves Firmata. It's at the core of how it controls robots. Every device uses it to send and receive data from the connected hardware. As a result we did a lot of testing around firmata. Making sure we were sending and receiving the right data. As a result we'd get code like this.

var off = function(test) {
  this.led.off();
  test.deepEqual(serial.lastWrite, [145, 64, 1]);
  test.done();
};

This looks for the the right firmata protocol messages that should turn the `led` off on pin 13. You'll have to take my word on that, which is part of the problem. Is this test correct? It's hard to tell. Here's the new code.

var off = function(test) {
  this.led.off();
  test.ok(this.digitalWriteSpy.calledWith(13, 0));
  test.done();
};

Here we check a sinon spy (or mock if you prefer) on a `MockFirmata` object to make sure it got called with the right data. `board.digitalWrite(13, 0)` if you check the docs, `digitalWrite` takes a pin number and a value. The test confirms that's what we called.

"You shouldn't test your libraries." has been said a lot to me and in general (assuming you can trust your libraries) you shouldn't test your libraries. Firmata has it's own tests to cover digitalWrite. So we don't need to care how it works.

Speed and Fake Timers

We sped up the test suite from 2729ms to 552ms. That's a 5 times improvement. I's inconsequential when compared to setup time and travis-ci's total build time but it makes me happy. If Johnny-Five was a larger project this would be amazing. I've seen people start fights over this stuff.

How we sped it up however is important. We mock `setTimeout` and the other timer functions with sinon's fake timers. We were able to use them everywhere except with lodash. I think Lodash keeps it's own references to the timers and doesn't use the global timers when it needs them. So our attempts to fake them didn't work.

Testing the `Led#Strobe` method shows the pitfalls of not using fake timers.

var strobe = function(test) {
  var captured = [];
  var startAt = Date.now();
  test.expect(1);
  this.led.off();
  this.led.strobe(100); < /code>

  var interval = setInterval(function() {

    captured.push( serial.lastWrite.slice() );

    if ( Date.now() > startAt + 500 ) {
      clearInterval( interval );

      test.deepEqual( captured.slice(0, 5), [
        [ 145, 96, 1 ],
        [ 145, 64, 1 ],
        [ 145, 96, 1 ],
        [ 145, 64, 1 ],
        [ 145, 96, 1 ]
      ]);

      test.done();
    }
  }, 100);
};

Our first version samples the strobes every 100ms for 500ms and then compares the sample. We had 2 issues here. First, the test took 500ms minimum. Second, sometimes our timers would run out of order and the test would fail. We were also testing firmata again.

var strobe = function(test) {
  var clock = sinon.useFakeTimers();
  test.expect(3);
  this.led.off();
  this.led.strobe(100);

  clock.tick(100);
  test.ok(this.spy.calledWith(13, 1));
  clock.tick(100);
  test.ok(this.spy.calledWith(13, 0));
  this.led.stop();
  clock.tick(100);
  test.equal(this.spy.callCount, 3);
  clock.restore();
  test.done();
};

Our second version runs without delay and runs reliably without timing issues. It's also really clear to follow what's going on. It isn't necessary to actually wait for the computer, nor do we care about how firmata is doing what it's doing.

I hope this helps anyone working on testing. I enjoyed working on this with @divanvisagie, @rwaldron and @porteneuve.

Back to features!

-Francis