Medryx Observations

In my last post, I talked about how to use Dojo’s DOH for unit testing. But the more I wrote about it and used it, the more I realized that there were a couple of things that made it difficult for me to wrap my head around.

First, setUp and tearDown were not similar to their counterparts in other unit-testing frameworks, such as JUnit.

Secondly, the whole model for programming an asynchronous test seemed a bit confusing.

So I have come up with what seems to me to be a ridiculously simple solution — a custom “test fixture”.

As you recall, when registering a set of tests, you pass the name of the set followed by an array of test methods or test fixtures. The methods are simple function calls. The test fixture had the following properties:

1. name
2. runTest
3. timeout
4. setUp
5. tearDown

In all the examples I found, this was declared as an associative array, setting each of these properties.

So a typical test looked like:

doh.register("some.test.package", [
function testOne() {
},
{
	name:"test2",
	setUp:function() {
		doSomeSetup();
	},
	tearDown:function() {
		doSomeTearDown();
	},
	runTest:function() {
		doh.assertTrue(doSomethingTrue());
	}
},
{
	name:"test3",
	setUp:function() {
		doSomeSetup();
	},
	tearDown:function() {
		doSomeTearDown();
	},
	runTest:function() {
		doh.assertTrue(doSomethingElseTrue());
	}
}
]);

But what if instead we write a class that implements all the above properties? Then we could subclass that class for a particular set of tests that shared a setUp/tearDown series. So in my first cut, I wrote a simple TestFixture class that looks like this:

dojo.declare("medryx.util.test.TestFixture", null, {
	constructor:function(testName, test) {
		this.name = testName;
		this.runTest = test;
	}
});

Okay, so its not rocket science. But already I feel like it cleaned up my tests substantially. Because now I have:

dojo.declare("some.test.fixture", medryx.util.test.TestFixture, {
	setUp:function() {
		doSomeSetup();
	},
	tearDown:function() {
		doSomeTearDown();
	}
});
 
doh.register("some.test.package", [
function testOne() {
},
new some.test.fixture("test2", function() {
	doh.assertTrue(doSomethingTrue());
}),
new some.test.fixture("test3", function() {
	doh.assertTrue(doSomethingElseTrue());
})
]);

Well that’s a start. But that got me thinking. Could I use this to simplify asynchronous tests?

As I discussed in my previous post, there is essentially a template for doing asynchronous testing:

1. Create a doh.Deferred
2. Do your deferred thing
3. Attach the doh.Deferred.getTestCallback(), instantiated with your own assertions, to you deferred thing’s callback/errback
4. Return the doh.Deferred

So I wrote a new and improved fixture:

/**
 * Use this test fixture for asynchronous tests.
 * 
 * @param {Object} testName
 * @param {Object} asyncTest: a method that ALWAYS returns a Deferred (dojo or doh)
 * @param {Object} assertionsCallback: a method that you will use to test your assetions once your defered is done
 */
dojo.declare("medryx.util.test.AsyncTestFixture", null, {
    constructor:function(testName, asyncTest, assertionsCallback) {
        this.name = testName;
        this.asyncTest = asyncTest;
		this.assertionsCallback = assertionsCallback;
    },
	runTest:function() {
		var d = new doh.Deferred();
		var actualDeferred = this.asyncTest();
		actualDeferred.addBoth(d.getTestCallback(dojo.hitch(this, "assertionsCallback")));
		return d;
	}
});

So now my asynchronous tests derive from this AsyncTextFixture and are created as:

	new subclass.of.AsyncTestFixture("testName",  function() {
		var def = doSomethingDeferred();
		return def;
	}, function(result) {
		doh.assertEqual("expected", result);
	});

I just think its easier to “remember” how to do aync testing with this fixture. Thoughts?

I have been spending the last few days experimenting with Dojo’s DOH for unit testing my javascript applications. While it has a relatively steep learning curve, and relatively little documentation, once you figure it out, it is actually quite simple and powerful. But it took me almost a whole day just to get up and running!

There are two great and simple articles about DOH on site pen that I found indispensible in learning DOH. The first talks about the logistics of setting up the tests and running them, and the second talks about how to do tests of asynchronous functions. There is a section of the Book of Dojo dedicated to this as well, but I found it hard to follow.

Setting up your test environment

The first trick to getting DOH running is to understand the “DOH Runner” application. This application lives in your /dojotoolkit/util/doh directory. If you launch it directly, it will run all the dojo unit tests. (This, by the way may be the most confusing part of runner.html. I’ve seen lots of posts about how people can’t figure out how to get runner working. I may work on cleaning this up to make it more intuitive for people to use without any difficult initial setup.) But you want to run your tests? To do this you provide the runner with a querystring argument called testModule that is your package with tests. For example, your url may look like:

../dojotoolkit/util/doh/runner.html?testModule=medryx.dao.tests.all

But that assumes that the module you pass in is a “sister” (i.e. at the same folder level) to dojo, dijit, and dojox. If it is not, then you will need a Dojo 1.2 implementation of runner.html and runner.js. Since that is not available yet, you will have to get a snapshot of the latest nightly build or access these files from subversion. These build in support for registerModulePath.

So lets say you have the following structure:

	/
	---/lib
	------/dojotoolkit
	---------/dojo
	---------/dijit
	---------/dojox
	---------/util
	------/mytoolkit
	---------/medryx

That means that when you call dojo.require(”medryx.something”) you want it to look at a path that dojo does not natively understand. So when loading dojo you have to configure djConfig with modulePaths or by calling dojo.registerModulePath() afer loading dojo. Well if you have this structure, and you are running dojo, then I am sure you know how to do this with dojo. The problem is that runner.html loads and tries to run your tests before you even get a chance to run any of your code. So the runner needs to know about the module paths when it is being called. The solution to this was adding registerModulePath as an option on the querystring for runner, so that now your url would look like:

../dojotoolkit/util/doh/runner.html?testModule=medryx.dao.tests.all&registerModulePath=medryx,../../medryx

So what does

medryx.dao.tests.all

look like? My general practice mirrors that of Dojo — I put a tests folder in every package.
The tests folder has the following content:

1. tests: each group in its own file (roughly a 1:1 number of test files to classes in the package)
2. all.js: a simple file that dojo.requires all the tests above
3. runTests.html file which is a simple meta redirect to runner.html with the appropriate querstring arguments already set. Such as:

   <html>
       <head>
       <title>Dojox Unit Test Runner</title>
       <meta http-equiv="REFRESH" content="0;url=../../../dojotoolkit/util/doh/runner.html?testModule=medryx.dao.tests.all&registerModulePath=medryx,../../medryx"></HEAD>
       <BODY>
           Redirecting to D.O.H runner.
       </BODY>
   </HTML>
   

So now execute runTests.html, and you run all the tests in that package. Hurray! But what does a test look like?

Writing Tests with DOH

To register a set of test methods with DOH to be run, you call doh.register with the name of the test, and an array of the actual tests. The tests in the array can be in one of two forms: a simple function(), or an associative array with the following properties: name, runTest (the test method), setUp, and tearDown — similar to junit. Notice that there is no setUp or tearDown that can be written once and then called from before and after every test (as there is in junit). The setUp and tearDown methods here only apply to the one call to runTest that is contained in that fixture. You can sort of get around this by creating a setUp and tearDown method of your own, as a sparsely populated fixture, and then just fill in name and runTest. For example:

dojo.provide("medryx.dao.tests.TeamsDaoTest");
dojo.require("medryx.dao.TeamsDao");
dojo.require("doh.runner");

dojo.declare("medryx.dao.tests.TeamsDaoTest.fixture", null, {
	constructor(name, runTest) {
		this.name = name;
		this.runTest = runTest;
	}
	setUp:function() {
		//do something before every test you want to run
	},
	tearDown:function() {
		//do something after every test you want to run
	}
});

doh.register("medryx.dao.tests.AbstractDaoTest",
[
	new medryx.dao.tests.TeamsDaoTest.fixture("someTest", function(){
		//do your test
	}),
	new medryx.dao.tests.TeamsDaoTest.fixture("someTest2", function(){
		//do your test2
	})
]);

I personally think that is a more natural use of setUp and tearDown. I mean, if I can only use the setUp and tearDown once, then why whouldn’t I just put it in the runTest method?

So now you have a simple set of synchronous tests. You have a set of assertions supported in doh (though I would like to see more in the future). But everything you need is there: assertTrue, assertFalse, and assertEqual. Note that these DO NOT support a message that is displayed during failure. I think that is a problem — it makes finding the reason for your failed test more work than it has to. I would love to see them extend each of these methods to contain an optional 3rd parameter that is a message that gets displayed when there is a failure.

There is a nifty assertError method that I would like to spend a couple seconds on as well. This assertion takes and expected error (i.e. an Error object — which is another reason to always subclass Error for your own exceptions!) as its first parameter. You then pass a scope, and the name of a function with an array of args, and it will execute the method inside of a try-catch, and if it does not get your expected error, it will fail. This is a little complicated, but here is an example that makes it perhaps a little more simple to understand.

{
	name:"testOneInstance",
	instantiate:function() {
		new medryx.dao.TeamsDao();
	},
	runTest:function() {
	   doh.assertError(medryx.exceptions.SingletonException, this, "instantiate");
	}
},

This will fail if creating a new medryx.dao.TeamsDao() does not return a SingletonException.

Asynchronous Tests

DOH really comes into its own with asynchronous testing. This is certainly not intuitive at first, but after you understand the pattern, it becomes fairly easy to implement. I have actually posted some suggestions to the dojo forum about ways that asynchronous testing could be made simpler with a minimal set of modifications, so we’ll see how that goes. Until then, this is how I suggest you implement asynchronous tests. Before you begin, make sure you really understand Deferred’s!

First, you need to understand the contract you are making with DOH when you run an asynchrounous test. You “mark” a test as asynchronous by having your runTest method return an instance of doh.Deferred. Notice that is NOT dojo.Deferred. DOH has its own Deferred implementation that you must return.

When you return a doh.Deferred from your runTest method, you are agreeing to, at some point run the .callback() or the .errback() methods on the deferred you return. The

d.errback(e)

method should be called with any exceptions that occur in your test, or if your assertions fail. Otherwise, call

d.callback(true)

So to summarize:

Notice how this is VERY different than the usual methodology for running tests in which you do something, and then make some assertions about the outcome, and the assertions themselves handle telling the test-framework whether the test passed or failed.

So an example of implementing an asynchronous test using this method would be: (crediting Dustin Machi)

doh.register("tests.DeferredExample",[
     function myDeferredTest(t){
           var dohDeferred = new doh.Deferred();
           var expectedResult = "foo";         

           var realDeferred = someAsyncFunction();

           realDeferred.addBoth(function(result){
                    if (result==expectedResult) {
                       //our test succeeded, fire the callback
                       dohDeferred.callback(true)
                    }else{
                       dohDeferred.errback(new Error("Unexpected Return: ", result));
                    }
           });

           return dohDeferred;
    },
    .... more test functions
]);

So can you even you assertions in this? Well, not quite. A failed assertion throw an error. If an error occurs in your test you must call errback. So if you surround your assertions in a try-catch block you can use them. So your new test looks like:

doh.register("tests.DeferredExample",[
     function myDeferredTest(t){
           var dohDeferred = new doh.Deferred();
           var expectedResult = "foo";         

           var realDeferred = someAsyncFunction();

           realDeferred.addBoth(function(result){
                    try{
		doh.assertEqual(result, expectedResult);

                       //our test succeeded, fire the callback
                       dohDeferred.callback(true)
                    } catch(e){
                       dohDeferred.errback(new Error("Unexpected Return: ", result));
                    }
           });

           return dohDeferred;
    },
    .... more test functions
]);

Well that is starting to feel a little more natural, but I’m still not quite satisfied. Its still a lot of glue code, and it still feels to hard to write quickly and easily and reliably. Enter the template method pattern.

doh.Deferred contains a method called getTestCallback that is a template for all of the above glue code. All you pass to getTestCallback is a function in which you perform your testing as usual, and the glue code is handled for you. So the above can be simplified to:

doh.register("tests.DeferredExample",[
     function myDeferredTest(t){
           var dohDeferred = new doh.Deferred();
           var expectedResult = "foo";         

           var realDeferred = someAsyncFunction();

           realDeferred.addBoth(dohDeferred.getTestCallback(function(result) {
		   doh.assertEqual(result, expectedResult);
	});

           return dohDeferred;
    },
    .... more test functions
]);

Now that is not too bad! Create the doh.Deferred, run your asynchronous test, and attach the doh.Deferred’s getTestCallback method (containing all of your assertions) to your asynchronous method’s callback and errback handlers. Excellent!

I hope this helps get you started on the wonderful road of javascript unit testing!

A quick look at the magic of dojo.Deferred.

I don’t know why it took me so long to grasp dojo.Deferred. But now that I have spent some time on it, I think I’ll be using it everywhere.

In previous implementations of asynchronous functions I would either pass into the asynchronous call a callback, or I would change the xhr to synchronous. Both are obviously fraught with problems.

Deferred’s operate as filter chain. They therefore REQUIRE that you return a value that will be passed to the next callback. So that means there not 100% safe to pass around willy-nilly, since you may be relying on an unfiltered result. But there is a very easy workaround for that too.

So here in a nutshell is the magic:

1. If you add a callback to a dojo.Deferred, it is guaranteed to get called unless there is an error. This means that regardless if your Deferred is already complete (i.e. your xhr request has already returned), if you add a callback to it 10 minutes later, it will still get called (immediately). That is brilliant, since it means you can code to a simple standard — with callbacks added to the Deferred’s chain.
2. If you have a Deferred that you do not want to allow clients to filter, then don’t pass the deferred to it. Instead, create a dojo.DeferredList that is composed of your “protected” deferred, and then pass that back to the client. They can muck with their return value all they want, and won’t touch your original Deferred. But the any callbacks added to the DeferredList will still be guaranteed to be called when your “protected” deferred returns.
3. One of the challenges of asynchronous programming for me in the past was this infinite propogation of callbacks, that I couldn’t tell what was going on. So I would forget about it, and use synchronous calls, and then switch back to a synchronous model of programming. But that certainly hurts the user experience. With Deferred, the flow of the program is much more evident, and easier to debug.
4. You can actually trigger a callback only when multiple Deferred’s return using a DeferredList. This is slightly different in its syntax, in that you will get a set of return values passed to the callback — all the Deferred items in the original DeferredList, with each result’s location corresponding to their index in the original DeferredList array.

Everybody likes to see some code, so here is an example of the magic:

dojo.require("dojo.DeferredList");

document.write("the first thing you should see is callback1 getting called with content = CONTENT ");

var def = new dojo.Deferred();
def.addCallback(function(content) {
	document.write("callback: 1, content:" + content + "");
	return content + "1";
})
def.callback("CONTENT");

document.write("the next thing you should see is callback2 getting called with content = CONTENT1 ");

def.addCallback(function(content) {
	document.write("callback: 2, content:" + content + "");
	return content + "2";
});

document.write("next, prove that a deferred list will give you the content (content12), but won't let you change the content");

var def2 = new dojo.DeferredList([def]);
def2.addCallback(function(content) {
    document.write("callback:3, content:" + content + "");
    return "SHOULD NEVER SEE THIS";
})

document.write("next, show callback4 with unchanged content (CONTENT12), despite callback3 changing it");

def.addCallback(function(content) {
    document.write("callback:4, content:" + content + "");
    return content + "3";
});

and the result should look like:

the first thing you should see is callback1 getting called with content = CONTENT
callback: 1, content:CONTENT
the next thing you should see is callback2 getting called with content = CONTENT1
callback: 2, content:CONTENT1
next, prove that a deferred list will give you the content (content12), but won’t let you change the content
callback:3, content:true,CONTENT12
next, show callback4 with unchanged content (CONTENT12), despite callback3 changing it
callback:4, content:CONTENT12