Medryx Observations

A while back, I sent a post to the Dojo discussion forum regarding lazy loading and relationships between entities. The very simple response by Alex:

stores don’t “do” references…

Aha! That was the part I hadn’t figured out. Ever try to open a phillips-head screw with a flat-head screwdriver? It felt a bit like that. Over time, I came up with many solutions, but never a comprehensive one that I could use in multiple projects. I came up with several design patterns, and perhaps some re-usable code, but it all still felt like a hack.

My path towards a more comprehensive solution to managing related entities came from the server world with a similar, but more lightweight relational model I started to explore with this post I made a little while back. Well, many all-nighters later, I think I have a 0.1 alpha version of a cool system that makes a relational, lazy enabled, dirty-checking javascript object model a piece of cake. And there is plenty of sugar to make it fun to use. Oh, and its dojo.data compatible.

(As an aside, I was excited when I first read about the JsonRestStore.. Indeed, it probably does a lot of this same work. So it may be an issue of style. I also don’t know a whole lot about JSON referencing, and my referencing model (to be documented) seemed a lot easier to translate on the server side. But as I said, I need to learn a lot more about it. I hope to start here.)

Having done quite a bit of work with Java in the past, I am big believer in Object-Relational-Mappers on the server side. I have used Hibernate most extensively, and have even dabbled with PHP-based ORM’s like Propel. I like the idea of creating a model, and letting the system figure out how to persist it.

In the client-side world of Javascript, obviously a full-fledged ORM would be overkill. But I think an analogy of an ORM on the server-side — which maps relational-database information to domain-objects — can be made. The only difference is that you are mapping your server communication to your domain objects. I’m not sure what to call this yet, but I am open to suggestions. For now, I’ll borrow “ORM”, though technically that is not what this is. Perhaps client-server object mapper? (CSOM?)

What is MedryxORM?

MedryxORM is an extension of the Dojo toolkit that allows for simple communication between your client-server services and your javascript domain model. It consists of an EntityManager that handles all the mapping of your simple javascript-objects to your javascript-services and can be used to query and persist your domain objects. Similar to Hibernate, this system aims to be as unobtrusive to your usual coding as possible, but provides access to several ways in which you can optimize the performance of your application once you have it working, making refactoring of your application for performance a much simpler process. There are several “shortcuts”, that make using the system easier, but may be a bit more intrusive on your usual coding practices, so they are optional.

Enough jargon. Lets do some coding, and then I’ll reveal how it all works. Since I am a physician, my domains almost always deal with medical records, so I’ll use that as an example. Lets say that we want to build an application that shows a table of patients. You can then drill-down on patient details like insurance information, clinical information, etc. You can add simple notes to the patient, and perhaps assign a responsible physician/team of physicians. You can then page the appropriate responsible physician.

So lets first describe out domain objects in javascript. Here is a model.Patient:

dojo.declare("model.Patient", null, {
  //id
  id:null,
 
  //simple properties
  fullName:null,
  initials:null,
  medicalRecordNumber:null,
  birthdate:null,
  gender:null,
 
  //associations
  visit:null
 
  toString:function() {
    return this.fullName || this.getFullName(); //I'll explain this later.    
  },
 
  getAge:function() {
  //snip... uses birthdate to calculate an age
  }
});

And here is a model.Visit:

dojo.declare("model.Visit", null, {
  //id
  id:null,
 
    //simple properties
  accountNumber:null,
  admissionDate:null,
  admissionTime:null,
  dischargeDate:null,
    dischargeTime:null,
  admittingDiagnosis:null,
    bed:null,
  room:null,
  ward:null,
 
  coverageNotes:null,
 
  //associations
  team:null,
 
  //collection of insurances
  insurances:null,
 
  getLengthOfStay:function() {
    //snip -- calculates number of days in the hospital based on the admission date.
  }
});

Well, you get the idea. Nothing very exciting here. But now let’s “register” these class with the EntityManager:

 
entityManager.registerClass({
  entityClassName:"model.Patient",
  alias:"Patient"
});
 
entityManager.registerClass({
  entityClassName:"model.Visit",
  alias:"Visit"
});

I’ll get into mapping and configuring the EntityManager in just a minute. But first lets jump ahead and see how you can use this:

 
var patient = new Patient(10);
alert( patient.getFullName() + " is " + 
      patient.getAge() + " years old!" );

Now, as long as their is a patient whose id = 10 on the server, then this will all *just work*. There is NO other necessary calls to xhr, or rpc, or REST, or anything else (Though those things happen behind the scenes, depending on how you configure your EntityManager.)

This might be even cooler:

 
var patient = new Patient(10);
 
alert ( "Uh-oh, " + patient.getFullName() + " has been here for " + 
    patient.getVisit().getLengthOfStay() + " days!" );

Again, there is no further coding required. And because you map your server data to your javascript objects, you can actually use methods declared on those objects as you normally would. This example would result in two calls to the server — one to get the patient, and the next to get the visit (though that would not actuall occur until the getLengthOfStay() method called getAdmissionDate()). But that is a default behavior. You could give the server some “hints” about how you will be using the Patient, and let it send you all the information you need on the first call. By default, the code above would behave use synchronous data access methods, but its a very simple tweak to convert this exact same code to friendly asynchronous calls. But all in due time…

Now, admittedly, this is a contrived use. How often will you be instantiating an object with an identifier and expecting it to connect to some backend? But a more realistic use-case would be to use the fetch/query methods of the EntityManager (using fetch if you want to use dojo.data style access, or query if you want to use my Deferred-style access). Once you make the query, your results are all instances of your object model, with all of the appropriate relationships initialized in all the appropriate ways.

var deferred = entityManager.query({query:{entityClass:"Patient", fullName:"Smi*"});
deferred.addCallback(function(patients) {
  dojo.forEach(patients, function(patient) {
 
    document.write ( patient.getFullName() + " is " 
        + patient.getAge() + " years old!" );
 
  });
})

And in terms of persistence, you can choose from a number of different options. Some people prefer the DAO design pattern, others prefer ActiveRecord. I think this (sort of) gives you both. Or you can choose to completely forget about persistence, and let it all happen automagically (flushing on unload).

 
var patient = new Patient(10);
patient.setMedicalRecordNumber("123456");
patient.save();

or…

 
var patient = new Patient(10);
patient.setMedicalRecordNumber("123456");
 
entityManager.saveEntity(patient);

or…

var patient = new Patient(10);
patient.setMedicalRecordNumber("123456");
 
entityManager.save(); //flush all dirty records to the server

You are guaranteed to have only a single instance representing each persistent entity. So, this works fine (not that it has much purpose):

var patient = new Patient(10);
 
var visit = patient.getVisit();
 
var deferred = entityManager.query({query:{entityClass:"Visit", patient:10}});
deferred.addCallback(function(results) {
    assertTrue ( visit === results[0] ) ;
});
 
//or...
 
var visitId = visit.getId();
var newVisit = new Visit(visitId);
 
assertTrue ( visit === newVisit );

I’ll get into the details of optimization, caching, and other goodies later. Now that you’ve seen a little of the magic, lets dive into how its all configured.

At the core of the EntityManager is the class metadata — or “mapping”, inspired by Hibernate. You can specify your mapping in two ways — as a JSON object or “natively”. Your choice metadata is essentially stylistic, and is equivalent to the Java argument of Annotations based configuration versus XML based configuration.

To start with, lets define some terms, to make writing this a bit easier:

• Managed Class: any class that is registered in the EntityManager
• Managed Entity: an instance of a managed class
• Managed Property: any property on your Managed Class that is managed by the EntityManager (Having unmanged properties in your ManagedClass is certainly allowed, but keep in mind that the EntityManager will totally ignore unmanaged properties.)

There are several types of ManagedProperty’s already built, and you are free to create your own to implement novel behavior.

• medryx.orm.Identifier: in this implementation, only numeric identifiers are allowed, and there must be one and only one Identifier per Managed Class
• medryx.orm.Property: used for any property of a Managed Class that is not an association — usually scalar values like numbers, strings, Dates, etc.
• medryx.orm.Association: any association to another Managed Entity. Note that this must be an association to a Managed class, otherwise you use Property above
• medryx.orm.Collection: a collection of Managed Entities. Again, this is inteded to be used for associations to elements of a Managed class only.

The details of each of the ManagedProperty types will be more fully discussed a bit later. Let look at a native mapping for a Patient. You could either create a subclass of patient, or just change your Patient class itself, which is the approach I will show here for simplicity.

dojo.declare("model.Patient", null, {
  //id
  id:new medryx.orm.Identifier(),
 
  //simple properties
  fullName:new medryx.orm.Property(),
  initials:new medryx.orm.Property(),
  medicalRecordNumber:new medryx.orm.Property({readOnly:false}), //this is a mutable field (not in real life, but for this example it is)
  birthdate:new medryx.orm.Property({converter:medryx.orm.converters.DateConverter}),
  gender:new medryx.orm.Property(),
 
  //associations
  visit:new medryx.orm.Association({associationClass:"model.Visit"});
 
  toString:function() {
    return this.fullName || this.getFullName(); //I'll explain this later.    
  },
 
  getAge:function() {
  //snip... uses birthdate to calculate an age
  }
});

and Visit is altered as follows:

dojo.declare("model.Visit", null, {
  //id
  id:null,
 
    //simple properties
  accountNumber:new medryx.orm.Property(),
  admissionDate:new medryx.orm.Property(),
  admissionTime:new medryx.orm.Property(),
  dischargeDate:new medryx.orm.Property(),
    dischargeTime:new medryx.orm.Property(),
  admittingDiagnosis:new medryx.orm.Property(),
    bed:new medryx.orm.Property(),
  room:new medryx.orm.Property(),
  ward:new medryx.orm.Property(),
 
  coverageNotes:new medryx.orm.Property({readOnly:false}),
 
  //associations
  team:new medryx.orm.Association({associationClass:model.Team}),
 
  //collection of insurances
  insurances:new medryx.orm.Collection({associationClass:model.VisitInsurance}),
 
  getLengthOfStay:function() {
    //snip -- calculates number of days in the hospital based on the admission date.
  },
 
  getRoomNumber:function() {
    return this.getWard() + "-" + this.getRoom() + "-" + this.getBed(); //always use getters to allow lazy initialization if needed
  }
});

Okay, so you are almost done with your configuration. (Obviously I’ve left model.Team and model.VisitInsurance out for brevity, but they are just as simple to “annotate”).

The next step is to plug in your PersistenceService. MedryxORM defines a PersistenceService interface that looks like:

dojo.declare("medryx.orm.PersistenceService", medryx.AbstractBase, {
 
    /**
     * the deferred calls back with an object:
     * {
     *    total: total number of records available for this query, or -1 if unknown
     *    items:array of items
     * }
     * returns a deferred (always)
     * 
     * @param {Function | String} entityClass
     * @param {Object} kwArgs (see ReadApi... still need to document queryOptions)
     * @param {Boolean} sync
     */
    $fetch:null,
 
    /**
     * returns a deferred (always) that callsback with 
     * the id if successful
     * 
     * if no id is null or < 0, then this is a put, otherwise it is
     * a post
     * 
     * @param {Function | String} entityClass, 
     * @param {Integer} id, 
     * @param {Object} propertiesToPersist
     */
    $save:null,
 
    /**
         * similar to WriteApi deleteItem
         * 
         * returns a deferred (always)
         * 
         * @param {Function | String} entityClass, 
     * @param {Integer} id
         */
        $deleteItem:null,
 
    /**
     * given an entiyClass, id, and properties, full load an instance 
     * that guarantees that each property in properties is initialized
     * (or if no properties are specified, then some contract
     * between client and server about "default" loading has
     * been fulfilled)
 
     * @param {Function | String} entityClass
     * @param {Integer} id
     * @param {Array of Strings} properties -- this is a list of properties that the 
     * server GUARANTEES will be present in the returned entity. The server *MAY* and 
     * frequently *WILL* return more than just these properties, and the EntityManager 
     * will handle that case appropriately and consume all the information the server sends
     * each item in the array may be either a property name, a dot-notated association property
     * or a *
     * for example: ["*", "visit.accountNumber", "visit.team.*"] would load all the properties
     * of the entityClass with id, the visit association with the account number loaded, and 
     * the team of the visit with ALL of its properties loaded. 
     * @param {Boolean} sync
     */
    $load:null
});

We also provide an RpcPersistenceService that gets rid of sync handling for you. You could easily create an SMD to your server that meets the requirements of this interface.

You provide a default PersistenceService to the EntityManager. If you do not specify a specific PersistenceService for a given Managed Class, then the default PersistenceService is used. However, it may be easier for you to implement a seperate SMD for each Managed Class, so you can supply a seperate PersistenceService for a particular class (or for every class). My server can easily interpret the “entityClass” and determine what to send back to the client, but if you wanted to make your server more generic, you may choose to ignore the entityClass on the server and let that mapping occur on the client. It would be extremely easily to build a REST PersistenceService (and I will probably do that when the need arises) into which you can adapt a Rest service to fit the PersistenceService API.

Let make this a bit less abstract. As I said, my server is pretty smart, so I it knows the right thing to do with entityClass. So I have a single PersistenceService that I specify when I “startup” my EntityManager (i.e. instantiate it).

ormPersistenceServiceSMD = {
  serviceType:"JSON-RPC",
  serviceUrl:"mySmartServer.php",
  methods: [
    {
      name: "load"
      parameters:[
        {name:"entityClass", type:"string"}
        {name:"id", type:"integer"}
        {name:"properties", type:"array"}
      ]
    },
    //snip... you get the idea
  ]
}

Then I can plug that into my EntityManager with a couple of wrappers:

var rpcPersistenceService = new medryx.orm.RpcPersistenceService(new dojo.rpc.JsonService(ormPersistenceServiceSMD));
var entityManager = new medryx.orm.EntityManager(rpcPersistenceService);
 
entityManager.registerClass({entityClass:"model.Patient", alias:"Patient"});
entityManager.registerClass({entityClass:"model.Visit", alias:"Visit"});
 
//and now everything above works!
 
var patient = new Patient(10);
patient.getVisit().setCoverageNotes("Some new coverage notes");
 
patient.save();

Woo-hoo! I obviously have a LOT of work to do in terms of documentation. But this is a start! Please, tell me what you think!!! I’ve posted the source on Google code. Maybe if I get lucky, some Dojo folks will take an interest, and we could be incubated as a dojox project! One can dare to have dreams, right?!

I’ve been thinking a lot about javascript client-side object models lately. I use dojo for my javascript oop tools, so creating classes, and inheritance is not really the problem. The problem is figuring out how to make as transparent as possible a rich relational object model. Then to figure how to do automatic dirty checking and updating.

The needs of a client side object model seem to be slightly different than those on the server. And obviously the client is more resource constrained. So we really only want to get the information we absolutely need for the client to run. But on the other hand, creating a single set of objects that can be used anywhere in our application, would obviously also be of benefit. So how to solve this?

Its all a question of optimization versus lazy-loading. If we use sparsely populated objects, and then lazy load the remaining properties on demand, then perhaps we will have the solution. We will be able to create a single object model, and use it reliably on every page of the application. Then as a performance optimization later — we can make the “laziness” disappear by populating the object less sparsely, or targeting which parts of the object model are loaded intially, and which are loaded lazy.

But if we want to create a system where we are always interacting with an object in the same way, and letting the performance optimizations occur in the background, then we need to define how exactly we are going to interact with the object.

So lets start walking through an example. Consider a Person object:

dojo.declare("medryx.model.Person", null, {
  firstName:null,
  middleName:null,
  lastName:null,
  roles:null
});

Now first things first. There is no way I can think of that we are going to be able to support direct property access of these properties and have any ability to instrument our entity to take care of performance optimization, dirty checking, and so-forth in the background. And since support for javascript property getters and setters is sparse to say the least, I think right off the bat, we are going to be left with using get and set methods for properties, ala javabeans. Don’t worry though, I am not saying you will need to necessarily write all those getters and setters. I’m just saying to access properties reliably, you will need to use them. Javascript is such a flexible language that we can handle creating these getters and setters automagically.

My standard model for instantiating an object is to pass in an associative array of properties and initialize each of the properties automatically. We could use our constructor that initializes the properties also create the getters and setters without to much additional effort. We could event handle “private” properties that do not need getters and setters. If we put this in a base class for all of our entities, we would have:

dojo.declare("medryx.model.Entity", null, {
constructor:function() {
  if (arguments[0]) { //mixin kwArguments if present, which is only valid argument to constructor of an Entity
    dojo.mixin(this, arguments[0]);  
  }
  //setup getters
  for (var key in this){
    if (key[0] !== "_") { //ignore underscore-prefixed properties
      if (!dojo.isFunction(this[key])) { //ignore functions
        this.setupGetter(key);
        this.setupSetter(key);
      }
    }
  }
},
setupSetter:function(key) {
  var setterName = "set" + medryx.util.StringUtils.capFirst(key);
  if (!dojo.isFunction(this[setterName])) { //don't use an auto-getter if a getter is already defined (allows override)
    //assume this is a property and needs a setter
    this[setterName] = dojo.hitch(this, "set", key);
  }
},
setupGetter:function(key) {
  var getterName = "get" + medryx.util.StringUtils.capFirst(key);
  if (!dojo.isFunction(this[getterName])) { //don't use an auto-getter if a getter is already defined (allows override)
    //assume this is a property and needs a setter
    this[getterName] = dojo.hitch(this, "get", key);
  }
},
get:function(property) {
  return this[property];
},
set:function(property, value) {
  this[property] = value;
}
});

and then we change person to inherit from Entity:

 
dojo.declare("medryx.model.Person", medryx.model.Entity, {
  id:null,
  firstName:null,
  middleName:null,
  lastName:null,
  roles:null
});

A quick walk-through of Entity shows that it iterates through all the properties that exist on the entity that do not start with a “_”. For each of these it creates a getter and a setter. It also mixes in a the properties from the initialization argument. Notice also that if a getter already exists, then the default getter is not used. So if you had some business logic you wanted to put into a getter or setter of your own, you
could simply create your own getProperty() method and it would not be over-ridden.

So our first cut is done. Let’s see how it works:

  Person person = new medryx.model.Person({firstName:"Maulin", roles:["super-user", "dev-user"]);
  div.innerHTML = "Hello " + person.getFirstName() + 
    "!" + " I see you have the following roles: " +
    person.getRoles().join(", ") + ".";

Simple enough. So how has this helped us? Well, it hasn’t. Yet. But now lets say that istead of explicitly instantiating the person with values, that we were loading a person via JSON in an ajax call to the server. Something like:

var deferred = loadPeople();
var people;
deferred.addCallback(function(peopleProperties) {
  people = dojo.map(peopleProperties, function(personProperties) {
    return new medryx.model.Person(personProperties);
  });
});

Cool. With very little work, I now have locally cached “people” store that I could easily build some query, sort, and filter functions for. (I’ll talk about that in a later article).

So now lets get back to the idea of lazy loading. Lazy loading can operate in two paradigms — synchronous and asyncrhronous. The advantage of synchronous is that is a very comfortable programming model. You really don’t change anything other than Entity, and you have lazy loading without changing any application code.

As an example, lets say that our call to loadPeople() above return an array of associative arrays that had firstName, middleName, and lastName for each person. But lets say roles was not sent down from the server. Now when we make a call to getRoles(), we get null, which is obviously not what we want. So somehow, getRoles() needs to know that its dealing with a lazy property.

One way would be to say that if a property === null, then try to load it lazily before returning from the getter. But what if the value is supposed to be null? Okay, then how about if we create “fake” value for any property that has the ability to be lazy loaded? Then the system could check if the property has been initialized or not, and if not, it can perform a lazy load.

So lets change our Person class a bit:

dojo.declare("medryx.model.Person", medryx.model.Entity, {
  id:null,
  firstName:null,
  middleName:null,
  lastName:null,
  roles:medyrx.model.Entity.LAZY
});

now that we have marked roles as LAZY, lets change our Entity to account for it:

setupGetter:function(key) {
  var getterName = "get" + medryx.util.StringUtils.capFirst(key);
  if (!dojo.isFunction(this[getterName])) { //don't use an auto-getter if a getter is already defined (allows override)
    //assume this is a property and needs a getter
    //check if this property is lazy
    var isLazy = (this[key] === medryx.model.Entity.LAZY); 
        //note that if this property was already initialized by the mixin in the constructor, that it would no longer be lazy
     if (!isLazy) {
        this[getterName] = dojo.hitch(this, "get", key); //default behavior
     } else {
       if (!dojo.isFunction(this.lazyLoadProperty)) {
        throw new Error("Entities with lazy properties MUST implement lazyLoadProperties");
       }
       this[getterName] = dojo.hitch(this, "getLazy", key, false);
     }
  }
},
getLazy:function(property) {
  if (this[property] === medryx.model.Entity.LAZY) {
    this.lazyLoadProperty(property); //a SYNCHRONOUS CALL that when it returns will have initialzed the property
  }
  return this[property];
}

So this code will check if a property is lazy. If it is, it will shunt the getter over to getLazy(). getLazy() will check if the property has been initialized, and if it has not, then it will ask the subclass to initialize it. Since this would be a synchronous call, the end result would be that a call the getRoles() would return the roles, just as it did before.

Finally, for completeness, lets look at the person object one last time:

dojo.declare("medryx.model.Person", medryx.model.Entity, {
  id:null,
  firstName:null,
  middleName:null,
  lastName:null,
  roles:medyrx.model.Entity.LAZY,
 
  //private:
  _allProperties:null,
 
  lazyLoadProperty(property) {
    if (!this._allProperties) {
      !this._allProperties = medryx.dao.loadCompletePerson(this.id); //SYNCHRONOUSLY laod all the properties for this person
    }
    this[property] = this._allProperties[property];
  }
});

While this implementation works fairly well for simple unininitialized properties, we now need to consider how we would handle related objects. For example, lets give a person an account object.

dojo.declare("medryx.model.Person", medryx.model.Entity, {
  id:null,
  firstName:null,
  middleName:null,
  lastName:null,
  roles:medyrx.model.Entity.LAZY,
  account:null,
 
  //private:
  _allProperties:null,
 
  lazyLoadProperty(property) {
    if (!this._allProperties) {
      !this._allProperties = medryx.dao.loadCompletePerson(this.id); //SYNCHRONOUSLY laod all the properties for this person
    }
    this[property] = this._allProperties[property];
  }
});

When we are initializing the person, we will only get an accountId from the server. We could add some custom work into the Person constructor to instantiate the team:

dojo.declare("medryx.model.Person", medryx.model.Entity, {
  id:null,
  firstName:null,
  middleName:null,
  lastName:null,
  roles:medyrx.model.Entity.LAZY,
  account:null,
 
  //private:
  _allProperties:null,
 
  //constructor:
  constructor:function(args) {
    if (args && args.accountId) {
      this.account = new medryx.model.Account({id:args.accountId, person:this});
    }
  }
 
  lazyLoadProperty(property) {
    if (!this._allProperties) {
      !this._allProperties = medryx.dao.loadCompletePerson(this.id); //SYNCHRONOUSLY laod all the properties for this person
    }
    this[property] = this._allProperties[property];
  }
});

And if account looks like:

dojo.declare("medryx.model.Account", medryx.model.Entity, {
  id:null,
  person:null,
  lastLogin:medryx.model.Entity.LAZY,
 
  lazyLoadProperty(property) {
    //to be implemented with a SYNCHRONOUS loader
  }
});

Then if I want to get the lastLogin I could write:

div.innerHTML = person.getAccount().getLastLogin();

and everything would *just work*! But the devil is always in the details. If you use this technique, you really need to think about how much data to supply to your objects in advance with enough data so that these lazy initializations are kept to a minimum. Synchronous calls to the server are by definition a bad thing.

Or alternatively, we can change our design to a slightly more complicated one, that handles lazy initialization more gracefully with ansynchronous calls. That will be the topic of my next post.

There are plenty of tutorials out there about Dojo widgets, so I won’t write another one. But I would like to document here my philosophy for writing widgets, allowing for separation of concerns and perhaps even an MVC architecture.

In my mind, a widget is a hybrid of view component and model component. It builds a UI, then receives events on that UI that it passes on the model. But this can get messy, especially for testing, debugging, and changes later on down the road. So I propose that we separate the widget into its elemental components, and then only combine them in the “final widget”. Here’s what I mean:

Lets create widget called TeamList, that creates a list of teams. When you click on a team you get a dialog with team details.

/widgets
---/ui
------/tests
---------/all.js
---------/runTests.html
---------/TeamListTest.js
------/_TeamList
---------/TeamList.html
---------/TeamListView.js
---------/TeamListController.js
---------/tundra.css
------TeamList.js
---/themes
------/tundra.css

As you can see, I actually seperate my widgets into several files:

1. TeamList.html: the standard widget template
2. TeamListView.js: a class that DOES NOT derive from the usual _Widget or _Templated, but that “pretends to”.

   This class is used to manipulate everything related to the UI, and to create the events that the widget will generate (or translate the low level user events to more useful ones for the application).

   The key is that none of the “high-level” events will trigger ANYTHING to happen in the model. That is the job of the controller.

   This makes this class quite isolated, and fairly easy to test and change, as long as it adheres to its contract of high-level events that it will trigger.

   Here is the code for my TeamListView:

    
   dojo.declare("medryx.widgets.ui._TeamList.TeamListView", medryx.AbstractBase, {
       //*** attributes
       //filter the teams
       filter:null,
    
       //sort the teams. this can be an array of sort objects (see AbstractDao.sort)
       //or can be a comma-seperated list of fields (simpler, but less flexible)
       sort:null,
       headerContent:"",
   	footerContent:"",
    
       //*** internal
       templatePath: dojo.moduleUrl("medryx","widgets/ui/teamList/teamList.html"),
       controller:null,
       teams:null,
    
       //** nodes defined in the template -- only defined here so we remember them
       list:null,
    
       /**
        *  abstract method
        *  MUST return a valid initialized controller
        */
       $buildController:null,
    
       postCreate:function() {
           this.controller = $buildController();
           this.loadTeams({
               filter: this.filter,
               sort: this.sort
           });
       },
      /**
        * delegate to the controller.
        */
       loadTeams:function() {
           var deferred = this.controller.loadTeams(arguments);
           deferred.addCallback(dojo.hitch(this, this.displayTeams, this.list);
       },
      displayTeams:function(list, teams) {
   	dojo.forEach(teams, function(team) {
   		var li = document.createElement("li");
   		li.innerHTML = team.getName();
   		list.appendChild(li);
   		dojo.connect(list, "onclick", dojo.hitch(this, "teamClicked", team));
   	});
      },
      teamClicked:function(team, event) {
      }

   So lets point out some interesting things about this “View”. Notice that it does not do anything about actually loading teams, or handling what happens once a team gets clicked. It just translates the low level click event to a high level teamClicked event that passes the team of interest to the event.

   Next, notice that while we implement postCreate(), we are not inheriting (yet) from _Widget or _Templated. Here’s why. Lets say I want to test this class. If it had already inherited from these classes, it would automatically call the lifecycle events of the class. But I don’t necessarily want that to happen… yet. I want to call my postCreate method manually and run my tests to be sure everything in this class is working right, before I start wiring things up.

   One other small point — look at displayTeams. Notice how I actually pass the list to which I want to attach the items to this method. Obviously I could use “this.list” throughout displayTeams, and then I wouldn’t have to pass the argument. But stylistically, I find it easier to write tests for methods that are “self contained” and have as little reference to “this.*” as possible. If I hadn’t passed the list as a parameter, I would have had to remember in my tests to set this.list to a new list before running my test. And you can easily forget to do that. In other words, I try to make my widgets as “stateless” as possible in the View. I like to let the final widget inject all the state that is necessary.

   Here is the (very simple) controller class:

   dojo.declare("medryx.widgets.ui._TeamList.TeamListController", null, {
       view:null,
       constructor:function(view) {
           this.view = view;
           dojo.connect(view, "teamClicked", this, "pageTeam");
       },
    
    /**
     * return a deferred that will callback for all teams
     * that are to be display. the arguments are filter and sort
     * either as parameters or as kw arguments
     */
       loadTeams:function() {
           return medryx.dao.TeamsDao.getAll(arguments);
       },
    
       pageTeam:function(team) {
           var pageForm = new medryx.widgets.ui.PageForm();
           pageForm.setTeam(team);
           pageForm.show();
       }
   });

   This is where I connect this widget to other layers of the application. So all interactions outside of the widget should occur through this controller. Notice how I connect up the “teamClicked” event to a “pageTeam” function. Another advantage of separating the view from the controller is that allI have to do is a use a different controller, and I can get a totally different widget. Again, this is a fairly easy class to test .

   So where does this all come together? In medryx.widgets.ui.TeamList, of course!

   dojo.declare("medryx.widgets.ui.TeamList", [dijit._Widget, dijit._Templated, medryx.widgets.ui._TeamList.TeamListView], {
   	$buildController:function() {
                return new medryx.widgets.ui._TeamList.TeamListController	(this);
             }
   });

   All this does is create the actual widget class, mixin the _Widget, _Templated to allow the magic of dojo to take over, and to define the controller that will be used.

In my last post, I talked about how to use Dojo’s DOH for unit testing. But the more I wrote about it and used it, the more I realized that there were a couple of things that made it difficult for me to wrap my head around.

First, setUp and tearDown were not similar to their counterparts in other unit-testing frameworks, such as JUnit.

Secondly, the whole model for programming an asynchronous test seemed a bit confusing.

So I have come up with what seems to me to be a ridiculously simple solution — a custom “test fixture”.

As you recall, when registering a set of tests, you pass the name of the set followed by an array of test methods or test fixtures. The methods are simple function calls. The test fixture had the following properties:

1. name
2. runTest
3. timeout
4. setUp
5. tearDown

In all the examples I found, this was declared as an associative array, setting each of these properties.

So a typical test looked like:

doh.register("some.test.package", [
function testOne() {
},
{
	name:"test2",
	setUp:function() {
		doSomeSetup();
	},
	tearDown:function() {
		doSomeTearDown();
	},
	runTest:function() {
		doh.assertTrue(doSomethingTrue());
	}
},
{
	name:"test3",
	setUp:function() {
		doSomeSetup();
	},
	tearDown:function() {
		doSomeTearDown();
	},
	runTest:function() {
		doh.assertTrue(doSomethingElseTrue());
	}
}
]);

But what if instead we write a class that implements all the above properties? Then we could subclass that class for a particular set of tests that shared a setUp/tearDown series. So in my first cut, I wrote a simple TestFixture class that looks like this:

dojo.declare("medryx.util.test.TestFixture", null, {
	constructor:function(testName, test) {
		this.name = testName;
		this.runTest = test;
	}
});

Okay, so its not rocket science. But already I feel like it cleaned up my tests substantially. Because now I have:

dojo.declare("some.test.fixture", medryx.util.test.TestFixture, {
	setUp:function() {
		doSomeSetup();
	},
	tearDown:function() {
		doSomeTearDown();
	}
});
 
doh.register("some.test.package", [
function testOne() {
},
new some.test.fixture("test2", function() {
	doh.assertTrue(doSomethingTrue());
}),
new some.test.fixture("test3", function() {
	doh.assertTrue(doSomethingElseTrue());
})
]);

Well that’s a start. But that got me thinking. Could I use this to simplify asynchronous tests?

As I discussed in my previous post, there is essentially a template for doing asynchronous testing:

1. Create a doh.Deferred
2. Do your deferred thing
3. Attach the doh.Deferred.getTestCallback(), instantiated with your own assertions, to you deferred thing’s callback/errback
4. Return the doh.Deferred

So I wrote a new and improved fixture:

/**
 * Use this test fixture for asynchronous tests.
 * 
 * @param {Object} testName
 * @param {Object} asyncTest: a method that ALWAYS returns a Deferred (dojo or doh)
 * @param {Object} assertionsCallback: a method that you will use to test your assetions once your defered is done
 */
dojo.declare("medryx.util.test.AsyncTestFixture", null, {
    constructor:function(testName, asyncTest, assertionsCallback) {
        this.name = testName;
        this.asyncTest = asyncTest;
		this.assertionsCallback = assertionsCallback;
    },
	runTest:function() {
		var d = new doh.Deferred();
		var actualDeferred = this.asyncTest();
		actualDeferred.addBoth(d.getTestCallback(dojo.hitch(this, "assertionsCallback")));
		return d;
	}
});

So now my asynchronous tests derive from this AsyncTextFixture and are created as:

	new subclass.of.AsyncTestFixture("testName",  function() {
		var def = doSomethingDeferred();
		return def;
	}, function(result) {
		doh.assertEqual("expected", result);
	});

I just think its easier to “remember” how to do aync testing with this fixture. Thoughts?

I have been spending the last few days experimenting with Dojo’s DOH for unit testing my javascript applications. While it has a relatively steep learning curve, and relatively little documentation, once you figure it out, it is actually quite simple and powerful. But it took me almost a whole day just to get up and running!

There are two great and simple articles about DOH on site pen that I found indispensible in learning DOH. The first talks about the logistics of setting up the tests and running them, and the second talks about how to do tests of asynchronous functions. There is a section of the Book of Dojo dedicated to this as well, but I found it hard to follow.

Setting up your test environment

The first trick to getting DOH running is to understand the “DOH Runner” application. This application lives in your /dojotoolkit/util/doh directory. If you launch it directly, it will run all the dojo unit tests. (This, by the way may be the most confusing part of runner.html. I’ve seen lots of posts about how people can’t figure out how to get runner working. I may work on cleaning this up to make it more intuitive for people to use without any difficult initial setup.) But you want to run your tests? To do this you provide the runner with a querystring argument called testModule that is your package with tests. For example, your url may look like:

../dojotoolkit/util/doh/runner.html?testModule=medryx.dao.tests.all

But that assumes that the module you pass in is a “sister” (i.e. at the same folder level) to dojo, dijit, and dojox. If it is not, then you will need a Dojo 1.2 implementation of runner.html and runner.js. Since that is not available yet, you will have to get a snapshot of the latest nightly build or access these files from subversion. These build in support for registerModulePath.

So lets say you have the following structure:

	/
	---/lib
	------/dojotoolkit
	---------/dojo
	---------/dijit
	---------/dojox
	---------/util
	------/mytoolkit
	---------/medryx

That means that when you call dojo.require(”medryx.something”) you want it to look at a path that dojo does not natively understand. So when loading dojo you have to configure djConfig with modulePaths or by calling dojo.registerModulePath() afer loading dojo. Well if you have this structure, and you are running dojo, then I am sure you know how to do this with dojo. The problem is that runner.html loads and tries to run your tests before you even get a chance to run any of your code. So the runner needs to know about the module paths when it is being called. The solution to this was adding registerModulePath as an option on the querystring for runner, so that now your url would look like:

../dojotoolkit/util/doh/runner.html?testModule=medryx.dao.tests.all&registerModulePath=medryx,../../medryx

So what does

medryx.dao.tests.all

look like? My general practice mirrors that of Dojo — I put a tests folder in every package.
The tests folder has the following content:

1. tests: each group in its own file (roughly a 1:1 number of test files to classes in the package)
2. all.js: a simple file that dojo.requires all the tests above
3. runTests.html file which is a simple meta redirect to runner.html with the appropriate querstring arguments already set. Such as:

   <html>
       <head>
       <title>Dojox Unit Test Runner</title>
       <meta http-equiv="REFRESH" content="0;url=../../../dojotoolkit/util/doh/runner.html?testModule=medryx.dao.tests.all&registerModulePath=medryx,../../medryx"></HEAD>
       <BODY>
           Redirecting to D.O.H runner.
       </BODY>
   </HTML>
   

So now execute runTests.html, and you run all the tests in that package. Hurray! But what does a test look like?

Writing Tests with DOH

To register a set of test methods with DOH to be run, you call doh.register with the name of the test, and an array of the actual tests. The tests in the array can be in one of two forms: a simple function(), or an associative array with the following properties: name, runTest (the test method), setUp, and tearDown — similar to junit. Notice that there is no setUp or tearDown that can be written once and then called from before and after every test (as there is in junit). The setUp and tearDown methods here only apply to the one call to runTest that is contained in that fixture. You can sort of get around this by creating a setUp and tearDown method of your own, as a sparsely populated fixture, and then just fill in name and runTest. For example:

dojo.provide("medryx.dao.tests.TeamsDaoTest");
dojo.require("medryx.dao.TeamsDao");
dojo.require("doh.runner");

dojo.declare("medryx.dao.tests.TeamsDaoTest.fixture", null, {
	constructor(name, runTest) {
		this.name = name;
		this.runTest = runTest;
	}
	setUp:function() {
		//do something before every test you want to run
	},
	tearDown:function() {
		//do something after every test you want to run
	}
});

doh.register("medryx.dao.tests.AbstractDaoTest",
[
	new medryx.dao.tests.TeamsDaoTest.fixture("someTest", function(){
		//do your test
	}),
	new medryx.dao.tests.TeamsDaoTest.fixture("someTest2", function(){
		//do your test2
	})
]);

I personally think that is a more natural use of setUp and tearDown. I mean, if I can only use the setUp and tearDown once, then why whouldn’t I just put it in the runTest method?

So now you have a simple set of synchronous tests. You have a set of assertions supported in doh (though I would like to see more in the future). But everything you need is there: assertTrue, assertFalse, and assertEqual. Note that these DO NOT support a message that is displayed during failure. I think that is a problem — it makes finding the reason for your failed test more work than it has to. I would love to see them extend each of these methods to contain an optional 3rd parameter that is a message that gets displayed when there is a failure.

There is a nifty assertError method that I would like to spend a couple seconds on as well. This assertion takes and expected error (i.e. an Error object — which is another reason to always subclass Error for your own exceptions!) as its first parameter. You then pass a scope, and the name of a function with an array of args, and it will execute the method inside of a try-catch, and if it does not get your expected error, it will fail. This is a little complicated, but here is an example that makes it perhaps a little more simple to understand.

{
	name:"testOneInstance",
	instantiate:function() {
		new medryx.dao.TeamsDao();
	},
	runTest:function() {
	   doh.assertError(medryx.exceptions.SingletonException, this, "instantiate");
	}
},

This will fail if creating a new medryx.dao.TeamsDao() does not return a SingletonException.

Asynchronous Tests

DOH really comes into its own with asynchronous testing. This is certainly not intuitive at first, but after you understand the pattern, it becomes fairly easy to implement. I have actually posted some suggestions to the dojo forum about ways that asynchronous testing could be made simpler with a minimal set of modifications, so we’ll see how that goes. Until then, this is how I suggest you implement asynchronous tests. Before you begin, make sure you really understand Deferred’s!

First, you need to understand the contract you are making with DOH when you run an asynchrounous test. You “mark” a test as asynchronous by having your runTest method return an instance of doh.Deferred. Notice that is NOT dojo.Deferred. DOH has its own Deferred implementation that you must return.

When you return a doh.Deferred from your runTest method, you are agreeing to, at some point run the .callback() or the .errback() methods on the deferred you return. The

d.errback(e)

method should be called with any exceptions that occur in your test, or if your assertions fail. Otherwise, call

d.callback(true)

So to summarize:

Notice how this is VERY different than the usual methodology for running tests in which you do something, and then make some assertions about the outcome, and the assertions themselves handle telling the test-framework whether the test passed or failed.

So an example of implementing an asynchronous test using this method would be: (crediting Dustin Machi)

doh.register("tests.DeferredExample",[
     function myDeferredTest(t){
           var dohDeferred = new doh.Deferred();
           var expectedResult = "foo";         

           var realDeferred = someAsyncFunction();

           realDeferred.addBoth(function(result){
                    if (result==expectedResult) {
                       //our test succeeded, fire the callback
                       dohDeferred.callback(true)
                    }else{
                       dohDeferred.errback(new Error("Unexpected Return: ", result));
                    }
           });

           return dohDeferred;
    },
    .... more test functions
]);

So can you even you assertions in this? Well, not quite. A failed assertion throw an error. If an error occurs in your test you must call errback. So if you surround your assertions in a try-catch block you can use them. So your new test looks like:

doh.register("tests.DeferredExample",[
     function myDeferredTest(t){
           var dohDeferred = new doh.Deferred();
           var expectedResult = "foo";         

           var realDeferred = someAsyncFunction();

           realDeferred.addBoth(function(result){
                    try{
		doh.assertEqual(result, expectedResult);

                       //our test succeeded, fire the callback
                       dohDeferred.callback(true)
                    } catch(e){
                       dohDeferred.errback(new Error("Unexpected Return: ", result));
                    }
           });

           return dohDeferred;
    },
    .... more test functions
]);

Well that is starting to feel a little more natural, but I’m still not quite satisfied. Its still a lot of glue code, and it still feels to hard to write quickly and easily and reliably. Enter the template method pattern.

doh.Deferred contains a method called getTestCallback that is a template for all of the above glue code. All you pass to getTestCallback is a function in which you perform your testing as usual, and the glue code is handled for you. So the above can be simplified to:

doh.register("tests.DeferredExample",[
     function myDeferredTest(t){
           var dohDeferred = new doh.Deferred();
           var expectedResult = "foo";         

           var realDeferred = someAsyncFunction();

           realDeferred.addBoth(dohDeferred.getTestCallback(function(result) {
		   doh.assertEqual(result, expectedResult);
	});

           return dohDeferred;
    },
    .... more test functions
]);

Now that is not too bad! Create the doh.Deferred, run your asynchronous test, and attach the doh.Deferred’s getTestCallback method (containing all of your assertions) to your asynchronous method’s callback and errback handlers. Excellent!

I hope this helps get you started on the wonderful road of javascript unit testing!

A quick look at the magic of dojo.Deferred.

I don’t know why it took me so long to grasp dojo.Deferred. But now that I have spent some time on it, I think I’ll be using it everywhere.

In previous implementations of asynchronous functions I would either pass into the asynchronous call a callback, or I would change the xhr to synchronous. Both are obviously fraught with problems.

Deferred’s operate as filter chain. They therefore REQUIRE that you return a value that will be passed to the next callback. So that means there not 100% safe to pass around willy-nilly, since you may be relying on an unfiltered result. But there is a very easy workaround for that too.

So here in a nutshell is the magic:

1. If you add a callback to a dojo.Deferred, it is guaranteed to get called unless there is an error. This means that regardless if your Deferred is already complete (i.e. your xhr request has already returned), if you add a callback to it 10 minutes later, it will still get called (immediately). That is brilliant, since it means you can code to a simple standard — with callbacks added to the Deferred’s chain.
2. If you have a Deferred that you do not want to allow clients to filter, then don’t pass the deferred to it. Instead, create a dojo.DeferredList that is composed of your “protected” deferred, and then pass that back to the client. They can muck with their return value all they want, and won’t touch your original Deferred. But the any callbacks added to the DeferredList will still be guaranteed to be called when your “protected” deferred returns.
3. One of the challenges of asynchronous programming for me in the past was this infinite propogation of callbacks, that I couldn’t tell what was going on. So I would forget about it, and use synchronous calls, and then switch back to a synchronous model of programming. But that certainly hurts the user experience. With Deferred, the flow of the program is much more evident, and easier to debug.
4. You can actually trigger a callback only when multiple Deferred’s return using a DeferredList. This is slightly different in its syntax, in that you will get a set of return values passed to the callback — all the Deferred items in the original DeferredList, with each result’s location corresponding to their index in the original DeferredList array.

Everybody likes to see some code, so here is an example of the magic:

dojo.require("dojo.DeferredList");

document.write("the first thing you should see is callback1 getting called with content = CONTENT ");

var def = new dojo.Deferred();
def.addCallback(function(content) {
	document.write("callback: 1, content:" + content + "");
	return content + "1";
})
def.callback("CONTENT");

document.write("the next thing you should see is callback2 getting called with content = CONTENT1 ");

def.addCallback(function(content) {
	document.write("callback: 2, content:" + content + "");
	return content + "2";
});

document.write("next, prove that a deferred list will give you the content (content12), but won't let you change the content");

var def2 = new dojo.DeferredList([def]);
def2.addCallback(function(content) {
    document.write("callback:3, content:" + content + "");
    return "SHOULD NEVER SEE THIS";
})

document.write("next, show callback4 with unchanged content (CONTENT12), despite callback3 changing it");

def.addCallback(function(content) {
    document.write("callback:4, content:" + content + "");
    return content + "3";
});

and the result should look like:

the first thing you should see is callback1 getting called with content = CONTENT
callback: 1, content:CONTENT
the next thing you should see is callback2 getting called with content = CONTENT1
callback: 2, content:CONTENT1
next, prove that a deferred list will give you the content (content12), but won’t let you change the content
callback:3, content:true,CONTENT12
next, show callback4 with unchanged content (CONTENT12), despite callback3 changing it
callback:4, content:CONTENT12

In my first installment in this series, I described an overall file structure and configuration scheme that made it easy to develop and deploy javascript web applications. In this installment, I will describe the structure of my javascript library, how it lends itself to a loosely coupled architecture that is easily tested, and how it integrates with Dojo’s DOH unit testing system.

First lets look at the file structure:

As you can see, I have created my own package of javascript files called medryx, which I register using djConfig as I illustrated in part 1. Lets dive into some of the details of the medryx package:

1. The first folder is called config. Anything that can change from deployment-todeployment is abstracted into various config files, that can be swapped out fairly easily. The “master” config is located in medryx/config.js. This file simply uses dojo.require() to build up the list of configuration “constants” that are available to the deployment. Other files need only add dojo.require(”medryx.config”) to get the current values of these constants.

   dojo.provide("medryx.config");
   
   dojo.require("medryx.config.base");
   dojo.require("medryx.config.rpc");
   

   A second benefit this method of declaring constants is that you can use a different set of constants for your tests, a second set for development and usability, and a third set for production. For example, my unit tests first call dojo.require(”medryx.config”), but then they explicity call dojo.require(”medryx.config.mock”), which effectively replaces the rpc related constants with the mock versions.

   Here is an example of what one of the “component” config files looks like:

   dojo.provide("medryx.config.mock");
   dojo.require("medryx.rpc.MockJsonService");
   
   (function() {
       var config = medryx.config;
      config.rpcService = medryx.rpc.MockJsonService;
      config.rpc = {
           userService: config.libUrl + "/mock/MockUserService.js",
         teamService: config.libUrl + "/mock/MockTeamService.js"
       }   
   
   })();
   

   There is a nice trick here for those who haven’t seen this kind of notation before. Notice how we define an anonymous function, and immediately call it? This allows us to avoid having to declare any global variables, or to use an extremely verbose name for every constant. We declare a local variable called “config” that points to medryx.config. Then, all the “config.*” constants, are actually “medryx.config.*”. A second benefit is that when shrinking this code, “config” will be replaced with a shorter variable, wherease if you used the full “medryx.config” no replacement could occur. (Only local variables are replaced during shrinking.)

2. The next folder to pay attention to is the “exceptions” folder. I always use my own exceptions (that extend Error) when throwing an error! Its much easier to debug, than searching for the cause of “Error” or “TypeError”. Especially when all this code is eval’d by javascript. (Yes, I know there are workarounds for the eval problem and lack of line numbers, but believe me, custom exceptions can really save you! And they are a cinch to create. All they really consist of is a declare statement and a custom constructor:

   dojo.provide("medryx.exceptions.UnimplementedAbstractMethodException");
   
   dojo.declare("medryx.exceptions.UnimplementedAbstractMethodException", Error, {
      constructor:function(unimplementedMethod) {
         this.name = "UnimplementedAbstractMethodException: " + unimplementedMethod;
         this.message = "Subclasses must implement: " + unimplementedMethod;
      }
   });
   

3. Finally lets exam the “layers” of my javascript. As you can see I have several packages — dao, model, rpc, and widgets. While there is some dependence between the packages, most of the packages can be tested in isolation by creating mock version of their external dependencies. So lets look more closely at the layers:

   • Model: This layer contains the entities that the application will interact with. They are typically similar to (though not identical to) their server side counterpart (or even more roughly, they correspond each to a table in a database). There can be relational dependence in the model (i.e. a Person can belong to a Team).

     Now I’ve had people tell me this is overkill for javascript. But I think that its not much more effort to build, and makes everything more extensible and more testable. For example, when first building this application, we didn’t ever really care about who the people on a team were, other than displaying their names. So a naive implementation of this might be:

     dojo.declare("medryx.model.Team", null, {
        teamName:null,
        attendingName:null,
        upperLevelName:null,
        internName:null
     });
     

     But then someone comes along and asks us to create a team that can have two interns on it. Hmmm. Well, we could change internName to:

        //...
        internNames:[]
        //...
     

     and then enter an array of intern names. But then you’ll have to remember to change every call to internName and figure out what to do with it. And then what if they tell you some teams have two attendings? Now consider if we used a more fine grained object model right from the start.

     dojo.declare("medryx.model.Person", null, {
        name:null
        role:null
     });
     dojo.declare("medryx.model.Team", null, {
        teamName:null,
        members:[] , // an array of People
        getTeamMemberByRole:function(role) {
           //to-do
        }
     });
     

     Creating a “Person” class was really simple. There are just a couple lines of code. But now consider first how easy it is to extend who can be on a team. And consider if a requirement to display the pager number for each team member comes up. Just add the field to Person, and you are done!

     The most important feature of a model entiy is its constructor, since this is how we will instantiate and populate the model in the next layer — the Dao.

   • Data Access Objects: The Dao layer is “glue code” the connects our raw server-side services to our model. It provides the application with a common mechanism to “query” and obtain instances of entities that it needs, or lists of them.

     Consider again our “Team” example. The TeamsDao provides a getAll() method that will allow for custom filtering and sorting of the list of teams, and a getById() method that will get a team with a given id.

     It is the Dao’s responsibility to query the server and to cache the Team entities for use by the client. If the team had any editable properties, it would be the Dao that would save the object back to the server via its rpc. Obviously some calls into the Dao can be synchronous, but many are not. As a rule, I always return Deferred objects from Dao methods, so my client code does not need to know when to use synchronous access and when to use asynchronous access.

     In previous implementations of this pattern, I put direct xhr requests into the dao. But this “hard-coding” of my interaction with the server proved difficult to unit test, and made my web application much more at the mercy of which server side technology we were using to get the data. Adding in the next layer — the rpc — abstracts the interaction with the server to yet another layer, making this layer easier to test. In addition, we don’t have to create mocks of any of our Dao objects. This means that as we test parts of our application that are dependent upon the Dao, we can rest assured that any bugs we find are not because of a faulty mock dao.

   • Business Objects: The Dao layer is used almost exclusively by the next layer — the service layer, sometimes called the business layer. I use business because the word service is overloaded, and could mean RpcService or Service layer. In addition to interacting with the Dao, the service layer may call upon business methods of the rpc layer.

     This is where the “business” of the application is done — enforcing security, validating data, manipulating data, deciding whether an object should be saved, etc. This is the layer that most frequently communicates with the UI layer above.

   • RPC Services: I talk about my RPC layer extensively in this post. But briefly, this is basically a set of SMD files that are used to create services that interact with the server in some way. Note that this is the ONLY way that this web application communicates with the server!

     As I said in my previous post, I look at the SMD as a “contract” with your server. As long as you both stick to your contract the rest of how you work should be completely independent of one another. There is always some negotiation that has to occur when defining the terms of the contract, but once it is established, it makes development easier for both sides, a known, reliable, and testable deliverable for both sides, and allows for parallell development with neither side blocking the other’s progress in any way.

     If the SMD definitions are the contract, then the unit tests in this layer are its enforcement. By creating a solid set of unit testing of the SMD contract, you can essentially validate (or invalidate) any server implementation of the contract. Server guys want to move from windows with M$-SQL and ASP to J2EE and MySql? Go fo it! Just make sure these tests pass when you are done!

     A small note about how I declare my SMD’s. I declare each SMD as a class (as opposed to an SMD file). This lets me use dojo.require to get the definition. Its probably just a style choice, but it seems easier to me.

   • Widgets: This is the layer that everyone seems to be focused on. But really, as long as the other layers are all working well, then this layer becomes very straightforward to implement. Need to validate some fields in a form widget? Don’t! Just delegate that work to the business layer. You know the business layer works, because it has been thoroughly unit tested.

     I make just about every piece of my web applications into a widget. I use compound widgets if needed to group smaller ones together. But I never want to see any real amount of javascript in my html page. Just markup!

     This is definitely the hardest layer to test. But, its still possible. The piece that becomes more complicated is testing the “look and feel”. This is probably in the realm of usability and acceptance testing realm than true unit testing. Tools that you could use to help could be something like Selenium. More on that in my unit-testing post.

In this article, I’ve covered the most important elements of a layered approach to javascript web applications. Creating layers allows multiple people to simultaneously develop portions of the application without impeding the progress of others, by making simple stubs or mocks of the behavior that will be expected by the surrounding layers. It allows for easier unit testing, and ultimately makes it easier to fulfill ever-changing specifications and requirements. In my next post, we’ll dive into Dojo’s DOH and talk about how we will implement this “easy” unit testing.

A lot of attention is paid to the structure of files and directories for server side web applications, and the use of swappable configuration. Think build scripts and ant. But almost none has been paid to the ideal structure for files that will be used by the client, such as javascript. In the past, javascript was just an afterthought stuck onto web pages. But that is no longer the case. And with today’s complicated javascript driven web applications, we have to start thinking about how to best structure our files to allow for easier development, maintenance, and creation of build scripts.

As you know, I use Dojo as my javascript toolkit of choice. Dojo’s file structure has matured and is now quite conducive to creating your own parallell structure. Their shrinksafe build system is quite flexible, and can easily optimize your script and get it ready for production. I think spending the time up front thinking about file structure and configuration, and giving at least a little thought to what your ultimate build will look like, will pay significant dividends in the long run.

My web application structure looks like this:

A little explanation:

1. Notice the /web/config directory. This is where I setup my djConfig variable. I make sure to make this the first script included on my pages. On a production server, this is just an empty file, since djConfig is baked into the scripts with shrinksafe.The content of this file configues djConfig, and then includes the dojo file and the corresponding “pages” javascript file (see below).

   /**
    * include this file with a SCRIPT tag on each dojo page
    * @author maulin
    */
   djConfig = {
   	isDebug:true,
   	debugAtAllCosts:false,
   	modulePaths:{medryx:"../../medryx"},
   	modulePaths:{pages:"../../pages"},
   	usePlainJson:true //for MOCKS ONLY
   };
   
   var CURRENT_PATH = window.location + "";
   //the path to dojo can be created from the /pages directory
   var PATH_TO_JS = CURRENT_PATH.substring(0, (CURRENT_PATH.indexOf('/pages'))) + "/lib/js";
   
   //now include the dojo.js
   document.write("");
   
   //finally include the page-specific javascript
   document.write("");
   

   This is actually quite magical, and allows for a lot of flexibility without having to touch your html (or your graphic designer’s html for that matter!)

2. The dojotoolkit contains the source folders from Dojo, and contains the dojo, dijit, dojox, and util directories.
3. Next, notice the “medryx” folder that is a sister to dojotoolkit. This is where all of my javascript lives. I’ll describe its configuration in my next post.
4. The js/pages directory is a folder for all page-related javascript. This is where I handle the dojo.require statements, and any specific page related javascript. Each page has a corresponding file in this directory (that is automatically injected into each page).
5. The lib/mock and lib/asp folders are the implementation of my RPC services. The asp folder uses ASP (duh), and the mock uses javascript via my MockJsonService. You could also create a php, or any other script-language implementation of rpc services in these folders. Or, if using J2EE with JSP for the service, the JSP files could live in a jsp folder (Though, if you are using RPC with java, you could avoid this altogether with a JSON-RPC servlet, but that is way beyond the scope of this post!).
6. The /pages directory contains all my html pages. You may need an index.html at / that redirects to your home page in the pages directory.
7. The /themes directory contains the css for the project. The css is setup in two types of files: “theme” css files that use css-includes to include all the appropriate css styles for all the widgets for a particular theme into one file (e.g. tundra.css), and the other that is the specific css for this application, and uses css includes to include the “theme” of choice as well as any page-specific css for this application’s pages.That sounds complicated. Let me illustrate:

   tundra.css

   @import url("../lib/js/dojotoolkit/dojo/resources/dojo.css");
   @import url("../lib/js/dojotoolkit/dijit/themes/tundra/tundra.css");
   @import url("../lib/js/medryx/widgets/themes/tundra/tundra.css");
   @import url("../lib/js/dojotoolkit/dojox/grid/_grid/tundraGrid.css");
   

   site.css

   @import url("tundra.css");
   
   body {
   	/* snip */
   }
   

So this leaves me with a very simple html header for my pages that looks like:

<head>
	<title>My Home Page</title>
	<script type="text/javascript" src="../config/dojo-config.js"></script>
	<link rel="stylesheet" type="text/css" href="../themes/site.css" />
</head>

And with a simple change to my dojo.config.js, I can completely change what happens on the production server versus in development. For example, I can change the dojo loading script tag to my compiled, optimized javascript file created by shrinksafe.

In this post I describe a simple folder structure that leads to easier development, maintenance, and conversion from development to production (or other) environments. I hope you found it helpful. In my next post about this, I will dive little deeper into my “medryx” js folder and describe how it can be used to make unit testing and development consistent and simple!

I’m tired of server side frameworks. There, I’ve said it. Take your pick — Struts, Tapestry, SpringMVC, php, ASP, ASP.NET. No matter which you choose, I’ve always felt like server side scripting to create web pages and flow in web applications seemed like an obvious violation of seperation of concerns.

But aside from the conceptual argument, I’ve always felt like building pages with these languages was more time consuming than it should be. Edit your page, make sure the server is running, make sure the database is running, make sure you are hitting the database in a known state… blah blah blah. It can take minutes to see the smallest of changes.

As I have grown more and more enamored by extreme programming and unit testing, creating a very rapid cycle of test, code, and test again was of utmost importance. And its just hard to do with heavy weight frameworks.

Now don’t get me wrong, obviously you need server side code. But I think server side code is for managing the model and the business logic. Leave the UI to the client!

So, if you are reading this, I am probably preaching to the choir. But I had to get that off my chest. Now, for the real business of this post. With RPC, we have the ability to use the server to simply expose services to the front end. And with dojo rpc, it is very easy to connect to these services in a very agnostic manner.

Okay, so now you can ditch the heavyweight frameworks for doing things they are not very good at, and use them for things they are good at. So I’ve addressed the SoC issue. (There’s loads of information about using JSON RPC out there, so I won’t repeat it all here.) But what about the extreme programming issue? We want to be able to write client side unit tests and develop against them. (I’ll be posting about my use of Dojo DOH for unit testing sometime soon.) This means not having to wait for the server, or worry about the server in any way.

Well, if you write a web application whose only connection to the server is via RPC, couldn’t you just change the type of RPC Service you are using, and create a “Mock” RPC that appears to talk to a server, but doesn’t? Couldn’t it then always return the data you want, in a known state, without ever having to make any changes on the server? And wouldn’t this then mean that you could test this application in a completely disconnected state? YES!

So here’s how I am doing it.

First, we create a data access layer for all communication with the server. This data access layer does little more than enforce some client side business logic, and delegate to an RpcService. For example:

dojo.declare("medryx.dao.AbstractDao", medryx.AbstractBase, {
        _service:null,
   constructor:function() {
      this._service = new medryx.config.rpcService(medryx.rpc.TeamsService);
   },
        getById:function(id) {
                 return this._service.getById(id);
        }
});

(I will address handling all the Deferred objects in a future post).

So this is a reasonable start. I could probably just change my SMD for the service to point to some static files that have a static JSON response object, and just ignore the parameters. For example:

dojo.provide("medryx.rpc.TeamsService");

medryx.rpc.TeamsService = {
   serviceType:"JSON",
   serviceUrl:"someStaticTeamFile.json",
   methods:[
      {
         name: "getById",
         parameters: [{
         name: "id",
         type: "integer"
      }
]
}

“someStaticTeamFile.json” could be simple text file with the json of a simple response to my service request. However, f I get the same response when the id=1, id=2, or id=null, its not going to feel like the app is really working until I connect to a real server. And what should I do if I want to add more methods to the service?

The SMD is really what I see as a CONTRACT with the server-developers. You promise to adhere to it, and they promise to adhere to it, and beyond that you never need to know how they are working, or what technology they are using. And vice-versa. This is especially true I think for the “methods” property of the SMD. So I would like to avoid changing that.

For a while, I thought my only way around this was to create a “mock” service, using server side code that could interpret the post request and return a different static file for each request, depending on the parameters. That too would have been better than hitting a live database, but it still was slower than I wanted.

And then it hit me. What if we mock the JsonService altogether? This mocked service is a static JSON file that contains all the functions defined in the SMD, and accepts the same parameters described in the SMD. Each function returns the object to the service. So lets look at this code, and you’ll get an idea of what I am talking about:

dojo.provide("medryx.rpc.MockJsonService");
dojo.require("dojo.rpc.JsonService");

dojo.declare("medryx.rpc.MockJsonService", [dojo.rpc.JsonService], {

/**
* this overwrites the bind in the original service.
* it loads the page in the service url first.
* the page should have be javascript object
* that has the method that accepts the given
* params and returns the result object
* @param {Object} method
* @param {Object} parameters
* @param {Object} deferredRequestHandler
* @param {Object} url
*/
bind: function(method, parameters, deferredRequestHandler, url){
   var def = dojo.xhrPost({
   url: url||this.serviceUrl,
   timeout: this.timeout,
   handleAs: "json-comment-optional"
   });

        def.addCallbacks(this.resultCallback(deferredRequestHandler, method, parameters), this.errorCallback(deferredRequestHandler));
},

/**
* this will apply the method and params of the request
* to the received object, and then pass the results
* to the callback of the deferredRequestHandler
*
* @param {Object} deferredRequestHandler
*/
resultCallback:function(deferredRequestHandler, method, parameters) {

var tf = dojo.hitch(this,
   function(mock){
      var obj;
      if (dojo.isArray(mock)) {
         mock = mock[0];
      }
                if (!mock[method]) {
         obj = {id:'unknown', error:{code:-1, message:"Method not implemented."}};
      } else {
         obj = mock[method](parameters);
      }
      if(obj.error!=null){
         var err;
         if(typeof obj.error == 'object'){
            err = new Error(obj.error.message);
            err.code = obj.error.code;
            err.error = obj.error.error;
         } else {
            err = new Error(obj.error);
            err.id = obj.id;
            err.errorObject = obj;
         }
         deferredRequestHandler.errback(err);
      } else{
         deferredRequestHandler.callback(this.parseResults(obj));
      }
   });
   return tf;
}

});

So now, I alter my SMD just a bit:

dojo.provide("medryx.rpc.TeamsService");

medryx.rpc.TeamsService = {
   serviceType:"JSON",
   serviceUrl:"MockTeamService.js", //note the change
   methods:[
      {
         name: "getById",
         parameters: [{
            name: "id",
            type: "integer"
         }]
      }
   ]
}

and then create MockTeamService.js as:

[{
getAll: function(){
      var ret = {};
      ret.result = this.teams;
      ret.id = 1;
      return ret;
   },

getById: function(id) {
   var ret = {};
   ret.id = id;
   for(var i in this.teams) {
      if (this.teams[i].id == id) {
         ret.result = this.teams[i];
         break;
      }
   }
   return ret;
},

teams: [{
   id: 18,
   name: "Adult Triage",
   color: "#C5C3C3",
   abbreviation: "T",
   category: "Triage Teams",
   categorySortIndex: "0",
   pagers: [{
      type: "Attending",
      id: "4900",
      number: "216-6338 x4900"
   }]
}
//... snip... add as many teams as you want here
]
}]

The final piece is making some generic configurations that can be swapped in and out for dev versus prod:

dojo.provide("medryx.config.mock");
dojo.require("medryx.rpc.MockJsonService");

(function() {
   var config = medryx.config;
   config.rpcService = medryx.rpc.MockJsonService;
   config.rpc = {
      userService: config.libUrl + "/mock/MockUserService.js",
      teamService: config.libUrl + "/mock/MockTeamService.js"
   }

})();

and last but not least, I change the “hardwired” information to these config variables:

In TeamsDao change

this._service = dojo.rpc.JsonService()

to

this._service= new medryx.config.rpcService()

and change the serviceUrl of the SMD to medryx.config.teamService from its previous hardcoded “MockTeamService.js”.

So what have we accomplished? Now that all the glue code has been written, each time we want to create or add to a new service, we simply add to or create more “Mock” services that implement the SMD contract. Then we just use them as we would a dynamically loaded server side service. And when you want to switch to the server-side service, simply change you config file (or better yet, swap in the new one).

Questions? Comments? Let me know!

One thing that is clearly missing from javascript is a clear way to distinguish private and public methods within a class. Convention dictates that methods and properties beginning with an underscore are “private”, and others are “public”. And this works fairly well.

But recently, I have been building a very complicated web application, and I have really started to miss abstract classes, interfaces, and protected methods. Dojo has addressed many of these issues with its dojo.declare() syntax. With it, you can mixin an interface, or a base class when creating a new class (or more than one for that matter).

But there still isn’t any real support for abstract classes and abstract methods. As you know, abstract methods must be implemented by subclasses. In addition, there is no support for protected methods. if underscore is private, and no underscore is public, what is a protected method that should only be called or implemented by subclasses?

First, we need to decide how to indicate a method is abstract/protected. I’ve decided a reasonable approach is to prefix the method name with a $. Its unobtrusive, valid javascript, and is far prettier than “protected_” or something like that.

So now we have a way to mark methods that are required to be implemented in a subclass. For example:

dojo.declare("medryx.dao.AbstractDao", medryx.AbstractBase, {
   /**
    * abstract - the entity managed by this dao
    */
   $getEntityClass:null,
 
   /**
    *abstract - the smd of a service that is used by this dao
    */
   $getService:null
});

Notice that these abstract methods are prefixed with “$”, and assigned to null. This is key. The subclasses should implement them as methods as:

dojo.declare("medryx.dao.TeamsDao", medryx.dao.AbstractDao, {
   $getEntityClass:function() {
      return medryx.model.Team;
   },
   $getService:function() {
      return medryx.rpc.TeamService;
   }
})

Now obviusly, we could just stop there, and adhere to this convention. But errors that are borne out of not implementing these methods can be hard to find, or come out quite late… only after trying to use these methods. To address this problem I have built a base abstract class that any abstract class can inherit from (or mixin). This AbstractBase class will check the class in its constructor, and be sure all $ methods have been implemented. If they haven’t, it will throw an exception — in the constructor. Which of course means that the moment you instantiate the class, the error will occur, and will be quite descriptive of the problem. Its not quite as good as “compile-time” checking, but its closer.

Here is the code:

dojo.provide("medryx.AbstractBase");
dojo.require("medryx.exceptions.UnimplementedAbstractMethodException");
dojo.declare("medryx.AbstractBase", null, {
   constructor:function(args) {
      for (var methodName in this) {
         if (methodName.charAt(0) === '$') {
            if (!dojo.isFunction(this[methodName])) {
               throw new medryx.exceptions.UnimplementedAbstractMethodException(methodName);
            }
         }
      }
   }
});

So what are your thoughts? I think the runtime overhead of this is likely to be fairly small. But honestly, this is mostly code you need at development time. You could conceivably completely remove this constructor for production code, since presumably you have ferreted out bugs like this long before production. Heck, it could even be part of your build script!