Medryx Observations

I’ve been thinking a lot about javascript client-side object models lately. I use dojo for my javascript oop tools, so creating classes, and inheritance is not really the problem. The problem is figuring out how to make as transparent as possible a rich relational object model. Then to figure how to do automatic dirty checking and updating.

The needs of a client side object model seem to be slightly different than those on the server. And obviously the client is more resource constrained. So we really only want to get the information we absolutely need for the client to run. But on the other hand, creating a single set of objects that can be used anywhere in our application, would obviously also be of benefit. So how to solve this?

Its all a question of optimization versus lazy-loading. If we use sparsely populated objects, and then lazy load the remaining properties on demand, then perhaps we will have the solution. We will be able to create a single object model, and use it reliably on every page of the application. Then as a performance optimization later — we can make the “laziness” disappear by populating the object less sparsely, or targeting which parts of the object model are loaded intially, and which are loaded lazy.

But if we want to create a system where we are always interacting with an object in the same way, and letting the performance optimizations occur in the background, then we need to define how exactly we are going to interact with the object.

So lets start walking through an example. Consider a Person object:

dojo.declare("medryx.model.Person", null, {
  firstName:null,
  middleName:null,
  lastName:null,
  roles:null
});

Now first things first. There is no way I can think of that we are going to be able to support direct property access of these properties and have any ability to instrument our entity to take care of performance optimization, dirty checking, and so-forth in the background. And since support for javascript property getters and setters is sparse to say the least, I think right off the bat, we are going to be left with using get and set methods for properties, ala javabeans. Don’t worry though, I am not saying you will need to necessarily write all those getters and setters. I’m just saying to access properties reliably, you will need to use them. Javascript is such a flexible language that we can handle creating these getters and setters automagically.

My standard model for instantiating an object is to pass in an associative array of properties and initialize each of the properties automatically. We could use our constructor that initializes the properties also create the getters and setters without to much additional effort. We could event handle “private” properties that do not need getters and setters. If we put this in a base class for all of our entities, we would have:

dojo.declare("medryx.model.Entity", null, {
constructor:function() {
  if (arguments[0]) { //mixin kwArguments if present, which is only valid argument to constructor of an Entity
    dojo.mixin(this, arguments[0]);  
  }
  //setup getters
  for (var key in this){
    if (key[0] !== "_") { //ignore underscore-prefixed properties
      if (!dojo.isFunction(this[key])) { //ignore functions
        this.setupGetter(key);
        this.setupSetter(key);
      }
    }
  }
},
setupSetter:function(key) {
  var setterName = "set" + medryx.util.StringUtils.capFirst(key);
  if (!dojo.isFunction(this[setterName])) { //don't use an auto-getter if a getter is already defined (allows override)
    //assume this is a property and needs a setter
    this[setterName] = dojo.hitch(this, "set", key);
  }
},
setupGetter:function(key) {
  var getterName = "get" + medryx.util.StringUtils.capFirst(key);
  if (!dojo.isFunction(this[getterName])) { //don't use an auto-getter if a getter is already defined (allows override)
    //assume this is a property and needs a setter
    this[getterName] = dojo.hitch(this, "get", key);
  }
},
get:function(property) {
  return this[property];
},
set:function(property, value) {
  this[property] = value;
}
});

and then we change person to inherit from Entity:

 
dojo.declare("medryx.model.Person", medryx.model.Entity, {
  id:null,
  firstName:null,
  middleName:null,
  lastName:null,
  roles:null
});

A quick walk-through of Entity shows that it iterates through all the properties that exist on the entity that do not start with a “_”. For each of these it creates a getter and a setter. It also mixes in a the properties from the initialization argument. Notice also that if a getter already exists, then the default getter is not used. So if you had some business logic you wanted to put into a getter or setter of your own, you
could simply create your own getProperty() method and it would not be over-ridden.

So our first cut is done. Let’s see how it works:

  Person person = new medryx.model.Person({firstName:"Maulin", roles:["super-user", "dev-user"]);
  div.innerHTML = "Hello " + person.getFirstName() + 
    "!" + " I see you have the following roles: " +
    person.getRoles().join(", ") + ".";

Simple enough. So how has this helped us? Well, it hasn’t. Yet. But now lets say that istead of explicitly instantiating the person with values, that we were loading a person via JSON in an ajax call to the server. Something like:

var deferred = loadPeople();
var people;
deferred.addCallback(function(peopleProperties) {
  people = dojo.map(peopleProperties, function(personProperties) {
    return new medryx.model.Person(personProperties);
  });
});

Cool. With very little work, I now have locally cached “people” store that I could easily build some query, sort, and filter functions for. (I’ll talk about that in a later article).

So now lets get back to the idea of lazy loading. Lazy loading can operate in two paradigms — synchronous and asyncrhronous. The advantage of synchronous is that is a very comfortable programming model. You really don’t change anything other than Entity, and you have lazy loading without changing any application code.

As an example, lets say that our call to loadPeople() above return an array of associative arrays that had firstName, middleName, and lastName for each person. But lets say roles was not sent down from the server. Now when we make a call to getRoles(), we get null, which is obviously not what we want. So somehow, getRoles() needs to know that its dealing with a lazy property.

One way would be to say that if a property === null, then try to load it lazily before returning from the getter. But what if the value is supposed to be null? Okay, then how about if we create “fake” value for any property that has the ability to be lazy loaded? Then the system could check if the property has been initialized or not, and if not, it can perform a lazy load.

So lets change our Person class a bit:

dojo.declare("medryx.model.Person", medryx.model.Entity, {
  id:null,
  firstName:null,
  middleName:null,
  lastName:null,
  roles:medyrx.model.Entity.LAZY
});

now that we have marked roles as LAZY, lets change our Entity to account for it:

setupGetter:function(key) {
  var getterName = "get" + medryx.util.StringUtils.capFirst(key);
  if (!dojo.isFunction(this[getterName])) { //don't use an auto-getter if a getter is already defined (allows override)
    //assume this is a property and needs a getter
    //check if this property is lazy
    var isLazy = (this[key] === medryx.model.Entity.LAZY); 
        //note that if this property was already initialized by the mixin in the constructor, that it would no longer be lazy
     if (!isLazy) {
        this[getterName] = dojo.hitch(this, "get", key); //default behavior
     } else {
       if (!dojo.isFunction(this.lazyLoadProperty)) {
        throw new Error("Entities with lazy properties MUST implement lazyLoadProperties");
       }
       this[getterName] = dojo.hitch(this, "getLazy", key, false);
     }
  }
},
getLazy:function(property) {
  if (this[property] === medryx.model.Entity.LAZY) {
    this.lazyLoadProperty(property); //a SYNCHRONOUS CALL that when it returns will have initialzed the property
  }
  return this[property];
}

So this code will check if a property is lazy. If it is, it will shunt the getter over to getLazy(). getLazy() will check if the property has been initialized, and if it has not, then it will ask the subclass to initialize it. Since this would be a synchronous call, the end result would be that a call the getRoles() would return the roles, just as it did before.

Finally, for completeness, lets look at the person object one last time:

dojo.declare("medryx.model.Person", medryx.model.Entity, {
  id:null,
  firstName:null,
  middleName:null,
  lastName:null,
  roles:medyrx.model.Entity.LAZY,
 
  //private:
  _allProperties:null,
 
  lazyLoadProperty(property) {
    if (!this._allProperties) {
      !this._allProperties = medryx.dao.loadCompletePerson(this.id); //SYNCHRONOUSLY laod all the properties for this person
    }
    this[property] = this._allProperties[property];
  }
});

While this implementation works fairly well for simple unininitialized properties, we now need to consider how we would handle related objects. For example, lets give a person an account object.

dojo.declare("medryx.model.Person", medryx.model.Entity, {
  id:null,
  firstName:null,
  middleName:null,
  lastName:null,
  roles:medyrx.model.Entity.LAZY,
  account:null,
 
  //private:
  _allProperties:null,
 
  lazyLoadProperty(property) {
    if (!this._allProperties) {
      !this._allProperties = medryx.dao.loadCompletePerson(this.id); //SYNCHRONOUSLY laod all the properties for this person
    }
    this[property] = this._allProperties[property];
  }
});

When we are initializing the person, we will only get an accountId from the server. We could add some custom work into the Person constructor to instantiate the team:

dojo.declare("medryx.model.Person", medryx.model.Entity, {
  id:null,
  firstName:null,
  middleName:null,
  lastName:null,
  roles:medyrx.model.Entity.LAZY,
  account:null,
 
  //private:
  _allProperties:null,
 
  //constructor:
  constructor:function(args) {
    if (args && args.accountId) {
      this.account = new medryx.model.Account({id:args.accountId, person:this});
    }
  }
 
  lazyLoadProperty(property) {
    if (!this._allProperties) {
      !this._allProperties = medryx.dao.loadCompletePerson(this.id); //SYNCHRONOUSLY laod all the properties for this person
    }
    this[property] = this._allProperties[property];
  }
});

And if account looks like:

dojo.declare("medryx.model.Account", medryx.model.Entity, {
  id:null,
  person:null,
  lastLogin:medryx.model.Entity.LAZY,
 
  lazyLoadProperty(property) {
    //to be implemented with a SYNCHRONOUS loader
  }
});

Then if I want to get the lastLogin I could write:

div.innerHTML = person.getAccount().getLastLogin();

and everything would *just work*! But the devil is always in the details. If you use this technique, you really need to think about how much data to supply to your objects in advance with enough data so that these lazy initializations are kept to a minimum. Synchronous calls to the server are by definition a bad thing.

Or alternatively, we can change our design to a slightly more complicated one, that handles lazy initialization more gracefully with ansynchronous calls. That will be the topic of my next post.

A quick look at the magic of dojo.Deferred.

I don’t know why it took me so long to grasp dojo.Deferred. But now that I have spent some time on it, I think I’ll be using it everywhere.

In previous implementations of asynchronous functions I would either pass into the asynchronous call a callback, or I would change the xhr to synchronous. Both are obviously fraught with problems.

Deferred’s operate as filter chain. They therefore REQUIRE that you return a value that will be passed to the next callback. So that means there not 100% safe to pass around willy-nilly, since you may be relying on an unfiltered result. But there is a very easy workaround for that too.

So here in a nutshell is the magic:

1. If you add a callback to a dojo.Deferred, it is guaranteed to get called unless there is an error. This means that regardless if your Deferred is already complete (i.e. your xhr request has already returned), if you add a callback to it 10 minutes later, it will still get called (immediately). That is brilliant, since it means you can code to a simple standard — with callbacks added to the Deferred’s chain.
2. If you have a Deferred that you do not want to allow clients to filter, then don’t pass the deferred to it. Instead, create a dojo.DeferredList that is composed of your “protected” deferred, and then pass that back to the client. They can muck with their return value all they want, and won’t touch your original Deferred. But the any callbacks added to the DeferredList will still be guaranteed to be called when your “protected” deferred returns.
3. One of the challenges of asynchronous programming for me in the past was this infinite propogation of callbacks, that I couldn’t tell what was going on. So I would forget about it, and use synchronous calls, and then switch back to a synchronous model of programming. But that certainly hurts the user experience. With Deferred, the flow of the program is much more evident, and easier to debug.
4. You can actually trigger a callback only when multiple Deferred’s return using a DeferredList. This is slightly different in its syntax, in that you will get a set of return values passed to the callback — all the Deferred items in the original DeferredList, with each result’s location corresponding to their index in the original DeferredList array.

Everybody likes to see some code, so here is an example of the magic:

dojo.require("dojo.DeferredList");

document.write("the first thing you should see is callback1 getting called with content = CONTENT ");

var def = new dojo.Deferred();
def.addCallback(function(content) {
	document.write("callback: 1, content:" + content + "");
	return content + "1";
})
def.callback("CONTENT");

document.write("the next thing you should see is callback2 getting called with content = CONTENT1 ");

def.addCallback(function(content) {
	document.write("callback: 2, content:" + content + "");
	return content + "2";
});

document.write("next, prove that a deferred list will give you the content (content12), but won't let you change the content");

var def2 = new dojo.DeferredList([def]);
def2.addCallback(function(content) {
    document.write("callback:3, content:" + content + "");
    return "SHOULD NEVER SEE THIS";
})

document.write("next, show callback4 with unchanged content (CONTENT12), despite callback3 changing it");

def.addCallback(function(content) {
    document.write("callback:4, content:" + content + "");
    return content + "3";
});

and the result should look like:

the first thing you should see is callback1 getting called with content = CONTENT
callback: 1, content:CONTENT
the next thing you should see is callback2 getting called with content = CONTENT1
callback: 2, content:CONTENT1
next, prove that a deferred list will give you the content (content12), but won’t let you change the content
callback:3, content:true,CONTENT12
next, show callback4 with unchanged content (CONTENT12), despite callback3 changing it
callback:4, content:CONTENT12