Medryx Observations

July 24, 2008

JsonRestStore retires MedryxORM

Filed under: MedryxORM, dojo — Tags: , , , — maulin @ 5:46 am

When I was first designing the MedryxORM, I had a need for a data store that could perform advanced functions that no store in Dojo 1.1 had available. However with the addition and significant improvement of dojox.data.JsonRestStore, I am happy to say, that I think, after only one month, I can safely retire MedryxORM. This is a good thing. I always opt for off-the-shelf over custom solutions. And, you get the whole dojo communtiy supporting you when you use an official dojo.data store.

In the last month, I’ve worked with Kris Zyp — the Json guru at Dojo — and I really think the JsonRestStore has come a long way to solving many problems with complicated client side data management. First off, let me make this very clear — despite its name, JsonRestStore does not require you to use a RESTful data source or web service. Any web service will do — RPC, or home-grown. You just plug in your services into the JsonRestStore, and the store handles the rest. I actually think its name is its biggest flaw. Perhaps it should simply be JsonStore, so that more people would use it… but I digress.

So here are the features of JsonRestStore that have allowed my to say goodbye to the EntityManager and MedryxORM:

  1. Unique Object Identity: One of my requirements was that if the server sent down an entity, I wanted to be sure that however I accessed that entity, that I got the same entity. So whether I used store.getValue() or item.foo, or item.getFoo(), or item.getBar().getFoo(), I always want the same entity. In my design, there was therefore one store that handled all kinds of entities. This proved a bit clunky, because to access anything from the store via fetch, you had to provide an “entityClass”. JsonRestStore accomplishes this same goal without the clunkiness. With JsonRestStore you create one store for each entityClass. Behind the scenes, JsonRestStore uses dojox.rpc.JsonRest to be sure object identity is maintained. This means that you need a store for each entityClass, but that makes it crystal clear what you are querying for when you do a fetch. Overall, a better design IMHO.
  2. Conversion of JSON hashes to first-class JS objects, allowing for custom functions and business logic to be implemented within entities: If I have an entity, I want to, for example be able to create a toString() method on it, and each time an entity of that type created, it should inherit the toString(). The server should not worry about javascript methods, so sending it in the native JSON data is not the right solution for lots of reasons. My EntityManager used the “class metadata” to determine the class of information being retrieved from the server, and instantiating each object from the metadata, and then mixing in the server data into the object instance. JsonRestStore has a similar “class metadata” concept, but it is more standardized — and based on JsonSchema. If you provide a schema, and it has “prototype” property, then any methods/fields in the prototype are used when the entity is instantiated. Since standardization is a good thing, and since JsonSchema is emerging in the JS world as a standard, I think this is probably a better approach than the custom metadata that was created for MedryxORM — despite the fact that it was loosely based on EJB3/Hibernate.
  3. Built-in laziness: Use store.getValue() and you lazy load items for free. There is a little bit of learning on how to handle laziness on the server side, but its pretty straightforward (and I hope to write a post about it soon).
  4. Client side filtering/sorting: JsonRestStore allows you to cache query results on the client and then filter them and sort them seamlessly on the client. This safely retires my “named queries”, and is both more efficient and more extensible.

Honestly, the only thing missing from JsonRestStore is de-novo support for javabean style getters and setters that also give you free laziness, and support for dot-path notation in getValue(). I’ve added this with a simple additional class called BeanStore that extends JsonRestStore, and I have submitted BeanStore to Dojo to see if they think it is broadly applicable enough to include in dojox.data. We’ll see.

I have to say, the most irritating part of all of this is that I lose 2 weeks of work. But I can always look back at how intricately I now understand dojo.data and JsonRestStore and know the time was not completely wasted. In my next post, I hope to share with you my understanding of JsonRestStore and its use.

June 9, 2008

A Javascript Object Model and Lazy Initialization

Filed under: dojo, javascript, oop — Tags: , , , , — maulin @ 9:06 pm

I’ve been thinking a lot about javascript client-side object models lately. I use dojo for my javascript oop tools, so creating classes, and inheritance is not really the problem. The problem is figuring out how to make as transparent as possible a rich relational object model. Then to figure how to do automatic dirty checking and updating.

The needs of a client side object model seem to be slightly different than those on the server. And obviously the client is more resource constrained. So we really only want to get the information we absolutely need for the client to run. But on the other hand, creating a single set of objects that can be used anywhere in our application, would obviously also be of benefit. So how to solve this?

Its all a question of optimization versus lazy-loading. If we use sparsely populated objects, and then lazy load the remaining properties on demand, then perhaps we will have the solution. We will be able to create a single object model, and use it reliably on every page of the application. Then as a performance optimization later — we can make the “laziness” disappear by populating the object less sparsely, or targeting which parts of the object model are loaded intially, and which are loaded lazy.

But if we want to create a system where we are always interacting with an object in the same way, and letting the performance optimizations occur in the background, then we need to define how exactly we are going to interact with the object.

So lets start walking through an example. Consider a Person object:

dojo.declare("medryx.model.Person", null, {
  firstName:null,
  middleName:null,
  lastName:null,
  roles:null
});

Now first things first. There is no way I can think of that we are going to be able to support direct property access of these properties and have any ability to instrument our entity to take care of performance optimization, dirty checking, and so-forth in the background. And since support for javascript property getters and setters is sparse to say the least, I think right off the bat, we are going to be left with using get and set methods for properties, ala javabeans. Don’t worry though, I am not saying you will need to necessarily write all those getters and setters. I’m just saying to access properties reliably, you will need to use them. Javascript is such a flexible language that we can handle creating these getters and setters automagically.

My standard model for instantiating an object is to pass in an associative array of properties and initialize each of the properties automatically. We could use our constructor that initializes the properties also create the getters and setters without to much additional effort. We could event handle “private” properties that do not need getters and setters. If we put this in a base class for all of our entities, we would have:

dojo.declare("medryx.model.Entity", null, {
constructor:function() {
  if (arguments[0]) { //mixin kwArguments if present, which is only valid argument to constructor of an Entity
    dojo.mixin(this, arguments[0]);  
  }
  //setup getters
  for (var key in this){
    if (key[0] !== "_") { //ignore underscore-prefixed properties
      if (!dojo.isFunction(this[key])) { //ignore functions
        this.setupGetter(key);
        this.setupSetter(key);
      }
    }
  }
},
setupSetter:function(key) {
  var setterName = "set" + medryx.util.StringUtils.capFirst(key);
  if (!dojo.isFunction(this[setterName])) { //don't use an auto-getter if a getter is already defined (allows override)
    //assume this is a property and needs a setter
    this[setterName] = dojo.hitch(this, "set", key);
  }
},
setupGetter:function(key) {
  var getterName = "get" + medryx.util.StringUtils.capFirst(key);
  if (!dojo.isFunction(this[getterName])) { //don't use an auto-getter if a getter is already defined (allows override)
    //assume this is a property and needs a setter
    this[getterName] = dojo.hitch(this, "get", key);
  }
},
get:function(property) {
  return this[property];
},
set:function(property, value) {
  this[property] = value;
}
});

and then we change person to inherit from Entity:

 
dojo.declare("medryx.model.Person", medryx.model.Entity, {
  id:null,
  firstName:null,
  middleName:null,
  lastName:null,
  roles:null
});

A quick walk-through of Entity shows that it iterates through all the properties that exist on the entity that do not start with a “_”. For each of these it creates a getter and a setter. It also mixes in a the properties from the initialization argument. Notice also that if a getter already exists, then the default getter is not used. So if you had some business logic you wanted to put into a getter or setter of your own, you
could simply create your own getProperty() method and it would not be over-ridden.

So our first cut is done. Let’s see how it works:

  Person person = new medryx.model.Person({firstName:"Maulin", roles:["super-user", "dev-user"]);
  div.innerHTML = "Hello " + person.getFirstName() + 
    "!" + " I see you have the following roles: " +
    person.getRoles().join(", ") + ".";

Simple enough. So how has this helped us? Well, it hasn’t. Yet. But now lets say that istead of explicitly instantiating the person with values, that we were loading a person via JSON in an ajax call to the server. Something like:

var deferred = loadPeople();
var people;
deferred.addCallback(function(peopleProperties) {
  people = dojo.map(peopleProperties, function(personProperties) {
    return new medryx.model.Person(personProperties);
  });
});

Cool. With very little work, I now have locally cached “people” store that I could easily build some query, sort, and filter functions for. (I’ll talk about that in a later article).

So now lets get back to the idea of lazy loading. Lazy loading can operate in two paradigms — synchronous and asyncrhronous. The advantage of synchronous is that is a very comfortable programming model. You really don’t change anything other than Entity, and you have lazy loading without changing any application code.

As an example, lets say that our call to loadPeople() above return an array of associative arrays that had firstName, middleName, and lastName for each person. But lets say roles was not sent down from the server. Now when we make a call to getRoles(), we get null, which is obviously not what we want. So somehow, getRoles() needs to know that its dealing with a lazy property.

One way would be to say that if a property === null, then try to load it lazily before returning from the getter. But what if the value is supposed to be null? Okay, then how about if we create “fake” value for any property that has the ability to be lazy loaded? Then the system could check if the property has been initialized or not, and if not, it can perform a lazy load.

So lets change our Person class a bit:

dojo.declare("medryx.model.Person", medryx.model.Entity, {
  id:null,
  firstName:null,
  middleName:null,
  lastName:null,
  roles:medyrx.model.Entity.LAZY
});

now that we have marked roles as LAZY, lets change our Entity to account for it:

setupGetter:function(key) {
  var getterName = "get" + medryx.util.StringUtils.capFirst(key);
  if (!dojo.isFunction(this[getterName])) { //don't use an auto-getter if a getter is already defined (allows override)
    //assume this is a property and needs a getter
    //check if this property is lazy
    var isLazy = (this[key] === medryx.model.Entity.LAZY); 
        //note that if this property was already initialized by the mixin in the constructor, that it would no longer be lazy
     if (!isLazy) {
        this[getterName] = dojo.hitch(this, "get", key); //default behavior
     } else {
       if (!dojo.isFunction(this.lazyLoadProperty)) {
        throw new Error("Entities with lazy properties MUST implement lazyLoadProperties");
       }
       this[getterName] = dojo.hitch(this, "getLazy", key, false);
     }
  }
},
getLazy:function(property) {
  if (this[property] === medryx.model.Entity.LAZY) {
    this.lazyLoadProperty(property); //a SYNCHRONOUS CALL that when it returns will have initialzed the property
  }
  return this[property];
}

So this code will check if a property is lazy. If it is, it will shunt the getter over to getLazy(). getLazy() will check if the property has been initialized, and if it has not, then it will ask the subclass to initialize it. Since this would be a synchronous call, the end result would be that a call the getRoles() would return the roles, just as it did before.

Finally, for completeness, lets look at the person object one last time:

dojo.declare("medryx.model.Person", medryx.model.Entity, {
  id:null,
  firstName:null,
  middleName:null,
  lastName:null,
  roles:medyrx.model.Entity.LAZY,
 
  //private:
  _allProperties:null,
 
  lazyLoadProperty(property) {
    if (!this._allProperties) {
      !this._allProperties = medryx.dao.loadCompletePerson(this.id); //SYNCHRONOUSLY laod all the properties for this person
    }
    this[property] = this._allProperties[property];
  }
});

While this implementation works fairly well for simple unininitialized properties, we now need to consider how we would handle related objects. For example, lets give a person an account object.

dojo.declare("medryx.model.Person", medryx.model.Entity, {
  id:null,
  firstName:null,
  middleName:null,
  lastName:null,
  roles:medyrx.model.Entity.LAZY,
  account:null,
 
  //private:
  _allProperties:null,
 
  lazyLoadProperty(property) {
    if (!this._allProperties) {
      !this._allProperties = medryx.dao.loadCompletePerson(this.id); //SYNCHRONOUSLY laod all the properties for this person
    }
    this[property] = this._allProperties[property];
  }
});

When we are initializing the person, we will only get an accountId from the server. We could add some custom work into the Person constructor to instantiate the team:

dojo.declare("medryx.model.Person", medryx.model.Entity, {
  id:null,
  firstName:null,
  middleName:null,
  lastName:null,
  roles:medyrx.model.Entity.LAZY,
  account:null,
 
  //private:
  _allProperties:null,
 
  //constructor:
  constructor:function(args) {
    if (args && args.accountId) {
      this.account = new medryx.model.Account({id:args.accountId, person:this});
    }
  }
 
  lazyLoadProperty(property) {
    if (!this._allProperties) {
      !this._allProperties = medryx.dao.loadCompletePerson(this.id); //SYNCHRONOUSLY laod all the properties for this person
    }
    this[property] = this._allProperties[property];
  }
});

And if account looks like:

dojo.declare("medryx.model.Account", medryx.model.Entity, {
  id:null,
  person:null,
  lastLogin:medryx.model.Entity.LAZY,
 
  lazyLoadProperty(property) {
    //to be implemented with a SYNCHRONOUS loader
  }
});

Then if I want to get the lastLogin I could write:

div.innerHTML = person.getAccount().getLastLogin();

and everything would *just work*! But the devil is always in the details. If you use this technique, you really need to think about how much data to supply to your objects in advance with enough data so that these lazy initializations are kept to a minimum. Synchronous calls to the server are by definition a bad thing.

Or alternatively, we can change our design to a slightly more complicated one, that handles lazy initialization more gracefully with ansynchronous calls. That will be the topic of my next post.

Powered by WordPress