Medryx Observations

A while back, I sent a post to the Dojo discussion forum regarding lazy loading and relationships between entities. The very simple response by Alex:

stores don’t “do” references…

Aha! That was the part I hadn’t figured out. Ever try to open a phillips-head screw with a flat-head screwdriver? It felt a bit like that. Over time, I came up with many solutions, but never a comprehensive one that I could use in multiple projects. I came up with several design patterns, and perhaps some re-usable code, but it all still felt like a hack.

My path towards a more comprehensive solution to managing related entities came from the server world with a similar, but more lightweight relational model I started to explore with this post I made a little while back. Well, many all-nighters later, I think I have a 0.1 alpha version of a cool system that makes a relational, lazy enabled, dirty-checking javascript object model a piece of cake. And there is plenty of sugar to make it fun to use. Oh, and its dojo.data compatible.

(As an aside, I was excited when I first read about the JsonRestStore.. Indeed, it probably does a lot of this same work. So it may be an issue of style. I also don’t know a whole lot about JSON referencing, and my referencing model (to be documented) seemed a lot easier to translate on the server side. But as I said, I need to learn a lot more about it. I hope to start here.)

Having done quite a bit of work with Java in the past, I am big believer in Object-Relational-Mappers on the server side. I have used Hibernate most extensively, and have even dabbled with PHP-based ORM’s like Propel. I like the idea of creating a model, and letting the system figure out how to persist it.

In the client-side world of Javascript, obviously a full-fledged ORM would be overkill. But I think an analogy of an ORM on the server-side — which maps relational-database information to domain-objects — can be made. The only difference is that you are mapping your server communication to your domain objects. I’m not sure what to call this yet, but I am open to suggestions. For now, I’ll borrow “ORM”, though technically that is not what this is. Perhaps client-server object mapper? (CSOM?)

What is MedryxORM?

MedryxORM is an extension of the Dojo toolkit that allows for simple communication between your client-server services and your javascript domain model. It consists of an EntityManager that handles all the mapping of your simple javascript-objects to your javascript-services and can be used to query and persist your domain objects. Similar to Hibernate, this system aims to be as unobtrusive to your usual coding as possible, but provides access to several ways in which you can optimize the performance of your application once you have it working, making refactoring of your application for performance a much simpler process. There are several “shortcuts”, that make using the system easier, but may be a bit more intrusive on your usual coding practices, so they are optional.

Enough jargon. Lets do some coding, and then I’ll reveal how it all works. Since I am a physician, my domains almost always deal with medical records, so I’ll use that as an example. Lets say that we want to build an application that shows a table of patients. You can then drill-down on patient details like insurance information, clinical information, etc. You can add simple notes to the patient, and perhaps assign a responsible physician/team of physicians. You can then page the appropriate responsible physician.

So lets first describe out domain objects in javascript. Here is a model.Patient:

dojo.declare("model.Patient", null, {
  //id
  id:null,
 
  //simple properties
  fullName:null,
  initials:null,
  medicalRecordNumber:null,
  birthdate:null,
  gender:null,
 
  //associations
  visit:null
 
  toString:function() {
    return this.fullName || this.getFullName(); //I'll explain this later.    
  },
 
  getAge:function() {
  //snip... uses birthdate to calculate an age
  }
});

And here is a model.Visit:

dojo.declare("model.Visit", null, {
  //id
  id:null,
 
    //simple properties
  accountNumber:null,
  admissionDate:null,
  admissionTime:null,
  dischargeDate:null,
    dischargeTime:null,
  admittingDiagnosis:null,
    bed:null,
  room:null,
  ward:null,
 
  coverageNotes:null,
 
  //associations
  team:null,
 
  //collection of insurances
  insurances:null,
 
  getLengthOfStay:function() {
    //snip -- calculates number of days in the hospital based on the admission date.
  }
});

Well, you get the idea. Nothing very exciting here. But now let’s “register” these class with the EntityManager:

 
entityManager.registerClass({
  entityClassName:"model.Patient",
  alias:"Patient"
});
 
entityManager.registerClass({
  entityClassName:"model.Visit",
  alias:"Visit"
});

I’ll get into mapping and configuring the EntityManager in just a minute. But first lets jump ahead and see how you can use this:

 
var patient = new Patient(10);
alert( patient.getFullName() + " is " + 
      patient.getAge() + " years old!" );

Now, as long as their is a patient whose id = 10 on the server, then this will all *just work*. There is NO other necessary calls to xhr, or rpc, or REST, or anything else (Though those things happen behind the scenes, depending on how you configure your EntityManager.)

This might be even cooler:

 
var patient = new Patient(10);
 
alert ( "Uh-oh, " + patient.getFullName() + " has been here for " + 
    patient.getVisit().getLengthOfStay() + " days!" );

Again, there is no further coding required. And because you map your server data to your javascript objects, you can actually use methods declared on those objects as you normally would. This example would result in two calls to the server — one to get the patient, and the next to get the visit (though that would not actuall occur until the getLengthOfStay() method called getAdmissionDate()). But that is a default behavior. You could give the server some “hints” about how you will be using the Patient, and let it send you all the information you need on the first call. By default, the code above would behave use synchronous data access methods, but its a very simple tweak to convert this exact same code to friendly asynchronous calls. But all in due time…

Now, admittedly, this is a contrived use. How often will you be instantiating an object with an identifier and expecting it to connect to some backend? But a more realistic use-case would be to use the fetch/query methods of the EntityManager (using fetch if you want to use dojo.data style access, or query if you want to use my Deferred-style access). Once you make the query, your results are all instances of your object model, with all of the appropriate relationships initialized in all the appropriate ways.

var deferred = entityManager.query({query:{entityClass:"Patient", fullName:"Smi*"});
deferred.addCallback(function(patients) {
  dojo.forEach(patients, function(patient) {
 
    document.write ( patient.getFullName() + " is " 
        + patient.getAge() + " years old!" );
 
  });
})

And in terms of persistence, you can choose from a number of different options. Some people prefer the DAO design pattern, others prefer ActiveRecord. I think this (sort of) gives you both. Or you can choose to completely forget about persistence, and let it all happen automagically (flushing on unload).

 
var patient = new Patient(10);
patient.setMedicalRecordNumber("123456");
patient.save();

or…

 
var patient = new Patient(10);
patient.setMedicalRecordNumber("123456");
 
entityManager.saveEntity(patient);

or…

var patient = new Patient(10);
patient.setMedicalRecordNumber("123456");
 
entityManager.save(); //flush all dirty records to the server

You are guaranteed to have only a single instance representing each persistent entity. So, this works fine (not that it has much purpose):

var patient = new Patient(10);
 
var visit = patient.getVisit();
 
var deferred = entityManager.query({query:{entityClass:"Visit", patient:10}});
deferred.addCallback(function(results) {
    assertTrue ( visit === results[0] ) ;
});
 
//or...
 
var visitId = visit.getId();
var newVisit = new Visit(visitId);
 
assertTrue ( visit === newVisit );

I’ll get into the details of optimization, caching, and other goodies later. Now that you’ve seen a little of the magic, lets dive into how its all configured.

At the core of the EntityManager is the class metadata — or “mapping”, inspired by Hibernate. You can specify your mapping in two ways — as a JSON object or “natively”. Your choice metadata is essentially stylistic, and is equivalent to the Java argument of Annotations based configuration versus XML based configuration.

To start with, lets define some terms, to make writing this a bit easier:

• Managed Class: any class that is registered in the EntityManager
• Managed Entity: an instance of a managed class
• Managed Property: any property on your Managed Class that is managed by the EntityManager (Having unmanged properties in your ManagedClass is certainly allowed, but keep in mind that the EntityManager will totally ignore unmanaged properties.)

There are several types of ManagedProperty’s already built, and you are free to create your own to implement novel behavior.

• medryx.orm.Identifier: in this implementation, only numeric identifiers are allowed, and there must be one and only one Identifier per Managed Class
• medryx.orm.Property: used for any property of a Managed Class that is not an association — usually scalar values like numbers, strings, Dates, etc.
• medryx.orm.Association: any association to another Managed Entity. Note that this must be an association to a Managed class, otherwise you use Property above
• medryx.orm.Collection: a collection of Managed Entities. Again, this is inteded to be used for associations to elements of a Managed class only.

The details of each of the ManagedProperty types will be more fully discussed a bit later. Let look at a native mapping for a Patient. You could either create a subclass of patient, or just change your Patient class itself, which is the approach I will show here for simplicity.

dojo.declare("model.Patient", null, {
  //id
  id:new medryx.orm.Identifier(),
 
  //simple properties
  fullName:new medryx.orm.Property(),
  initials:new medryx.orm.Property(),
  medicalRecordNumber:new medryx.orm.Property({readOnly:false}), //this is a mutable field (not in real life, but for this example it is)
  birthdate:new medryx.orm.Property({converter:medryx.orm.converters.DateConverter}),
  gender:new medryx.orm.Property(),
 
  //associations
  visit:new medryx.orm.Association({associationClass:"model.Visit"});
 
  toString:function() {
    return this.fullName || this.getFullName(); //I'll explain this later.    
  },
 
  getAge:function() {
  //snip... uses birthdate to calculate an age
  }
});

and Visit is altered as follows:

dojo.declare("model.Visit", null, {
  //id
  id:null,
 
    //simple properties
  accountNumber:new medryx.orm.Property(),
  admissionDate:new medryx.orm.Property(),
  admissionTime:new medryx.orm.Property(),
  dischargeDate:new medryx.orm.Property(),
    dischargeTime:new medryx.orm.Property(),
  admittingDiagnosis:new medryx.orm.Property(),
    bed:new medryx.orm.Property(),
  room:new medryx.orm.Property(),
  ward:new medryx.orm.Property(),
 
  coverageNotes:new medryx.orm.Property({readOnly:false}),
 
  //associations
  team:new medryx.orm.Association({associationClass:model.Team}),
 
  //collection of insurances
  insurances:new medryx.orm.Collection({associationClass:model.VisitInsurance}),
 
  getLengthOfStay:function() {
    //snip -- calculates number of days in the hospital based on the admission date.
  },
 
  getRoomNumber:function() {
    return this.getWard() + "-" + this.getRoom() + "-" + this.getBed(); //always use getters to allow lazy initialization if needed
  }
});

Okay, so you are almost done with your configuration. (Obviously I’ve left model.Team and model.VisitInsurance out for brevity, but they are just as simple to “annotate”).

The next step is to plug in your PersistenceService. MedryxORM defines a PersistenceService interface that looks like:

dojo.declare("medryx.orm.PersistenceService", medryx.AbstractBase, {
 
    /**
     * the deferred calls back with an object:
     * {
     *    total: total number of records available for this query, or -1 if unknown
     *    items:array of items
     * }
     * returns a deferred (always)
     * 
     * @param {Function | String} entityClass
     * @param {Object} kwArgs (see ReadApi... still need to document queryOptions)
     * @param {Boolean} sync
     */
    $fetch:null,
 
    /**
     * returns a deferred (always) that callsback with 
     * the id if successful
     * 
     * if no id is null or < 0, then this is a put, otherwise it is
     * a post
     * 
     * @param {Function | String} entityClass, 
     * @param {Integer} id, 
     * @param {Object} propertiesToPersist
     */
    $save:null,
 
    /**
         * similar to WriteApi deleteItem
         * 
         * returns a deferred (always)
         * 
         * @param {Function | String} entityClass, 
     * @param {Integer} id
         */
        $deleteItem:null,
 
    /**
     * given an entiyClass, id, and properties, full load an instance 
     * that guarantees that each property in properties is initialized
     * (or if no properties are specified, then some contract
     * between client and server about "default" loading has
     * been fulfilled)
 
     * @param {Function | String} entityClass
     * @param {Integer} id
     * @param {Array of Strings} properties -- this is a list of properties that the 
     * server GUARANTEES will be present in the returned entity. The server *MAY* and 
     * frequently *WILL* return more than just these properties, and the EntityManager 
     * will handle that case appropriately and consume all the information the server sends
     * each item in the array may be either a property name, a dot-notated association property
     * or a *
     * for example: ["*", "visit.accountNumber", "visit.team.*"] would load all the properties
     * of the entityClass with id, the visit association with the account number loaded, and 
     * the team of the visit with ALL of its properties loaded. 
     * @param {Boolean} sync
     */
    $load:null
});

We also provide an RpcPersistenceService that gets rid of sync handling for you. You could easily create an SMD to your server that meets the requirements of this interface.

You provide a default PersistenceService to the EntityManager. If you do not specify a specific PersistenceService for a given Managed Class, then the default PersistenceService is used. However, it may be easier for you to implement a seperate SMD for each Managed Class, so you can supply a seperate PersistenceService for a particular class (or for every class). My server can easily interpret the “entityClass” and determine what to send back to the client, but if you wanted to make your server more generic, you may choose to ignore the entityClass on the server and let that mapping occur on the client. It would be extremely easily to build a REST PersistenceService (and I will probably do that when the need arises) into which you can adapt a Rest service to fit the PersistenceService API.

Let make this a bit less abstract. As I said, my server is pretty smart, so I it knows the right thing to do with entityClass. So I have a single PersistenceService that I specify when I “startup” my EntityManager (i.e. instantiate it).

ormPersistenceServiceSMD = {
  serviceType:"JSON-RPC",
  serviceUrl:"mySmartServer.php",
  methods: [
    {
      name: "load"
      parameters:[
        {name:"entityClass", type:"string"}
        {name:"id", type:"integer"}
        {name:"properties", type:"array"}
      ]
    },
    //snip... you get the idea
  ]
}

Then I can plug that into my EntityManager with a couple of wrappers:

var rpcPersistenceService = new medryx.orm.RpcPersistenceService(new dojo.rpc.JsonService(ormPersistenceServiceSMD));
var entityManager = new medryx.orm.EntityManager(rpcPersistenceService);
 
entityManager.registerClass({entityClass:"model.Patient", alias:"Patient"});
entityManager.registerClass({entityClass:"model.Visit", alias:"Visit"});
 
//and now everything above works!
 
var patient = new Patient(10);
patient.getVisit().setCoverageNotes("Some new coverage notes");
 
patient.save();

Woo-hoo! I obviously have a LOT of work to do in terms of documentation. But this is a start! Please, tell me what you think!!! I’ve posted the source on Google code. Maybe if I get lucky, some Dojo folks will take an interest, and we could be incubated as a dojox project! One can dare to have dreams, right?!