JsonRestStore — Custom Services, Schemas, and Lazy Loading
The newest and most complete store in the dojox.data arsenal is dojox.data.JsonRestStore. Its primary purpose is to make interaction between a RESTful data source and dojo.data seamless and simple, and it accomplishes that goal with ease. But a quick peek under the covers and you will find that you may very well be using JsonRestStore for all your Json dojo.data needs. This article aims to show you how you can use JsonRestStore and its more advanced features like lazy entity and property loading and client side filtering, and how you can adapt JsonRestStore to any data source that provides JSON data to your client.
RESTful web services map CRUD services to the HTTP protocol: Create = Put, Read = Get, Update = Post, and Delete = Delete. You can easily sort through the data that would be passed to each of these services as well — Create gets the full data of a new object without an id, Update gets the full data of the updated object with an id, read gets an id who’s object should be returned and delete gets an id whose object should be deleted. Its not rocket science. But if you broaden your thinking from REST, you’ll see this is essentially an API for interfacing with server side data stores. REST is simply one way of implementing it. Instead of REST, you could use an RPC service, or any other service, so long as you could map the service to this API.
JsonRestStore expects you to provide it a service that implements the REST API. The easiest way to do this is to simply specify a target url as a String. JsonRestStore will then automatically send that URL the commands in the API using the REST protocol described above. However, instead of simply providing a target URL, you can actually initialize a JsonRestStore with your own service. The service is simple an object that has the following methods that each return a dojo.Deferred:
- A directly callable function that takes an id or a query/queryOptions pair and returns the results of the query
- A put function that takes an id and value and creates the value at the given id
- A post function that takes an id and value and appends/updates the value at the given id
- A delete function that takes an id and deletes the value at that id
It is important to know that if you do provide a custom service, you must still provide a unique string as the “target” attribute for the JsonRestStore. As I will explain later — JsonRestStore actually manages interdependencies between objects and between data stores — and the target attribute is how it uniquely identifies one store from another, as well as how it identifies which store a particular object belongs to.
So lets start coding. First lets define a custom service that we will be providing to JsonRestStore. Lets say we are building a system for paging physicians. Each physician has a pager. We’ll start simple (this is illustrative code, not real code!):
var mdService = function(query, queryOptions) { return dojo.xhrGet({url:"queryUrl.php", handleAs:'json', content:{query:query, queryOptions:queryOptions}}); } mdService.put = function(id, value) { mdService.post(id, value); } mdService.post = function(id, value) { return dojo.xhrPost({url:"actionUrl.php", handleAs:'json', content:{action:"update", id:id, value:value}}); } mdService['delete'] = function(id) { return dojo.xhrPost({url:"actionUrl.php", handleAs:'json', content:{action:"delete", id:id}}); }
And that is it. Now lets feed this service to JsonRestStore and create new store for our physicians:
var mdStore = new dojox.data.JsonRestStore({target:"md", service:mdService});
You can now use this store as you would any other store — feed it to a tree, or a grid, or to any other dojo.data enabled widget, and everything just works. But it gets better. Much better.
As I stated before, lets build our paging system. In our system we have about 10 pagers and 30 physicians. At any one time, each pager is assigned to 0 or 1 physicians. So in addition to being able to navigate from physician to pager given a list of physicians, we need to be able to navigate from pager to physician given a list of pagers. So lets build that store:
var pagerService = function(query, queryOptions) { return dojo.xhrGet({url:"pagers.php", handleAs:'json',content:{query:query, queryOptions:queryOptions}}); } pagerService.put = function(id, value) { var d = new dojo.Deferred(); d.callback(); //put is a noop for pagers return d; } pagerService.post = function(id, value) { return dojo.xhrPost({url:"updatePager.php", handleAs:'json', content:{id:id, value:value}}); } pagerService['delete'] = function(id) { var d = new dojo.Deferred(); d.callback(); //delete is a noop for pagers return d; } var pagerStore = new dojox.data.JsonRestStore({target:"pager", service:pagerService});
This where the magic starts. Lets create some data:
[ {"id":"1", "name":"Maulin Shah, MD", "pager":{$ref:"pager/1"} }, {"id":"2", "name":"Doogie Howser, MD", "pager":{$ref:"pager/2"} } ]
and here are some pagers
[ {"id":"1", "md":{$ref:"md/1"}, "number":"12345", "name":"Yellow Pager"}, {"id":"2", "md":{$ref:"md/2"}, "number":"54321", "name":"Triage Pager"} ]
Notice the “$ref” notation. This is JsonSchema style notation for indicating references to other objects. This lets you simply define the reference without loading the referenced object, and the referenced object will be lazily loaded when necessary. Notice the the $ref is essentially a “path” to the referenced object — or in less RESTy lingo, its the “class” of the referenced object along with its id (or perhaps its the “table” of the referenced object along with its primary key, if that’s the lingo that floats your boat).
Note that you do not have to make a reference lazy, or you make the reference partially lazy. I’ll go over these concepts a bit later.)
So what happens when we load some physicians into a grid?
<table border="0"> <thead> <tr> <th>Physician</th> <th>Pager</th> </tr> </thead> </table>
This would create a grid that looks something like:
| Physician | Pager |
|---|---|
| Maulin Shah, MD | [Object object] |
| Doogie Howser, MD | [Object object] |
Woops. Not quite what we wanted. The problem here is that the pager object has no toString() method. So we have two options to solve this. Thie first involves using a schema and object prototype with your store.
Each store accepts a “schema” attribute as an initialization parameter. The schema is a JsonSchema style object describing the properties of the json objects that are returned for that store. In addition to the usual “properties” attribute of the schema, the schema can have a special “prototype” attribute that defines an object prototype that is used when the object of this class are instantiated. Thus you can mixin your own custom methods or properties into each instance of objects managed by your store. So lets create the prototype for the pager store:
var pagerSchema = { prototype: { toString:function() { return this.number; } } }
Feed this to the store when it is created, and we’ve solved our problem:
var pagerStore = new dojox.data.JsonRestStore({target:"pager", service:pagerService, schema:pagerSchema});
Now your table will look like you expect:
| Physician | Pager |
|---|---|
| Maulin Shah, MD | 12345 |
| Doogie Howser, MD | 54321 |
(The second way to solve this problem involves over-riding the getValue method of JsonRestStore to allow dot-path notation. I’ll describe that in a future post that discusses my BeanStore extension of JsonRestStore.)
So what happened behind the scenes in rendering this table? The grid called physicianStore.fetch with no query (which is the same as “fetch all”). The Json described above was returned, with its references to the pager objects. JsonRestStore magically instantiates a new first class javascript object for each item in the Json, using the prototype in our schema as the prototype for the object. When mdStore.getValue(item, “pager”) is called by the Grid, the mdStore initiates a synchronous lazy call to pagerStore’s fetchItemByIdentity method which returns the pager, instantiated with the pager schema, which has a toString method which returns the number.
You could see how laziness could get out of hand very quickly. If we have a hundred physicians, we would have 100 synchronous calls to the server to lazy load pagers. That is not ideal. Lazy loading should really be reserved for more of a master-detail type usage when most lazily referenced items are never actually returned. So how could we make this more efficient? We can modify the data returned in our physicians data to include the pager number as well as a reference to the pager.
[ {"id":"1", "name":"Maulin Shah, MD", "pager":{$ref:"pager/1", "number":"12345"} }, {"id":"2", "name":"Doogie Howser, MD", "pager":{$ref:"pager/2", "number":"54321"} } ]
Now this is really cool. Now when rendering the grid, we incur no new requests to the server, since the pager number is already available locally. But now lets say we added a function where clicking on a row of our grid popped up an alert with the name of the pager. Since the name is not available, it would now lazily load the pager (i.e. call pagerStore.fetchItemByIdentity) and flesh out the rest of the pager object. Pretty clever eh?
In this article I have touched on just a few advanced features available with JsonRestStore — using a custom service, using a schema to provide an object prototype, lazily loading objects, and sparsely loading objects. But there is plenty more goodness to be found in this store! Stay tuned for an article on client-side caching and other great features of JsonRestStore!
This is an fantastic tutorial! You have certainly been instrumental in helping to evolve and mature JsonRestStore, great job and thanks for the help!
Comment by Kris Zyp — July 30, 2008 @ 7:17 pm
i’m having problems using a service definition the way you’ve described. i wonder if things have changed since you wrote this. any updates?
Comment by ben hockey — September 16, 2008 @ 5:20 am
@ben:
Make sure you set your XHR handler to handle the response as JSON:
handle:”json”
Comment by Kris Zyp — September 17, 2008 @ 9:43 pm
That’s:
handleAs:”json”
Comment by Kris Zyp — September 17, 2008 @ 9:44 pm
>> pagerService.delete = function(id) {
>> var d = new dojo.Deferred();
>> d.callback(); //delete is a noop for pagers
>> return d;
>> }
Do we trip up here with delete being a reserved word in ie?
Comment by Mark Antrobus — September 19, 2008 @ 11:13 am
Same issue as Mark, I can’t make this work in IE.
Any suggestions?
Comment by Braden Rogness — September 19, 2008 @ 4:06 pm
I made a couple modes — delete is reserved, so access via obj['delete']. Also added the handleAs attribute, and finally fixed the content= to content:. I hope this solves. When I get a chance I’ll actually build the example so you can look at it live.
Comment by maulin — October 3, 2008 @ 12:55 am
I tried to use the JsonRestStore with a dojo grid, but it’s not working.
My problem is that the grid doesn’t show the data ; I see the data is fetch, but either I have a “no data” message written on the grid, either I have an error message, depending on the data format I fetch.
Here’s the code :
var Service = function(query, queryOptions) {
return dojo.xhrGet({url:”/test/”, handleAs:”json” }); }
Service.post = function(id, value) {
return dojo.xhrGet({url:”/test/”, content:{method:”create”, id:id, value:value}}); }
store = new dojox.data.JsonRestStore({target:”test”, service:Service});
// the grid
dojo.addOnLoad(function(){
// set the layout structure for the grid version 1.2
var layout4 = [
{ name: "Test", field: "post_id", width: '40px'},
];
// create a new grid:
var grid4 = new dojox.grid.DataGrid({
query: { post_id: ‘*’ },
store: store,
structure: layout4,
}, document.createElement(’div’));
dojo.byId(”gridContainer4″).appendChild(grid4.domNode);
grid4.startup();
});
Now, /test/ returns this data format :
[
{"post_id":"1","user_id":"1","url":"new-post","ts_created":"2008-10-01 14:16:30","status":"Live","profile_key":"title","profile_value":"new post"},
{"post_id":"2","user_id":"1","url":"new-post-draft","ts_created":"2008-10-01 14:17:38","status":"Draft","profile_key":"title","profile_value":"New post draft"},
{"post_id":"20","user_id":"1","url":"test-3","ts_created":"2008-10-01 22:28:46","status":"Draft","profile_key":"title","profile_value":"test"}
]
I tried also to return another data format that worked using IwriteFileStore that I tried, but that got me nowhere :
{”identifier”:”post_id”,”items”:
[{"post_id":"1","user_id":"1","url":"new-post","ts_created":"2008-10-01 14:16:30","status":"Live","profile_key":"title","profile_value":"new post"},
{"post_id":"2","user_id":"1","url":"new-post-draft","ts_created":"2008-10-01 14:17:38","status":"Draft","profile_key":"title","profile_value":"New post draft"}
{"post_id":"20","user_id":"1","url":"test-3","ts_created":"2008-10-01 22:28:46","status":"Draft","profile_key":"title","profile_value":"test"}]
}
Anyone an idea of what’s going wrong ?
Z.
Comment by Zladivliba — October 8, 2008 @ 11:36 am
mckatpckoug0r7k2
Comment by Kathy Nichols — November 12, 2008 @ 10:09 pm
This article illustrates why I have tended to use Prototype over Dojo in most cases. Is there a SIMPLE example of accessing a restful resource using JsonRestStore? and the tying is to a DataGrid?
Thx
Eric
Comment by Eric — November 28, 2008 @ 2:48 am
Hi,
I have the same problem as Nr. 8 - Zladivliba: My JSON service returns the data in the correct format (I assume?!?), but the data doesn’t show in the JsonRestStore (in Firebug), nor in the dataGrid which the store is attached to. Interestingly, only dojo release 1.2.0. actually calls the url specified in the first Service function:
var Service = function(query, queryOptions) {
return dojo.xhrGet({url:”/test/”, handleAs:”json” }); }
dojo 1.2.1 and 1.2.2. do not make the server call to “/test/” for whatever reason. I can see all this happening (or not) in Firebug. Is that a good thing?
Isn’t there a working example of a JsonRestStore implementation to be found anywhere, so we can take it apart with Firebug etc. and see how it works? I have found a “JsonRestStore.js” file in the “dojox/data/tests/stores” folder of the developer distribution, but there seems to be no .html file to go with it?
Many thanks for taking the time to write this tutorial,
philou
Comment by philou — November 30, 2008 @ 8:28 pm
Hi,
I’m trying to use your “$ref:” notation but the dojox.data.Grid doesn’t show anything, not even [Object object] in the cell with the reference.
I have a “user” store and a “role” store. The “user” with id “1″ has a “user_privilege_level”:{$ref:”role/admin”, “role_id”:”admin”}
I have also used this schema for the roleStore:
var roleSchema = { prototype: {toString:function() { return this.role_id;}}};
According to your tutorial, this should show the “role_id” (=”admin”) in the “user_privilege_level” cell of “user/1″. But it doesn’t.
However, firebug shows this when I check the usersStore item in the DOM that should have the reference (for the field: user_privilege_level):
user/1……….Object user_id=1 user_full_name=Joe
->
__id……….”user/1″
user_full_name……….”Joe”
user_privilege_level……….Object $ref=role/admin role_id=admin __id=user/role/admin
->
$ref……….”role/admin”
__id……….”user/role/admin”
role_id……….”admin”
_loadObject……….function()
->
prototype
user/2 …. etc etc
Do you have any idea how I can track down where I’m going wrong?
Many thanks,
p.
Comment by philou — December 9, 2008 @ 10:23 pm
OK, silly me:
my dojo.data.Grid wasn’t sowing anything because I had made a mistake in my grid “layout” definition.
However, I still can’t get your protoype: toString method to work. I have copied the Schema definition exactly as you have it above and passed it to the store, but the grid still shows [object Object].
I have tried to use a formatter function for the grid cell which should display the referenced object and then had the formatter function return object.toString, but it still shows [object Object] in the grid.
nonWorkingFormatterFunction (object) {
return object.toString;
}
if I use the formatterFunction to return object.myProperty, then the content of myProperty is correctly displayed in the grid. That works, but it means I will have to put a formatter function somewhere for every kind of reference….
Any idea?
Many thanks!
p.
Comment by philou — December 12, 2008 @ 8:07 pm
Dear Maulin,
I realise you’re providing this tutorial as a service to the community presumably in your spare time. Many thanks for that. However, I feel that there are surely more people like myself out and about who fail to benefit from tutorials like these, because we are simply not able to translate your “illustrative code” into something that works. It is true that I am not a professional programmer, but I would consider myself Pro enough to eventually get a grasp of what is being discussed here. Alas, I find that I am spending hour after hour trying to piece together these tutorials (be they yours or those by Kris Zyp), and I often stumble over a little detail which was not mentioned anywhere, since it may be obvious to those who are as familiar with dojo as you are. (Example: The JsonRestStore data would not show up in my dataGrid for days, because I wasn’t calling “grid.startup();”, which apparently wasn’t necessery for the itemFileWriteStore I was using before.)
My question:
Is it not possible to provide an archive with a complete html and a json file which can be used to see the basic functionality of the tutorial in action (here: referencing)? Bryan Forbes has done that on sitepen for dataGrid and it has helped my understanding tremendously.
Either way many thanks,
p.
Comment by philou — December 15, 2008 @ 10:36 pm
Thanks for this post. I’d like to offer a tip I got from neonstalwart on #dojo. I’ve got a servlet that needs to do some custom authentication based on a security token (a simple string) known by the browser, and I was able to make my JsonDataStore pass it with every server request by adding it to the ‘content’ attributes in the service. e.g. to add a ‘myCustomContent’ attribute with a value from the ‘foobarwibble’ variable:
var mdService = function(query, queryOptions) {
return dojo.xhrGet({url:”queryUrl.php”, handleAs:’json’, content:{myCustomContent: foobarwibble, query:query, queryOptions:queryOptions}});
}
(and do that for the other xhr methods too) Then the receiving GET,POST etc. handler may perform its own authentication before deciding whether to service the request.
However, a correction: Your first example needs to return from the ‘put’ method, i.e. change:
mdService.put = function(id, value) {
mdService.post(id, value);
}
to:
mdService.put = function(id, value) {
return mdService.post(id, value);
}
Otherwise the code in JsonRest.js that uses the result of ‘put’ as a Deferred (as a result of executing service[action.method]) runs into a ‘dfd is undefined’ error. Trying to debug that led me to learn a little more about debugAtAllCosts and how it can help firebug reveal the line of code experiencing an error
Comment by Nick Fenwick — May 13, 2009 @ 4:35 am
There’s a subtle correction you need to make in this article. In RESTful architectures, POST will create, and PUT is used to update — which is exactly opposite of what you describe. JsonRestStore does do the correct thing here — it’s designed to POST content that needs to be created on the server, and PUT content to update.
Comment by Matthew Weier O'Phinney — September 1, 2009 @ 10:35 pm
Hello, Medryx, I have a problem with the server side scripting. I use python. And I fail at deletion. The jsonreststore send the delete method to the server and I successfully deleted the specified uuid item in the database. But it seems that the client side item didn’t delete. It is still there when I try to list all items in the store. I guess it probably the problem of the return value of delete request. What should the server return for the delete operation? Right now I just return nothing to the client.
Comment by dynaturtle — December 24, 2009 @ 10:30 am