Medryx Observations

Well its been a while since my first post about a search widget for dojo.data stores. There are a lot of reasons — the beautiful Portland, Oregon summers being the most significant. But now the kids are back in school, life is normalizing, and its time to show you how I finished what I think is a very nifty widget.

First a recap: My goal was to make a pretty widget that simply allowed you to search one or more dojo.data stores. You could use this in a lot of ways. I’ve posted a demo here that uses several dojox.data stores to allow you simultaneously search Google, Yahoo!, and Wikipedia from your site. While this is one use case, the use case I find most interesting is doing a “global” search on your own dojo.data stores that are driving your web application. But the sky is the limit! You can use this to build OSX Searchlight-style functionality into your app. You can “skin” the widget itself to make it match your site’s style. You an change the prompt from “Search” to anything you want, that may be more useful in your app. You can even include a dropdown menu that appears upon clicking the magnifying glass to the left of the search box that allows your users to choose which stores (or which methods) they would like to use to search. You can define a template for your result to make it display in just about any way you can imagine. (The default simply shows the “label” text from the resulting items.)

Before going any further, I want to make it clear that this works, but is clearly alpha-level code. And I think there are some simplifications that can be made in the usage of the widget in the future if this is useful to more people than me. But for now, think of it as the 0.9 dojox.grid — way too complicated, but really functional. And hopefully this widget will evolve into a simpler and more elegant widget just as the grid has. FYI, this widget is dependent on dojo 1.2 (primarily because it uses the 1.2 dojox DataGrid).

So lets dive into how to use the widget. The first concept we need to go over is how to create an interface into your stores to allow searching them in various ways. To solve this, you need to create a SearchStore. A SearchStore is really a composite of interfaces into other dojo.data stores. As is my usual practice, sometimes things are explained more easily with code, so here goes. Lets build the demo widget. We’ll start with just the WikipediaStore:

dojo.require("medryx.widget.Search");
dojo.require("dojox.data.WikipediaStore");
 
//create the WikipediaStore
var wikiStore = new dojox.data.WikipediaStore();
 
//create the SearchStore
var searchStore = new medryx.widget.SearchStore();
 
//add the wikiStore to the SearchStore
searchStore.addStore({
  name:"Wikipedia", // what will this search be called (useful when there is more than one store
                    // in your SearchStore (or more than one method or more than one method of
                    // searching the same store
  store:wikiStore, //the source store
 
  //this is the most important piece. given a search string (searchTerms) and
  //and a requested searchType, return the query that should be passed to the
  //the store (in this case to wikiStore.fetch())
  queryBuilder:function(searchTerms, searchType) {
    //if the user specifies "Wikipedia" search, or any search, then use this store
    if (!searchType || searchType == "Wikipedia" || searchType == "*") {
      return {query:{text:searchTerms, action:"query"}}; //see WikipediaStore for how to query Wikipedia
    }
  }
});

Okay, that does seem a little complicated. But conceptually its pretty simple. When a user types a string into the search box, the widget needs to know how to handle it with respect to your source stores — so you provide a “queryBuilder” method that converts generic search terms to your specific query. When there is just one member of you SearchStore, this seems like overkill. So lets add some more stores. I used the dojox.data.ServiceStore to create a Google store and Yahoo store that can search those services. The code for how to create those services is a bit beyond the scope of this article, but you can see the full code in the demo. For now, lets take it on faith that there is a yahooStore and a googleStore that I have created. So lets wire them up to our SearchStore:

searchStore.addStore({
  name:"Google",
  store:googleStore,
  queryBuilder:function(searchTerms, searchType) {
    if (!searchType || searchType == "Google" || searchType == "*") {
      return {query:{q:searchTerms}}; //see google smd for format
    }
  }
}); 
 
searchStore.addStore({
  name:"Yahoo",
  store:yahooStore,
  queryBuilder:function(searchTerms, searchType) {
    if (!searchType || searchType == "Yahoo" || searchType == "*") {
      return {query:{query:searchTerms}}; //see yahoo smd for format
    }
  }
});

So now you can see how we can create a very complicated method for searching multiple different stores with the same search string. There are several more features to the SearchStore, but I’ll come back to those. Right now, lets change gears and see how we can actually use this widget with this SearchStore. In the simplest case we simply declare the following in our markup:

<div dojoType="medryx.widget.Search" store="searchStore">
  <script type="dojo/connect" event="onHitSelected" args="item, store, hit">
    console.debug("Hit selected:", item, store, hit);
  </script>
  <script type="dojo/connect" event="showAllResults" args="terms">
    alert("search for more results for \"" + terms + "\" yourself, you lazy bum!");
  </script>
</div>

You can see that you can easily connect to the “onHitSelected” event to decide how you will handle the user selecting one of your search results. This event passes the original item, and its original source store as parameters. It also provides access to the “hit” itself, which is an item in the SearchStore that maps to the original item from your source store. This can be very useful, once you know about some more features of the SearchStore. We’ll get there, I promise.

Notice also that you can connect the the “showAllResults” event, which is really expected to behave as a link to your “complete search” page (or dijit.Dialog, or whatever). The “Complete search results” link is at the bottom of the search results in all cases.

Lets look at some ways to customize the widget a little more:

<div dojoType="medryx.widget.Search" store="searchStore">
  <div dojoType="dijit.Menu">
    <div dojoType="dijit.MenuItem" searchType="Google" iconClass="Google">Google</div>
    <div dojoType="dijit.MenuItem" searchType="Yahoo" iconClass="Yahoo">Yahoo</div>
    <div dojoType="dijit.MenuItem" searchType="Wikipedia" iconClass="Wikipedia">Wikipedia</div>
  </div>
  <script type="dojo/connect" event="onHitSelected" args="item, store, hit">
    console.debug("Hit selected:", item, store, hit);
  </script>
  <script type="dojo/connect" event="showAllResults" args="terms">
    alert("search for more results for \"" + terms + "\" yourself, you lazy bum!");
  </script>
</div>

If you include a dijit.Menu inside your declaration, then this menu will be displayed when you click the magnifying glass to the left of the search box. You can see there is a special (REQUIRED) attribute on each MenuItem called “searchType”. This is the searchType that is sent to your “queryBuilder”, and can be any string that your queryBuilder understands. If a user does not expand the menu, and simply presses “Enter”, then the searchType is by default set to “*” (though you can change this via the defaultSearchType attribute).

There are plenty of attributes to help you customize your search widget, and they are all documented in the source of medryx.widget.Search.

So now lets look at how we might do something useful with our “onHitSelected” event. Let say we want to open an iframe with the url of the hit (as in the demo). First add the iframe to your markup:

<div align="center">
  <iframe style="width:90%; height:600px; border:1px solid black; display:none;" id="preview"></iframe>
</div>

So one way to get to the “right” url regardless of which store your result came from would be to make a big, complicated event handler (can you tell this will not be the preffered method?)

var hitHandler = function(item, store, hit) {
  var src = "";
  if (store == yahooStore) {
   src = store.getValue("ClickUrl");
 } else if (store == googleStore) {
   src = store.getValue(item, "unescapedUrl", store.getValue(item, "url", defaultValue));
 } else if (store == wikiStore) {
   src = "http://en.wikipedia.org/w/index.php?title=" + store.getValue(item, "title");
 }
  dojo.byId("preview").src = src;
  dojo.byId("preview").style.display = "";
}

This is perfectly reasonable, but its quite “spaghetti-like”. And every time you add a store, you have to change you handler. What if we could define for each store how to obtain the “src” string, and then use a common way of accessing it? Much cleaner IMHO. So SearchStore has another attribute you can include when calling addStore — “map”. A map is a list of pseudo-fields that will be accessible for each hit from the SearchStore itself. Each pseudo-field begins with “search”. So lets upgrade our searchStore as follows, adding in the maps:

searchStore.addStore({
  name:"Google",
  store:googleStore,
  queryBuilder:function(searchTerms, searchType) {
    if (!searchType || searchType == "Google" || searchType == "*") {
      return {query:{q:searchTerms}};
    }
  },
  map:{
    searchUrl:function(item,attribute,defaultValue) {
	return this.getValue(item, "unescapedUrl", this.getValue(item, "url", defaultValue));
    }
  }
}); 
 
searchStore.addStore({
  name:"Yahoo",
  store:yahooStore,
  queryBuilder:function(searchTerms, searchType) {
    if (!searchType || searchType == "Yahoo" || searchType == "*") {
      return {query:{query:searchTerms}};
    }
  },
  map:{
    searchUrl:"ClickUrl" //simple! so no "function" needed, just map from "searchUrl" to yahoo's "ClickUrl"
  }
}); 
 
searchStore.addStore({
  name:"Wikipedia",
  store:wikiStore,
  queryBuilder:function(searchTerms, searchType) {
    if (!searchType || searchType == "Wikipedia" || searchType == "*") {
      return {query:{text:searchTerms, action:"query"}};
    }
  },
  map:{
    searchUrl:function(item, attribute, defaultValue) {
      return "http://en.wikipedia.org/w/index.php?title=" + this.getValue(item, "title");
    }
  }
});

Now the event handler can be simplified, using the “hit” parameter:

var hitHandler = function(item, store, hit) {
    dojo.byId("preview").src = searchStore.getValue(hit, "searchUrl");
    dojo.byId("preview").style.display = "";
}

Much nicer, don’t you think? And you could add more stores to this widget or more ways to search the same store, and never have to touch the event handler.

Last but not least, you can customize how you “hits” appear on the dropdown list, as well as how the “category headers” appear. Simple include a child of your widget with an attribute of template=”category” or template=”hit”. You can then access any of  the attributes (or pseudo-attributes) of your item with the familiar ${attributeName} notation.

Building this widget taught me a LOT about dojo widgets. Unfortunately it wasn’t until I was almost done with the widget, that I read about the dojo 1.2 attributeMap. So obviously, to really take this widget to the next level, some work would need to be done. But right now it is quite functional, and can quickly and easily add a lot of robust functionality to your applications with very little additional work.

For now, you’ll need to grab the files you need from my dev server at http://dev.medryx.org/lib/js/medryx/. I’ll try to submit this to dojo as a dojox.widget sometime soon.

Let me know what you think, and if I can answer any questions!

Transparent PNG’s are fabulous. They are the key to making your app look really professional and slick. And they allow you to make easily themed web pages without having to redesign all your images, since you can just rely on the background colors shining through the transparency. Transparent PNG support is available on Firefox and Safari, and IE7. But IE6 does not support it.

There are, however, workarounds. And when it comes to making all browsers behave the same in the background, equalizing browser differences so you can just focus on writing great web applications — dojo is the best tool around. So it seems only natural to me that we should try to create a solution to this problem with dojo.

Fortunately, there has already been great work done by Angus Turnbull (which by the way is a GREAT name!) He has created a nifty solution to the problem. It turns out that IE6 does support transparent PNG’s when a special “filter” is used to load the image. Well angus has created a script that uses IE6’s “behaviors” via *.htc javascript files to implement a simple solution that will replace all your images with this filter-loaded version. This includes css background images and IMG tag images. And in his latest version he handles background-repeat and background-position, so this really becomes a simple drop-in solution, and your transparent PNG’s suddenly work perfectly on all the major browsers.

There is just a *little* configration needed to get Angus’s IEPNGFix to work. First you drop the iepngfix.htc, iepngfix_tilebg.js, and blank.gif files somewhere on your server. You then add the following to your CSS file:

img, div { behavior: url(/path/to/iepngfix.htc) }

You also need to open and change the path to blank.gif in iepngfix.htc to the absolute path to the blank.gif file. (Or a relative path, but relative to *any* page that will actually use the file, which is inconvenient). Finally you use a typical script tag to include iepngfix_tilebg.js in your page if you want background-repeat and background-position to work.

Its really not too bad, but its still more work that I want to do. And I would love to be able to use it any dojo web app without all this configuration. Well, dojo to the rescue! With a few modifications, you can make this available with a simple:

dojo.require("medryx.util.png");

(Or you can use whatever namespace you want!) Ideally, this could some day be included in dojo natively, so you wouldn’t have to do any work to get this support. In this technique, you don’t worry about absolute or relative paths — because you replace all that stuff with calls to dojo.moduleUrl
and you can eliminate the change to your CSS with a nifty CSS trick that Angus provides. Finally, you can eliminate the script tag to load iepngfix_tilebg.js by just adding a dojo.provide() call at the top of that file and using a dojo.require() to get it.

Sounds like a lot of work, but really it wasn’t, and now I can use it in any of my web applications without any real work. So here is the step-by-step process:

1. download the IEPNGFix.zip file
2. create the following directory structure and copy the files from the zip as follows — lets assume you will put this in a “medryx” module, though obviously yours would be named differently:

   dojotoolkit/
   -- dojo/
   --dojox/
   --dijit/
   --medryx/
   ----util/
   ------png.js --> see below
   ------png/
   --------iepngfix.htc
   --------iepngfix.php --> optional, see below
   --------iepngfix_tilebg.js
   --------blank.gif

3. Create a file called “png.js” whose contents are quite simple. It simply checks your browser, and if needed, adds a rule to your first stylesheet to handle png’s with the behavior fix (this is a slight modification right from Angus’s demo page). The content of the file is as follows:

   dojo.provide("medryx.util.png");
    
   (function() {
   //a fix for transparent png in ie6 (only works with php, or change this to the iepngfix.htc file if you can get the server to set content-type correctly!
    
   if (dojo.isIE && dojo.isIE < 7 &&  document.styleSheets && document.styleSheets[0] && document.styleSheets[0].addRule){
   	//dojo.require("medryx.util.png.iepngfix_tilebg"); needed to support repeat, but doesn't work for me.
   	var fixUrl = dojo.moduleUrl("medryx.util.png", "iepngfix.php"); //use *.php if your server doesn't server the *.htc file correctly
   	document.styleSheets[0].addRule('*', 'behavior: url("' + fixUrl.path + '")');
   }
    
   })();

   One small thing to notice — the iepngfix.php file is really only needed to set the content-type header of the *.htc file correctly so IE6 will know what to do with it. Obviously there are other (better) ways of doing this on each type of server. But the php file is a simple workaround if you don’t have access to changing how *.htc content-type is set on your server.

4. Open the iepngfix.htc file and search for “blank.gif”. Replace that line with:

   IEPNGFix.blankImg = dojo.moduleUrl("medryx.util.png", 'blank.gif').path;

   You might also want to search for “alert” and change it to console.warn instead, but that’s not mandatory!

5. Open the iepngfix_tilebg.js file and add the following to the top line:

   dojo.provide("medryx.util.png.iepngfix_tilebg");

6. Now on any app that you want to use transparent PNG, just add to your set of dojo.require statements:

   dojo.require("medryx.util.png");

A couple more side notes — I can’t seem to get background-repeat or background-position to work right with the *.js file. I haven’t had time to really troubleshoot it, since I don’t need it right now. If anyone has the solution, please let me know! A second note is that you should consider NOT using 1px images that are repeated, but instead use a more reasonable size, to limit the number of repeats. So if you know the smallest size of your display is 100px wide, then make your PNG 100px wide, and then repeat. Each repeat takes browser processing, and these days the download size of a 1px image and 100px image is not significant enough to make much of a difference.

Hope this helps someone! It has certainly helped me. Thanks Angus!

Search has become ubiquitous on web sites and applications. Its long since replaced categorization or filing. On my new mac, I can’t remember if I have ever dug through my Applications folder looking for an app. I just type a few letters in Spotlight, and voila. On the dojo website its the same story.

But what is conspicuosly missing (IMHO) from the dojo widget set (dijit) is a robust “search” widget. While you can certainly use the widgets available, or easily create new widgets to make a search field and present the results, this seems to me like it would have a lot of broad applicability throughout the dojo community. And honestly, without someone spending a lot of time up front, most of these quickly put together solutions don’t look great.

In my opinion the search feature on apple.com is an ideal “Web 2.0″ flavored search. Type a few letters and you get pertinent results, almost instantly, and without a page refresh. The results are present in a categorized fashion, and there is a link to see all of your results or do a more complete search of the apple website. Aside from its functionality, I think its just plain pretty. A nice rounded text box, that is very intuitive to use, and declares it purpose without any real prompting or explanation.

So I’ve embarked on creating a search widget for dojo — dijit.Search. (Well that is awfully presumptious of me. Obviously , none of this is sanctioned Dojo code, nor has it even been submitted, let alone accepted as a part of Dojo. So for now, I’ll keep it in my own namespace — medryx.widget.Search).

So let’s get started. First up — lets describe what the widget will do:

1. A pretty search field that gives feedback about what’s going on, and allows the user to select a specific type of search to use if desire (similar to the Firefox 3 search field).
2. Allow searching via a few mechanisms: perhaps searching more than one data store on the client, or more than one field in the same store for a match. But also obviously provide the ability to perform the search on the server and return pertinent results. I think this is an important point — I think search has really come into its own as a “navigation tool” since it has been able to search all over the place for a text string, and provide relevant results with links to either launch a function, view a page, or view result details.
3. Create a nice looking, categorizable search result window that pops up in some fashion on the screen, behaves correctly from an A11y standpoint, and provides the ability to request a more detailed search if the “preliminary” search results have not delivered satisfactory results.

Okay, so its not exactly a “spec”, but its something to work with. I always find it easiest to start really thinking about the “backend” of a widget, by first fleshing out the front end. So lets get to work on the pretty search field.

I want the field to have rounded edges, a magnifying glass that can be clicked to reveal search options, a “prompt” inside the text field that disappears/re-appears correctly, and an animated gif of some sort inside the field to tell you when the search is in progress.

The easiest way to accomplish this will be with thee background images (that may eventually merged into the same sprite) — one for the left semicircle with magnifying glass, one for the right semicircle, and one for the text field portion. So I want my final search box to look like this:

A couple notes: obviously not everyone will be working with a gray background onto which to put the search box. But I don’t want to create an infinite number of colors. So I want to use PNG images which have excellent transparency support. That way you cut put the box on any color background with good effect:

Then only other configuration I will need will be if there is no background (white or transparent background). But I’ll deal with that later.

So what’s the catch? IE6 (of course). It does not support transparent PNG’s natively. Thankfully there is a nifty fix available via LGPL called IEPNGFix that solves this problem rather well. And with very little effort you can dojo-ize this solution (which I have done). Now I just make a call to dojo.require("medryx.util.png") and I can reliably use transparent PNG’s in a cross-browser compliant way. (That piece in and of itself took me a whole night! See why it would be great to have an even richer set of dijits? If someone else had solved that problem or the pretty search widget problem already, I could’ve spent my time doing better things last night! Nah, who am I kidding? I would’ve been doing something else nerdy…)

So we want to be able to use these boxes anywhere on the page, and allow them to be of any size. I think the best solution is to have them adhere to the dimensions of their original DOM node as such:

<div dojoType="medryx.widget.Search" style="width:400px;"></div>

So here is the CSS and html that can put this together to make the images above:

<div class="medryxSearch">
	<label for="search_${id}">
		<div class="medryxSearchBox" style="width:${paddedWidth}px">
			<span class="left" dojoAttachEvent="onmousedown:_ignoreBlur, onclick:onOptionsClicked" dojoAttachPoint="optionsNode" id="searchOptions_${id}"></span>
			<div style="width:${width}px"><input style="width:${inputWidth}px" id="search_${id}" autocomplete="off" dojoAttachPoint="inputField" dojoAttachEvent="onkeyup:_onInputKeyUp, onblur:_onInputBlur, onclick:clearPrompt" /></div>
			<span class="right"></span>
			<span class="searching" dojoAttachPoint="searchingNode"></span>
		</div>
	</label>
</div>

I’ll explain some of the template stuff a bit later. Lets focus on the “medryxSearchBox”: This is a fixed-width box, whose size is taken from the contentBox of the declaring node. The width is then padded to include the width of the left/right portions of the widget. The first span is the left bit, the next is the text field, and the next span is the right bit. The “searching” span is where the animated gif is created, and displayed as needed. The reason that the node after the “left” span is wrapped in a div has to do with the way different browsers render background images on a text field. Suffice it to say it is incosistent, and much more easily managed on a div.

So here is the CSS:

.tundra .medryxSearch {
	margin: 0;
	padding: 0;
}

.tundra .medryxSearchBox {
	position:relative;
	margin:0;
	padding:0;
	left:19px;
}

.dj_ie .tundra .medryxSearchBox .left {
	top:-1px;
}

.tundra .medryxSearchBox .left {
	background: transparent url('../themes/images/searchLeft.png') no-repeat scroll left;
	width: 19px;
	height: 24px;
	top:0;
	margin:0;
	left:-19px;
	padding:0;
	position: absolute;
	cursor:pointer;
}

.tundra .medryxSearchBox .right {
	background: transparent url('../themes/images/searchRight.png')
		no-repeat scroll right;
	height: 24px;
	top:0;
	right: 19px;
	width: 10px;
	margin:0;

	padding:0;
	position: absolute;

}

.tundra .medryxSearchBox .searching {
	background: transparent url('../themes/images/searching.gif') no-repeat scroll right;
	width:16px;
	height:16px;
	padding-left:16px;
	position:absolute;
	top:-9999px;
	right:25px;
	margin-top:4px;

}

.dj_ie .tundra .medryxSearchBox .right {
	top:-1px;
	right:18px;

}

.tundra .medryxSearchBox div {
	background: transparent url('../themes/images/search.png') repeat-x	scroll bottom;

	margin:0;
	padding:0;
	height: 24px;
}

.dj_ie .tundra .medryxSearchBox input {
	font-weight: 500;
}

.tundra .medryxSearchBox input {
	color: white;
	background: transparent;
	border: 0;
	font-size: 13px;

	margin:2px 0 0 5px;
	padding:0;

	height: 24px;
	outline-color: -moz-use-text-color;
	outline-style: none;
	outline-width: medium;
	text-shadow:0 1px 1px #323232;
}

Okay, that’s a lot of CSS. Let’s break it down. First off, it uses the standard dojo method of “namespacing” classes by prefixing each declaration with “.tundra”. (On a side note, I think I should probably break this out into pieces that apply to any theme vs those that are more themeable. Those that apply regardless of theme should not be prefixed, but this is a work in progress…)

A couple cool tricks you may not know about. Instead of beating yourself over the head with the different ways browsers render background images, and fighting the different interpretations of margin and padding, you can use a .dj_ie prefixed class to tweak you CSS class for IE. Dojo quietly adds the dj_ie class to the HTML tag of the page, so this is always the top-most class. Brilliant!

Notice also that I have nested the meat of my widget within my DOM node. This is so I can allow the positioning of the widget to be relative (actually absolute) to the DOM node and get positioning anywhere on the page. CSS position:absolute interprets “absolute” as placement on the page not within the containing element, unless the containing element is relatively positioned.

Okay, now we have complete the first step of widget creation — we’ve made a pretty widget that loads nicely when placed on a page.

Now lets handle the “prompt” bit. I want some variable text presented to the user, but when they click on the field it should disappear, but if they leave the field empty, it should re-appear. So on my widget declaration we add prompt:"Search", and users can easily override this. Then in postCreate, we set this.inputField.value = this.prompt. Finally we wire the onclick event from the inputField to the clearPrompt method on the widget, which clears the input field of the prompt via a string.replace — this.inputField.value.relplace(new RegExp(”^” + this.prompt), “”). Then when a search is done, we could reinstate the prompt. Certainly if there is a blur-event and the field is empty, we should reinstate the prompt.

The last “easy” thing to do before we have to really have to start thinking hard about the “backend” of this widget is to connect the magnifying glass icon to a menu of options. Since I want this to be pretty flexible, I simply check if the declaration has a dijit.Menu as a child, and if it does, it wires it up correctly. So the usage would be something like:

<div dojoType="medryx.widget.SearchBox" style="width:300px;">
  <div dojoType="dijit.Menu">
    <div dojoType="dijit.MenuItem">Google</div>
    <div dojoType="dijit.MenuItem">Yahoo</div>
  <div>
</div>

So how do I wire this up? This took me a few minutes to figure out. The first thing to realize is that the child widgets have not been instantiated yet when the parent is being instantiated by dojo.parser.parse(). So I can’t attach to this yet. My options are to wait until my child widget is instantiated (by connecting to the menu in some way in startup() and perhaps making this widget a “Container”). But that seemed like a lot of work. My first attempt was to simply call dojo.attr and add the targetNodeIds value and leftClickToOpen values to the DOM node of the dijit.Menu before it has been parsed. And it works! In FF3. Unfortunately IE6 doesn’t like this and throws an “Unexpected method or property access” exception. Damn!

So here is my trick: I parse the dijit.Menu myself! So, do a dojo.query(”[dojoType='dijit.Menu']“, this.srcNodeRef) to get the DOM node. Then call dojo.parser.parse on the returned node. Then use the return value from the the call to parse to get a handle to my Menu widget. Then use widget.attr to change leftClickToOpen to true, and then call bindDomNode to my optionsNode (the magnifying glass). So the code looks like this:

var menuWidgets = dojo.query("> [dojoType='dijit.Menu']", this.srcNodeRef);
var menuNode = menuWidgets.length && menuWidgets[0];
if (menuNode) {
// DOESN'T WORK IN IE6!
//	dojo.attr(menuNode, "targetNodeIds", dojo.attr(this.optionsNode, "id"));
//	dojo.attr(menuNode, "leftClickToOpen", "true");
//parse the menu now so we can get a handle for it
	this.menu = dojo.parser.instantiate([menuNode])[0];
	this.menu.attr("leftClickToOpen", true);
	this.menu.bindDomNode(this.optionsNode);
 
	this.connect(this.menu, "onItemClick", "menuItemClicked");
}

And now you have a “search menu”.

Okay, enough procrastination, time to think about the model. But wait, before we do, we should think through how the results might be displayed. With the almost infinite flexibility of the dojox.grid, and its inherent connection to dojo.data via DataGrid, I think it would be silly to roll my own table to display the results of the search. I imagin wrapping the grid into some kind of popup node that display once a search is initiated. That is work to be done.

But if I want to use dojo.data, how will I meet my need for allowing searching multiple different fields of a store, or multiple different data sources? I pondered this for a day or two, with lots of complicated ideas. And then it came to me: a composite data store. Call it medryx.data.SearchStore. This store takes other dojo.data stores as part of its initialization parameters. And for each store, you can describe a way to create a “search” query on the store. Heck you can even supply the same store multiple times, each with a different “search strategy”. When the Search widget calls fetch, the query will be {searchTerms:"foo", searchMethod:"*"} where searchMethod is the value of any item that was selected from your menu of search methods, the searchTerms are the just the text from the search box. The composite SearchStore will initiate a fetch on each of its member stores, using the “search strategy” to create store-specific queries. As each store returns we’ll glue the results together, with some indication as to where they came from, and the DataGrid will know how to display them. Its actually quite flexible.

So, now all I have to do is built the SearchStore, subclass the DataGrid to display these results flexibly but correctly, add in “more…” type functionality, and I’m done. Jeez. All this just for a lousy little search field. Man how I wish someone had done this before me! Well, I’ll finish writing this widget, and then post another entry on how it goes, and hopefully include the source so you all won’t have to go through what I have!

The newest and most complete store in the dojox.data arsenal is dojox.data.JsonRestStore. Its primary purpose is to make interaction between a RESTful data source and dojo.data seamless and simple, and it accomplishes that goal with ease. But a quick peek under the covers and you will find that you may very well be using JsonRestStore for all your Json dojo.data needs. This article aims to show you how you can use JsonRestStore and its more advanced features like lazy entity and property loading and client side filtering, and how you can adapt JsonRestStore to any data source that provides JSON data to your client.

RESTful web services map CRUD services to the HTTP protocol: Create = Put, Read = Get, Update = Post, and Delete = Delete. You can easily sort through the data that would be passed to each of these services as well — Create gets the full data of a new object without an id, Update gets the full data of the updated object with an id, read gets an id who’s object should be returned and delete gets an id whose object should be deleted. Its not rocket science. But if you broaden your thinking from REST, you’ll see this is essentially an API for interfacing with server side data stores. REST is simply one way of implementing it. Instead of REST, you could use an RPC service, or any other service, so long as you could map the service to this API.

JsonRestStore expects you to provide it a service that implements the REST API. The easiest way to do this is to simply specify a target url as a String. JsonRestStore will then automatically send that URL the commands in the API using the REST protocol described above. However, instead of simply providing a target URL, you can actually initialize a JsonRestStore with your own service. The service is simple an object that has the following methods that each return a dojo.Deferred:

• A directly callable function that takes an id or a query/queryOptions pair and returns the results of the query
• A put function that takes an id and value and creates the value at the given id
• A post function that takes an id and value and appends/updates the value at the given id
• A delete function that takes an id and deletes the value at that id

It is important to know that if you do provide a custom service, you must still provide a unique string as the “target” attribute for the JsonRestStore. As I will explain later — JsonRestStore actually manages interdependencies between objects and between data stores — and the target attribute is how it uniquely identifies one store from another, as well as how it identifies which store a particular object belongs to.

So lets start coding. First lets define a custom service that we will be providing to JsonRestStore. Lets say we are building a system for paging physicians. Each physician has a pager. We’ll start simple (this is illustrative code, not real code!):

var mdService = function(query, queryOptions) {
return dojo.xhrGet({url:"queryUrl.php", handleAs:'json', content:{query:query, queryOptions:queryOptions}});
}
mdService.put = function(id, value) {
mdService.post(id, value);
}
mdService.post = function(id, value) {
return dojo.xhrPost({url:"actionUrl.php", handleAs:'json', content:{action:"update", id:id, value:value}});
}
mdService['delete'] = function(id) {
return dojo.xhrPost({url:"actionUrl.php", handleAs:'json', content:{action:"delete", id:id}});
}

And that is it. Now lets feed this service to JsonRestStore and create new store for our physicians:

var mdStore = new dojox.data.JsonRestStore({target:"md", service:mdService});

You can now use this store as you would any other store — feed it to a tree, or a grid, or to any other dojo.data enabled widget, and everything just works. But it gets better. Much better.

As I stated before, lets build our paging system. In our system we have about 10 pagers and 30 physicians. At any one time, each pager is assigned to 0 or 1 physicians. So in addition to being able to navigate from physician to pager given a list of physicians, we need to be able to navigate from pager to physician given a list of pagers. So lets build that store:

var pagerService = function(query, queryOptions) {
return dojo.xhrGet({url:"pagers.php", handleAs:'json',content:{query:query, queryOptions:queryOptions}});
}
pagerService.put = function(id, value) {
  var d = new dojo.Deferred();
  d.callback(); //put is a noop for pagers
  return d;
}
pagerService.post = function(id, value) {
return dojo.xhrPost({url:"updatePager.php", handleAs:'json', content:{id:id, value:value}});
}
pagerService['delete'] = function(id) {
  var d = new dojo.Deferred();
  d.callback(); //delete is a noop for pagers
  return d;
}
 
var pagerStore = new dojox.data.JsonRestStore({target:"pager", service:pagerService});

This where the magic starts. Lets create some data:

[
  {"id":"1", "name":"Maulin Shah, MD", "pager":{$ref:"pager/1"} },
  {"id":"2", "name":"Doogie Howser, MD", "pager":{$ref:"pager/2"} }
]

and here are some pagers

[
  {"id":"1", "md":{$ref:"md/1"}, "number":"12345", "name":"Yellow Pager"},
  {"id":"2", "md":{$ref:"md/2"}, "number":"54321", "name":"Triage Pager"}
]

Notice the “$ref” notation. This is JsonSchema style notation for indicating references to other objects. This lets you simply define the reference without loading the referenced object, and the referenced object will be lazily loaded when necessary. Notice the the $ref is essentially a “path” to the referenced object — or in less RESTy lingo, its the “class” of the referenced object along with its id (or perhaps its the “table” of the referenced object along with its primary key, if that’s the lingo that floats your boat).

Note that you do not have to make a reference lazy, or you make the reference partially lazy. I’ll go over these concepts a bit later.)

So what happens when we load some physicians into a grid?

<table border="0">
<thead>
<tr>
<th>Physician</th>
<th>Pager</th>
</tr>
</thead>
</table>

This would create a grid that looks something like:

Woops. Not quite what we wanted. The problem here is that the pager object has no toString() method. So we have two options to solve this. Thie first involves using a schema and object prototype with your store.

Each store accepts a “schema” attribute as an initialization parameter. The schema is a JsonSchema style object describing the properties of the json objects that are returned for that store. In addition to the usual “properties” attribute of the schema, the schema can have a special “prototype” attribute that defines an object prototype that is used when the object of this class are instantiated. Thus you can mixin your own custom methods or properties into each instance of objects managed by your store. So lets create the prototype for the pager store:

 var pagerSchema = { prototype: {
      toString:function() {
        return this.number;
      }
    }
  }

Feed this to the store when it is created, and we’ve solved our problem:

  var pagerStore = new dojox.data.JsonRestStore({target:"pager", service:pagerService, schema:pagerSchema});

Now your table will look like you expect:

(The second way to solve this problem involves over-riding the getValue method of JsonRestStore to allow dot-path notation. I’ll describe that in a future post that discusses my BeanStore extension of JsonRestStore.)

So what happened behind the scenes in rendering this table? The grid called physicianStore.fetch with no query (which is the same as “fetch all”). The Json described above was returned, with its references to the pager objects. JsonRestStore magically instantiates a new first class javascript object for each item in the Json, using the prototype in our schema as the prototype for the object. When mdStore.getValue(item, “pager”) is called by the Grid, the mdStore initiates a synchronous lazy call to pagerStore’s fetchItemByIdentity method which returns the pager, instantiated with the pager schema, which has a toString method which returns the number.

You could see how laziness could get out of hand very quickly. If we have a hundred physicians, we would have 100 synchronous calls to the server to lazy load pagers. That is not ideal. Lazy loading should really be reserved for more of a master-detail type usage when most lazily referenced items are never actually returned. So how could we make this more efficient? We can modify the data returned in our physicians data to include the pager number as well as a reference to the pager.

[
  {"id":"1", "name":"Maulin Shah, MD", "pager":{$ref:"pager/1", "number":"12345"} },
  {"id":"2", "name":"Doogie Howser, MD", "pager":{$ref:"pager/2", "number":"54321"} }
]

Now this is really cool. Now when rendering the grid, we incur no new requests to the server, since the pager number is already available locally. But now lets say we added a function where clicking on a row of our grid popped up an alert with the name of the pager. Since the name is not available, it would now lazily load the pager (i.e. call pagerStore.fetchItemByIdentity) and flesh out the rest of the pager object. Pretty clever eh?

In this article I have touched on just a few advanced features available with JsonRestStore — using a custom service, using a schema to provide an object prototype, lazily loading objects, and sparsely loading objects. But there is plenty more goodness to be found in this store! Stay tuned for an article on client-side caching and other great features of JsonRestStore!

Another nifty feature of MedryxORM is dot-path support. Since this ORM is all about relationships beween domain objects, it only makes sense to give the you access to traversing your relationship paths. This is used in two places primarily: getValue and properties. Let me give you a couple examples.

Lets say you are making a table of patients. You want to display their name, room number, and insurance. The object model looks something like:

In javascript, you would represent this model as:

 
dojo.declare("model.BaseEntity", null, {
	id:new medryx.orm.Identifier()
});
 
dojo.declare("model.Patient", model.BaseEntity, {
	firstName:new medryx.model.Property({label:2}),
	lastName:new medryx.model.Property({label:1}),
	visit:new medryx.model.Association({associationClass:"model.Visit"})
 
	getDisplayName:function() {
		return this.getLastName() + ", " + this.getFirstName();
	}
 
	queries:[
		{name:"AllPatients", properties:["*", "visit.*"]}
	]
});
 
dojo.declare("model.Visit", model.BaseEntity, {
	room:new medryx.model.Property(),
	bed:new medryx.model.Property(),
	insurance:new medryx.model.Association({associationClass:"model.Insurance")
});
 
dojo.declare("model.Insurance", model.BaseEntity. {
	companyName:new medryx.model.Property(),
	groupNumber:new medryx.model.Property()
});
 
entityManager.registerClass("model.Patient", "Patient");
entityManager.registerClass("model.Visit", "Visit");
entityManager.registerClass("model.Insurance", "Insurance");

So after running all this code, you now have a managed model that you can query (assuming you have set up your perisistenceService) and display the results, lets say in a grid. So lets define a grid:

	var view = {
		cells:[
			[
				{name:"Name", field:"displayName"},
				{name:"Room", field:"visit.room"},
				{name:"Bed", field:"visit.bed"},
				{name:"Insurance", field:"visit.insurance.companyName"},
		]]
	};
	var grid = new dojox.grid.DataGrid({
		store:entityManager,
		query:{},
		queryOptions:{entityClass:"Patient", namedQuery:"AllPatients", properties:dojo.map(view.cells[0], function(column) {
			return column.field;
		})},
		structure:[view]
	}, dojo.byId("patientsGridNode");

And that is it. So what have we done here?

As you can see, I’ve declare 3 simple classes. Each has some simple properties, and for kicks, I’ve declared a simple getDisplayName() method on my Patient.

I’ve defined a named query that get all patients from the server, and for each, returns all properties of the Patient and the patient’s associated Visit. (Note that this means the server will return the id of the insurance association, but not any attributes of the insurance company (i.e. the companyName and the groupNumber are not returned.)

Next we declare a simple dojox.grid.DataGrid. (This uses the grid model from Dojo 1.2, where DojoData has been removed and essentially plunked into DataGrid itself.)

As you can see, the “field” name in the view attached to the grid’s stucture uses “dot-path” notation to get to the desired field to display. Since our query is returning Patient’s, we need to navigate from the patient to the desired display field. This fails gracefully, retuning null if any part of the path does not exist.

Now lets look at the queryOptions in the grid. First, see how there is no query? I don’t absolutely have to have one — which means by default I will show all patients. The queryOptions are used to tell the entityManager which query to run. We then send in a new list of properties for the entityManager. Since we know that we will certainly need all the displayed fields in our data set, we can simply map() the view into a list of properties that we would like to instruct the entityManager to fetch.

Here is where things get really nice. Without any work on your part, the entityManager will have only made one call to the server when all of this is done, and it will have all the data you need to display your grid.

Now lets say you wanted to create a “detail” view that allowed people to see the patient’s insurance groupNumber if they clicked on the insurance information. Simple!

dojo.connect(grid, "onRowClick", function(e) {
	var patient = grid.getItem(e.rowIndex);
	alert("GroupNumber: " + patient.getVisit().getInsurance().getGroupNumber());
});

Notice how there is no query here, or anything else for you to write! You just use your getter, and you get lazy initialization of the groupNumber for free!

In my last post, I talked about a system I have built for managing domain objects on the client side, and making communication with services that communicate with the server more opaque to the average developer.

In this post, I would like to further describe this system, which for the time being I have called MedryxORM. Aside from the “free” lazy loading and relationship management you get with this system, you get a simplified method of handling queries. While there is a wrapper that allows for usual “fetch” access, there are several advanced features you get with the ORM.

First, lets look at what is different from a usual fetch and a fetch done from the ORM. Since the ORM manages multiple domain object classes (Managed Classes), you must specify the “root” class of your query. This is analogous to the “FROM” statement in SQL. The results are always a list of Managed Entities of the root class of your query.

Using the native api, you run a query as follows:

entityManager.query("Patient", {query:{lastName:"Smi*"}, sort:["lastName", "firstName"]});

and using the dojo ReadApi you would do:

entityManagerStore.fetch({query:{dischargeDate:null, lastName:"Smi*"}, sort:["lastName", "firstName"], queryOptions:{entityClass:"Patient"});

(I’ve left out some of the details for brevity).

So far, nothing to earth-shattering. But the ORM gives you a couple of neat features that can simplify your queries. Queries can be either client-side or server-side. A client-side query is always a subquery of a server-side query that has already been executed. So you can do filtering and sorting of the server-side query without having to re-query your server or marshall more data back and forth. This is quite useful for most queries — up to a point. If you have an enormous number of records that make server-side paging necessary (i.e., your server query does not return the entire set of records), then client queries don’t work. But this is really an exception to many circumstances. In my world, I will usually have on the order of a couple hundred patients and a couple dozen physicians as my major queries. I can easily load all that data into the client on a single call. But your results may vary.

So how do you set this up? In a further attempt to keep end-developer (i.e. page developer and perhaps widget-developer) usage as simple as possble, we have created a concept of “Registered Queries”. This is similar to PreparedStatements in Java or Stored Procedures in SQL. Or, perhaps even more closly, this resembles NamedQueries in Hibernate.

A Registered Query is simply a query that is created when the Entity Manager is first configured. Its just another field in the mapping of an entity. You register a query in one of two ways:

entityManager.registerQuery("Patient", "AllPatients", {query:{dischargeDate:null}, sort:["lastName", "firstName"]});

or you can include a “queries” property on your mapping:

dojo.declare("model.Patient", null, {
	firstName:null,
	lastName:null,
	//snip...
	queries:[
		{name:"AllPatients", parameters: {query:{dischargeDate:null}, sort:["lastName", "firstName"]}}
	]
});

Once you have done this, then executing your query becomes even easier:

entityManager.query("Patient", {query:{lastName:"Smi*"}}, "AllPatients");

This will first load all patients from the server with dischargeDate = null, and then filter them on the client to all those whose lastName starts with “Smi”. Suppose you then want to filter on a different name, or sort in a different order? Just run query() again, with your new parameters. This also enables easy client-side paging.

To use the ReadApi, you add a registeredQueryName parameter in either your keywordArgs or query args or query options, and it will map correctly to the EntityManager’s query method.

entityManager.fetch({query:{lastName:"Smi*"}, queryOptions:{entityClass:"Patient", root:"AllPatients"}});

Since the EntityManager is built on the concept of sparsely-populated objects that get their more complete representation lazily or eagerly if you know you will be using them, all query operations also support a “properties” queryOption. This is similar to the properties valuein loadDeferred described in the previous post. You provide a list of properties that you know you will be using. If this is a server-query, then this is passed to the server during the request. However, if this is a client query, and the properties passed are different than those in the server query, then the results of the query are iterated, and each object that is missing a required property has those properties eagerly fetched from the server. This is transparent to the end user. (In fact, if your sub-query contains fields that are not yet loaded from the server, then this loads those in one request prior to running the sub-query). This functionality all assumes that your server can handle a query where the identifier field is passed an array of id’s to load.

The Dojo(x) Grid widget is fabulous. It makes creating incredibly fast, functional tables a breeze.

But periodically, you want to extend the Grid. For example, right now I have a need for “auto-filter”-like capability for a grid. Right click on a header row, show a dropdown of values, and let me filter the grid on that value. This isn’t a universal need, but certainly one that has been asked for on the dojo forum.

Now, aside from actually building the auto-filter, I’d like to propose creating a way of systematically extending Grid. Perhaps a concept of “plugins” would work. I could see something like this:

<div dojoType="dojox.Grid" store="store">
	<div dojoType="dojox.Grid.AutoFilterPlugin"></div>
</div>

Each plugin could decide how it incorporated itself into the grid. Mine, for example adds an optional attribute to each column in your structure called “autoFilter” which, if you set to true, makes autoFiltering just work if you are using a dojo.data store. And if you aren’t you simply provide a hook to your own custom “filter()” method, and it still works.

In any case, since the above declarative style plugin capability doesn’t yet exist (though I am thinking hard about how to do it), my AutoFilterPlugin is actually a mixin you use in declaring your own subclass of Grid like so:

	dojo.declare("MyGrid", [dojox.grid.DataGrid, medryx.widgets._GridPlugins.AutoFilter], {});

Then you can create your layout with the autoFilter field:

var layout = [
	{name: 'Alpha', field: 'col1', autoFilter:true},
	{name: 'Beta', field: 'col2'},
	{name: 'Gamma', field: 'col3'},
	{name: 'Delta', field: 'col4', width: "150px", autoFilter:true},
	{name: 'Epsilon', field: 'col5'},
	{name: 'Nexilon', field: 'col6'},
	{name: 'Zeta', field: 'col7'},
	{name: 'Eta', field: 'col1'},
	{name: 'Omega', field: 'col8'}
];

The rest of the usage is the same, and you end up with a Grid that looks like this:

A little explanation on how this works: The plugin over-rides postCreate and connects the onHeaderContextMenu event to a showAutoFilterDialog method in the plugin. That method opens a TooltipDialog with the select list of values. Chaning the value on the select list fires updateFilterValue which in turn first this.filter. The DataGrid already has a filter() method. If you are not using DataGrid, you need to mix in a method called filter() as well (in the dojo.declare of your subclass is where this would go).

How does the tooltip know what the values of the column are? This was the trickiest bit to figure out. Because the get() method is called from the context of a cell, not from the context of the grid, you couldn’t do a simple over-ride with a call to this.inherited(arguments) . That took a while to figure out. So instead, I create the modified get() method in my postCreate method. This runs the original get() and then stores the value before returning. Of course, this implies that you can only filter on values that the grid itself knows about and has rendered. For me that is fine. You could always create a different way of getting your filter values (for example by querying you store) if you wanted to.

So, if you understand any of that mess, I applaud you. And without further ado… here is the code:

/**
 * This is an aspect to your grid that will add in AutoFilter ability to the grid
 * You must provide the filter function for your grid. If will be called with an object
 * {
 *      fieldName:filterValue,
 *      fieldName2:filterValue2
 *      
 * }
 * to attach this plugin to your grid you mix this in to your custom grid (last)
 * and then define a filter() method
 * 
 * TODO: [Blank] and multiple filters is still a little buggy
 * 
 * @author maulin
 */
dojo.provide("medryx.widgets._GridPlugins.AutoFilter");
dojo.require("dojox.lang.aspect");
dojo.require("medryx.widgets._GridPlugins._AutoFilterTooltip");
 
dojo.declare("medryx.widgets._GridPlugins.AutoFilter", null, {
 
	constructor:function() {
		this._filterableValues = [];
 
	},
 
	/**
	 * return a list of values that this column can be filter by,
	 * sorted alphabetically
	 * @param {Object} column
	 */
	getFilterableValues: function(column) {
	   	var values = this._filterableValues[column.index];
		return this.cleanFilterValues(values);
	},
 
	cleanFilterValues:function(values) {
		var unique = {};
 
        //get rid of duplicates
        return dojo.filter(values, function(value) {
            if (!unique[value]) {
                unique[value] = true;
                return true;
            }
            return false;
        }).sort();
	},
 
	/**
	 * add a string value to the list of all filterable values
	 * for a column
	 * @param {Object} column
	 * @param {Object} value
	 */
	addFilterableValue:function(column, value) {
		var columnIndex = column.index;
		if (!this._filterableValues[columnIndex]) {
			this._filterableValues[columnIndex] = [];
		}
		if (typeof(value) !== 'undefined' && value !== null) {
			this._filterableValues[columnIndex].push(value);
		}
	},
 
	updateFilter:function(filter) {
        if (this.filter) {
			this.filter(filter, true);
		}
    },
 
	updateFilterValue:function(column, value) {
		var filterField = {};
		filterField[column.field] = value;
 
		if (!this.filterFields) {
            this.filterFields = {};
        }
        if (filterField) {
            dojo.mixin(this.filterFields, filterField); //makes any other filters "sticky"
        }
 
		this.updateFilter(this.filterFields);
	},
 
	getTooltip:function(column) {
		if (!this.tooltips) {
			this.tooltips = [];
		}
		if (!this.tooltips[column.index]) {
			this.tooltips[column.index] = new medryx.widgets._GridPlugins._AutoFilterTooltip({column:column.name});
			dojo.connect(this.tooltips[column.index], &