I’ve not intentionally delayed my response to Richard Wallis’s recent comments/reply. I’ve been a little busy rolling out our “3.1″ feature-set over at aadl.org. I plan a reflective post on “Zen and the Art of OPAC Wrangling” in the next week and another geeky post in which I 1apse 1nt0 b1nar100101 10101001 100101. Damn! Did it already!

But, as it turns out, I’m glad I let this conversation simmer because a number of people have added their thoughts to the mix as well, and I’m really pleased with the fact that this issue is getting some attention. As I’ve said before, what we are allowed to and able to do with our ILSs will determine how pertinent libraries are over the next ten years and beyond. We need to get everything we want from our vendors and we need it fast.

Originally, Richard had suggested that if we were given open read-only access to the data, we would not “[understand] what the data in the tables actually means”, to which I replied, “This does not give enough credit to those of us who hack away on these systems every day. We’ll figure it out.” Want proof? Richard’s response:

Looking at the issue from John’s end of the telescope it sounds so obvious and simple. Imagine looking at it from as a support analyst’s point of view. From her end of the telescope she can see [in Talis’ case potentially 100+ Johns hacking away on their systems every day – a thought to drive you straight toward the caffeine in the morning!

Let’s just put aside the fact that 100+ Johns, alone, would probably drive you straight toward the closest stiff drink. I’m really not sure how to take this comment, except to say that support analysts are there to “support”. When I talk about changing the way vendors and libraries interface, this falls right into that slot. Those support analysts need to be retrained (reprogrammed?) to be able to think about the caller on the other end of the phone as an equal–that way, they become an advocate for the customer, not just a problem solver. I just can’t buy into the thinking that we can’t be granted the essential right to access our data because that might make it harder to support. That is a challenge for vendors to deal with and I have little interest in how comfortable support analysts are with their job.

The development and support needed to provide an ILS that was capable of being relied upon by 100’s of academic and public libraries, predicated the customer vendor relationship we are now used to.

To some extent, yes, and I’ve mentioned before that libraries are partially to blame for enabling vendors to pursue their current business models. But the prevailing winds were different, even a few years ago. I’m my mind I tend to liken ILS vendors to the recording industry. A colossal shift is happening within libraries, due in large part to the emergence of web 2.0 technologies. This shift is wresting control of the development process away from vendors and putting it in the hands of the library coder. I believe this will eventually blossom into a large library developers network (and I’m hoping AADL can play a key role in getting that moving). Under current business models, it just doesn’t make sense for vendors to allow that to happen. The arguments against any component of my bill-of-rights have seemed tenuous at best.

Read Paul & Ken’s whitepaper and you will see that we at Talis, believe that things are due for a change, and not a small one; If Libraries are to continue to matter, which we believe they must. We Libraries, librarians, ILS & other system vendors, we will have to embrace and promote that change.

I’m not at all worried whether libraries will matter. I’m worried to what degree they will be pertinent in the lives of our constituents. What vendors need to be asking themselves is, “If we don’t change, will we still be in business?” I see rough times ahead for vendors as the gravy train comes screeching to a halt. I’m glad for Richard that Talis seems to be jumping the rails for a workable business model–I just think they ought to be committed to providing the four basic rights I’ve proposed to libraries. Libraries, in turn, need to be insisting upon them when it comes time to migrate.

You want your OPAC to differentiate your Library; you want your users’ account information to appear in their favorite portal; you want to communicate with your customers by email, RSS, SMS Text, IM, etc.; if all the core functionality was available via a software service you could – and would you care how and on what it ran? I think not.

I think I probably would. As you know, one of my assertions is that we ought to have administrative control over the box. We also need to be able to integrate the box into our enterprise. That means being able to deploy software such as our own backup and restore agents on the server. We’d need access to logs, the ability to monitor processes, etc. Telling an organization to entrust it’s livelihood to you without allowing them to look under the hood is not right. Even philosophically, it doesn’t jive with the idea of web 2.0.

It’s hard not to notice that there is a measure of bitterness directed toward vendors. I myself have been quoted as saying vendors are “money hungry” (when, in fact, my point had been that vendors are perceived as money hungry. An important distinction). At this point, I think it’s important that both sides approach the other in a manner that does not incite defensiveness. In my opinion, the debate over how much we’ve been paying vendors is not important to this discussion and is probably counterproductive. That’s a matter for market forces (and accounting) to deal with. I want this discussion to evolve as it has and hopefully lure other vendors into it as well.

So that’s my response to Richard’s response to my response to his response to my ILS customer bill-of-rights. Headache anyone?

I have to admit that getting this to work was born out of my own selfish motivations. I really wanted a way to get a jump on requesting some of the more popular items and thought, short of mugging someone in collection development, this would be the most fruitful way of going about it.

In the first iteration of this enhancement, RSS feeds were only available for hitlists generated by a keyword search. I do 90% of my searches using keyword, but thought that the feature really ought to be extended to all search types. The problem with doing so, however, is wrangling enough information from the catalog and URL to determine a) what kind of search it is, and b) what the search term is. The III catalog poses a couple of challenges to this. First, the III catalog seems to oscillate between GET and POST variables as you navigate through the OPAC. This makes it quite difficult to reliably put your finger on the right data. Second, each search type seems to be handled in its own special way. Again, this makes it difficult to generate reliable results.

So the solution was to create a set of “rules” in the code that is applied to each page of the catalog. These rules basically score the content and return a “best guess” which can then be applied toward RSS generation. It’s goofy, but again, that’s what we have to work with!

The result looks something like this in the catalog:

And something like this in NetNewsWire (My RSS reader):

I had an evil thought to write a piece of software that loops continuously, automatically requesting new items in an RSS feed. That way I’d be sure to be the first one in the hold queue. I don’t think doing that is ethically sound, but it does raise the question, what are we going to do about the people who find ways to take advantage of this new technology at the expense of others? Library 2.0, like Web 2.0 is most certainly going to widen the gap between the techno-savvy and non techno-savvy. The ‘nons’ are going to get left behind. It’s our responsibility to coax those users into using these services and technologies. We’re probably in the best possible position to do that because local citizens and community members are the main users on our sites and they have a vested interest in the services we provide.

I had a pleasant surprise this morning when I read that Casey had extended my XMLOPAC class. He’s written a function that takes a search term, then returns an array of bib-nums for that search. He’s hoping to create clustered search results, and his function may be a component of that. Yes, necessity is the mother of all invention.

Casey’s comment sums up why we go through the trouble to squeeze out every last bit of functionality from our OPAC, “Because problems are meant to be overcome.” I’d just like to see the odds stacked in our favor. Imagine what we could accomplish if our ILS actually enabled us to do this sort of work.

Thanks Casey!

I’ve incorporated his code into the class, with some minor changes. I also found another instance where III’s XML is malformed and added a fix for that. The complete package can be found here.

Another update to the III XMLOPAC class. The function get_opac_data() now returns the number of holds a bib-item has. Originally I thought this wasn’t going to be possible since I didn’t see anything overtly obvious in the XML result set, but I did see a bunch of weird “HOLD” elements with a lot of useless associated data. My hunch was that each one of these represented a bib-level hold, and I was right! Anyway, by tallying the number of these hold objects in the XML, I was able to determine the number of holds for that record.

Incidentally, this is what the nonsense looks like to SimpleXML:

[21] => SimpleXMLElement Object (
   [VARFLDPRIMARY] => SimpleXMLElement Object ( 
    [VARFLD] => SimpleXMLElement Object (
       [HEADER] => SimpleXMLElement Object ( 
        [TAG] => 8
        [NAME] => HOLD 
        [LABEL] => Hold 
        [SEQUENCENUM] => 12
      )
      [DisplayForm] => SimpleXMLElement Object ( )
      [RTL] => 0
      [FIELDDATA] => SimpleXMLElement Object ( )
    )
  )
)

What a mess. Anyway, it’s workable, the number of holds is now passed back, keyed by “holds”. If there are no holds, there will be no holds key in the result.

Here is a great way to quickly convert III item numbers to bib numbers. You could stick this function in your php script and go to town:

(PHP 4 >= 4.3.0, PHP 5 with fopen_wrappers)

It's really so simple, I can't believe I didn't think of it before. I'm actually using it now to cache the results in a database so I'm not always chewing on the OPAC to get the result.

Enjoy!

[update]

In response to your comment/question Ryan, this is verbatim the code I'm using to cache the bib numbers. First, this is the table I'm using:

[mysql]
DESCRIBE item2bib;
+---------+------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+---------+------------+------+-----+---------+----------------+
| id | int(8) | NO | PRI | NULL | auto_increment |
| itemnum | varchar(9) | NO | MUL | | |
| bibnum | varchar(9) | NO | | | |
+---------+------------+------+-----+---------+----------------+
3 rows in set (0.00 sec)
[/mysql]

This is the code (which is actually inside a class, so you'll need to strip the class-specific stuff, but you get the general idea):

[php]
public function get_bibnum($itemnum) {
$db = DB::connect($this->dsn);
$bibnum = $db->getone("SELECT bibnum FROM item2bib WHERE itemnum = '$itemnum' LIMIT 1");
if ($bibnum) { return $bibnum; }
$item_url = 'http://' . $this->opac_server . '/record=i' . $itemnum;
$item_rec = file_get_contents($item_url);
if (preg_match('%record=b(.+?)"><%s', $item_rec, $bnum_matches)) {
$bibnum = $bnum_matches[1];
$db->query("INSERT INTO item2bib VALUES (0, '$itemnum', '$bibnum')");
return $bibnum;
} else {
return 0;
}
}
?>
[/php]

Run some data through and verify that it's working correctly:

[mysql]
mysql> select count(distinct itemnum) as itemnum, count(distinct bibnum) as bibnum, count(*) as total from item2bib;
+---------+--------+-------+
| itemnum | bibnum | total |
+---------+--------+-------+
| 45809 | 33303 | 45809 |
+---------+--------+-------+
1 row in set (0.00 sec)
[/mysql]

[/update]