To say that working at AADL was a honor would be an understatement. It is a fine, fine institution, filled with bright, intelligent, hardworking, and committed individuals. They have allowed me to grow tremendously, and I feel that I’ve been able to make a significant contribution to AADL in return. So I extend my deepest thanks and gratitude to every last person at AADL.

I am leaving Ann Arbor because I have accepted a position at the Darien Public Library in Darien, Connecticut. My official title will be Head of Technology and Digital Initiatives. I have to say that my pulse quickens when I think about what the Darien Library has in store for the future, and to be part of it is a dream come true. I will be working for Alan Gray, who is currently Associate Director. DPL’s director is Louise Berry.

I will continue to blog here at blyberg.net. I also have no plans to discontinue work on such projects as PatREST, or SOPAC. In fact, they will be crucial components of my future work. Darien Public Library is, like AADL, an Innovative Interfaces customer and I plan to join Darien in their effort to forge a srong and close relationship with III. Though I may have been critical of them in the past, and very well may be in the future, that does not mean that I don’t value a meaningful development partnership with them. I tend to think in terms of “how can what I’m doing now benefit not just my institution, but all institutions?” So you can be sure that I will continue to share our successes, and perhaps more importantly, our failures.

If you currently use my AADL email address to contact me, I suggest you use my posted contact information to get hold of me instead. My personal email address is john@blyberg.net.

This little feature uses data from our opt-in patron history system. Since we've been offering that system for about a year now, we've been able to accumulate over 300,000 data points. This is another great example of how to reuse your existing data to enrich use experience. Like I've said before, you need to use the data you have to its full potential--even if it's not the data's primary purpose.

Ok, so let's look at the nuts and bolts of this. I was able to cobble this together with two SQL queries. Actually, I was able to get the same results with a single nested query, but for some reason, non-cached results took upward of ten seconds, so splitting the query seemed like the right thing to do.

Anyway..

So basically, you can see what's happening here. First I get an array of all the other users who have checked out this item. Then I get a list of items that three or more of those people have checked out and display it randomly (so it's not always the same list). Pretty easy, of course you need to grow that data before it becomes useful. 300,000 records makes the results pertinent, 300 does not.

If you're curious, head on over to AADL's catalog and test it out.. you might stumble on something new.

The next step, of course, in all this SOPAC business is to try to determine how check-out patterns are changing and whether it will have an impact on collection development.

I just snapped this screenshot about five minutes ago of our tag cloud. As you can see, a real cloud is emerging. But let's look at stats for a second.

As far as tagging is concerned, 330 tags have been entered. 97 catalog items have been tagged with 245 distinct tags. 27 people have put tags into our catalog.

Stats for reviews are a little smaller, probably because reviewing an items demands a greater time investment on the part of the user. As of now, 21 reviews have been placed on 21 items by 16 people (4 of whom are staff members). That's about an average of 9 per day. I expect that will pick up as more users become aware of this service.

Of course, compared to a high-traffic social web site like Flickr, these stats are unimpressive, but for a single library, I think they are promising.

Library web site users tend to cross a wider spectrum of computer literacy, so the whole concept of reviewing and tagging may be completely alien to a lot of aadl.org's visitors. We're in a little bit of uncharted territory here, so I'll be sure to do a number of updates as time goes on.

It only took a year, but I finally got permission to go ahead with implementing what I've dubbed "The SOPAC" here at AADL. That would be "cute-speak" for Social OPAC. The SOPAC represents a slew of features that I've wanted to implement for quite some time now. I'm rather excited to see if library users will respond to these tools in an OPAC setting as much as Web 2.0 users have to commercial social networking sites. I'm fairly confident they will. Mainly, I'm relieved that I no longer need to talk conceptually about features I've been planning to build on top of the catalog.

So what is the SOPAC? It's basically a set of social networking tools integrated into the AADL catalog. It gives users the ability to rate, review, comment-on, and tag items. The concept is nothing new, but the nature of our systems do not yield readily to this kind of retrofitting--something I plan to really start tackling in earnest, but that's a topic for another post.

If you're wondering (and didn't know already), AADL's automation system is III which recently released a software package called "Encore" that does some of what the SOPAC does. We did not purchase it, nor do we intend to. Instead we're going to use the money we saved to buy a Lexus. *grin*

Anyway, I've been messing around a bit with Snapz Pro, and thought that since this is a pretty big upgrade to AADL's site, I would include a screencast covering most of the new features. So for those with 15 minutes or so to kill (ignore the screaming kids in the background):

SOPAC Features

The "front door" to the SOPAC is, of course, the main catalog search screen. Drupal's API made development of this code relatively painless. For example, the blocks on the right-hand side use Drupal's hook_block function, making the development of those blocks simply a matter of writing a function that would return the content. In this case, the right-hand column contains search, tag, and review information.

Let's take a look at some of those blocks:

These two blocks represent the contextual nature of SOPAC. The first block appears in the regular SOPAC, while the second is displayed in the use management tools.

Here are some sample review pages:

Top of review page:

Reviews themselves:

Public view of all my reviews:

Private view of all my reviews:

While writing a review, you can simultaneously add tags for the item you're reviewing. Or you can simply tag catalog items without reviewing them. Here are some same examples of the tag system:

Personal tag cloud (My tags in cloud-vew mode):

Personal tag list. This is where users can manage their tags. Delete, modify, view, etc.

List of items in the catalog tagged with "dogs":

Feel free to visit the AADL catalog to tag and/or review some items. You do need an account to create content, but you don't need a library card to get an account, so these features are not limited to cardholders in any way.

Because I feel that this version of AADL.org is a significant milestone, I've made a tarball of the source code publicly available for download. Included in the tarball is our middle-ware "glue" that allows us to interface Drupal with the III server in addition to all the SOPAC code and supporting libraries. Bear in mind that this code will definitely not work out-of-the-box but could definitely be made to work with any III server with XMLOPAC support.

You can download the package here, or from my files section.

Drupal 4.7 sports a number of great new features. I'm most excited about the new search engine which does a much better job of indexing the site and allows users to do an advanced search. Searches now actually return meaningful results. Other features include a new Ajax-enabled content creation system with nifty improvements such as re-sizable text fields, collapsible elements, a file upload system that doesn't require authors to leave their work, and live menu updates. On the development side, these new features are accessible via the new form-handling system. In other words, coders can easily incorporate these new Ajax elements in their own work. Theme developers will be happy with the ability to create an infinite number of regions--nice to achieve that highly-polished CSS look. I think a couple new block types were added as well.

Another great feature is the wiki-style revision system that allows editors to roll-back their work and leave editorial log messages (a very useful feature in large, collaborative environments). Commenting benefits, as well, with the ability of site administrators to manage and moderate multiple entries at once. Finally, Drupal 4.7 supports free tagging. Not something we're using at this point, but, from my point of view, it means that the engine is there for future module work. I have a feeling I'll be using those hooks for some forthcoming feature upgrades on the website itself...

The upgrade was fairly smooth. Drupal ships with an update script which ran flawlessly, but that's the easy part. A fair amount of prep-work was done ahead of time to ensure that all of our custom modules were 4.7-compatible. Basically, this meant updating all of our form-handling code to handle the new system. We also segregated all of our own code and theme information from Drupal's using the multi-site capability. This means that we can easily keep track of our own work without it getting mixed up with the vanilla code-base. This wasn't completely necessary, but it was worth the work because it'll make all future upgrades much easier to do. Doing things this way is also in-line with my philosophy of never touching stock code unless you absolutely have to.

The long and the short of this whole upgrade means that our users will probably not notice a lot of difference, but we're now in a good position to work on AADL 3.2. And that they will notice.

For more info, check out these Drupal videocasts:

• An overview of screencasts
• What's new in Drupal 4.7
• Installing Drupal 4.7
• Upgrading to Drupal 4.7
• Contribute your own Drupal Videocast

Actually, that's fine, I have no problem with that, which means... They should have no problem with me incorporating Google Books into our hit-list. Right?

Now when users search the AADL catalog, they will be given the option to peek inside the books on the hit-list--that is, if there is a record over at Google Books. Basically, the first time that record is displayed in the list, the middleware queries Google Books to see if it has that item in its database. If it does, the middleware makes note of that in a MySQL table so that the remote query doesn't need to be run again. That way, future queries save time and bandwidth.

Looking at the Syndetics offerings next to it, this seems like a much richer and more useful resource. Enjoy!

** Update 1: 8/24/06 9:45 PM **

Ha! It looks like that was short-lived! (Thanks to Ryan for giving me the heads-up), Google apparently doesn't return the favor:

We're sorry...

... but your query looks similar
to automated requests from a computer virus or spyware
application. To protect our users, we can't process your request
right now.

We'll restore your access as quickly as possible, so try again soon. In the meantime, if you suspect that your computer or network has been infected,
you might want to run a virus checker or spyware remover to make sure that your systems are free of viruses and other spurious software.

We apologize for the inconvenience, and hope we'll see you again on Google.

And here I was, trying to be nice by caching the results... Guess we'll have to wait for the API.

** Update 2: 8/25/06 8:50 AM **

So, I think I found a way to fix this. Essentially, the way I was previously determining if Google Books has a record for and ISBN what by using this URL template:

http://books.google.com/books?vid=ISBN$isbn&printsec=frontcover&dq=isbn:$isbn

Now I'm using a different URL that does not return 404:

http://books.google.com/books?as_isbn=$isbn

If there was no record for that ISBN, Google would throw a 404. I think the fact that one IP was requesting so many 404s is what spooked Google, not the retrieval rate. Also, I noticed that I could no longer use wget on the command-line to grab the data--Google would return a 403 (Forbidden). So, my thought was to ditch PHP's file_get_contents for CURL which allows you to spoof a user agent. I took a peek at our apache logs and chose:

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.6) Gecko/20060728 Firefox/1.5.0.6

So, instead of looking like a "virus or spyware", the script now appears, to Google, as an extremely zealous Google Books user. We'll see how long it lasts, but it seems to be holding...

** Update 3: 8/25/06 11:40 AM **

No go, they've blocked us again. I'm sending an email to the kind folks at Google Books, and we'll see if they reply. Until then, I've got a few more tricks up my sleeve... In the meantime, I'll leave the cached information active...

** Update 4: 8/25/06 4:07:PM **

Google scores major points in my book! One of the managers over at Google Books just emailed me to say that he likes the idea of the hit-list links and that he is going to see if they can accommodate these types of queries.

First, two new functions have been added to the service. One provides access to tops (or popular items) lists. The other provides access to the new materials lists. I believe these are significant-enough additions to the service that they merit the 1.1 version number.

The top items query is scalable by result size and can be paginated, just like the search results. In addition, it can be scoped by material type: books, cd, dvd, or bocd (books on CD). When applied to the AADL XSLT, it looks something like this:

The new items query is similar to the top query in that it can be scaled by material type, size and be paginated. You can also search new items using their subject headings--useful for querying those new knitting books, Ed... It looks something like this:

Second, and probably more importantly, I've finally drafted a specification for PatREST which includes an explanation of it's XML schema and some documentation for it's various functions. It's about time, I know. It can be found in the files section, or downloaded here. (PDF)

I'm not going to go in to too much detail about the two new functions in this post because the new documentation contains everything you need to know to get started with them.

We've got a lot in store for the rest of this year. In many ways, it feels like we're just getting started. Time willing, we can continue to serve our community by rolling out the new and exciting features they deserve.

In response to Hennepin's new commenting capability, Talis's Paul Miller asks the question, "Participation is an important part of moving forward. How much better might shared participation be?" What he's talking about is allowing other libraries to access Hennepin's comments in an effort to provide a more enriching search experience beyond Hennepin's OPAC, say, at Ann Arbor, or wherever.

What Paul goes on to propose in a follow-up post is a shared collectionof user participation much like the UK's archival project. This would provide a central database and, presumably a set of web tools to access and interact with the data. Libraries anywhere in the world would have access to add and read content. It would be a shared, collaborative clearinghouse of participation.

I'm all for it--but with some caution. Isn't that what Amazon is now? If you take away the e-commerce, Amazon is a collection of reviews, tags, and ratings on an insanely large amount of material. Interesting? Indeed. Useful? Of course. But I feel the need to point out that libraries are community-based institutions. They are supported by local taxpayers and are run, mostly, by members of the communities they serve. As such, wouldn't we want any social element that is incorporated into our OPAC to reflect the tastes and opinions and personality of our community? I think so, and so does Ed Vielmetti:

...here in Ann Arbor there are a lot of book readers, and it'd really be rather nice to read comments from people who shared the same town with you. If I want to read random untrusted comments from people all over the world there's already Amazon.

I mentioned some of this a while ago, though never specifically addressed the local vs. global social data repository idea. All this is not to say that in addition to community-driven social software we can't access and make use of a shared data store. In response to a comment I made to Paul's second post on this, Fiona Leslie made two very good points that she has seen come up repeatedly.

1. Libraries have reading groups and staff who create reviews, and there is no mechanism for getting those reviews "available" beyond posting them on notice boards, or perhaps on the library's web pages. A few of them have an OPAC implementation that allows the reviews to be viewed in their OPAC.

2. Libraries have few or no reviews, but they see the value in having them, and would really appreciate a kickstart by having access to a shared repository of reviews created by other readers and staff in other areas.

Ok, so I'll address each one separately. First, she is absolutely right, there is no de facto method of integrating participation in to our OPACs. This is compounded by the fact that our OPACs themselves tend to be unalterable beasts and we must rely on vendors themselves to make changes and enhancements to them. Many of you who follow what I write here know that's a contentious issue for me, but I'll keep my hackles down for now and simply remind everyone that this is another reason to demand a few basic rights from our vendors.

But even if we did all have unfettered access to our OPACs, or were resourceful and persistent enough to subvert the plain vanilla foisted upon us, what would a socialPAC look like? Fiona specifically mentions reviews, so lets stick with that for the purposes of this post. Actually, let's not, because it doesn't matter what the implementation looks like--that's the fun part you and your development team get to mess about with when you do your redesign. What's important is what we do with that review data after we get it and the value it adds to the process of searching for material. I'd suggest that the data be made available via two conduits. First would be the end-user interface. That is, the website or application patrons use to both consume and produce the content. How you weave this in to your OPAC is ultimately up to you (or... up to your vendor).

Richard Wallis weighed in ad responded to my comment. He writes:

So what are my assumptions then? Well firstly, the contributions of the citizens of Ann Arbor would be of great use, interest, and value to a far wider audience than just their district. Secondly, contributions to any global pool should be tagged as to their source and type. Thirdly, because of that tagging, selection of results should be able to be via many filters such as library, library authority or institution, library type, country, language etc.

So following through those assumptions in John's situation, I would hope that contributions for my community would add value to the global pot; be displayable locally in isolation as a coherent set; and optionally could be supplemented by those from other appropriate communities around the country and the rest of the world.

I suppose I was a little unclear in my comment on Panlibus. I'd agree with Paul that, once the data is in, it would be nice to have a way to share it with other libraries. And I also agree with Richard that there is a place for supplementing existing data with a larger pool. In fact, I believe we have an obligation, as libraries, to do some manner of both. I envision Ann Arbor's system providing a very lean web service on top of this entire system. Using this model, we will be able to share our community-driven social data beyond our borders. Libraries who do not enjoy the same community support that Ann Arbor, has will still benefit from the data. I believe this distributed approach to generating and maintaining socialPAC data will ultimately offer both redundancy and diversity. The thought of hundreds of libraries making their data available is certainly a more appealing alternative than that of the monolithic database. Metadata itself is an archive--it tells a story.

Fiona's other point was that kick-starting a service may be difficult, especially in communities that are not likely to respond to and prime a service like this. Personally, I think we all might be surprised at the response that socialPACs will have with our constituents. Almost everyone has an opinion, and most people want to share it. That aside, however, Fiona is right. There will be cases where social software is not successful, popular, whatever.

So how do you evaluate your own situation with respect to social social software? Can your community sustain a socialPAC in perpetuity in a manner that will continually reflect a unique attitude and personality? If it can, how do you get it started?

First, you're going to have to be honest with yourselves about the project itself. Do you want to pursue social software because it's cool and hip, or do you really want to change the way your users interface with your collection in profound and personal ways while building a legacy at the same time? You can get a good feel for the level of Web 2.0 participation your community engages in by using existing Web 2.0 services which often let you dial in on specific locales. This may give you a good indication of whether a program like this might be a success.

If you're convinced that your community will support a socialPAC, the next step is to come up with a a design and interface concept that will truly let your existing catalog shine while providing all the Web 2.0 immersion people expect. (easier said than done). This is where the innovators need to step in and start coming up with tangible examples of how this might work. I suspect that most libraries that do this will pursue a variation on a theme, but bear in mind that there are a lot of directions to take this stuff and in the end, it's the one-of-a-kind feature that will give your OPAC its fingerprint. I suppose that vendors will dial in on the more popular and successful models and run with those. I have no problem with that as long as they adhere to the Web 2.0 spirit.

What about the initial "jump-start"? While I was writing this, my thoughts kept drifting to sourdough and I remembered a good friend of mine who, on occasion, liked to bake sourdough bread. It was this person who introduced me to my first sourdough starter. A shapeless blob that lives in your refrigerator and which, on occasion, you feed. At any rate, he was telling me how some sourdough starters have very rich and colorful histories because they have been passed down, literally, through generations. Some are closely guarded, while others have been disseminated and passed around liberally. It's pretty fascinating.

But before I digress completely, Fiona's concern about kick-starting can be addressed if we make our metadata available to systems starting up. Instead of one or two large repositories, however, wouldn't it be great if we could choose from hundreds and all we had to do was send a request against a web service to get started? The tools are in place to allow this kind of interaction. All it takes is a willingness to communicate and share.

I suppose I may be searching for blue sky here, but Web 2.0 gives us a chance to do things properly from the beginning. Ultimately, the successful system will be rich with good data and useful to your patrons. The deeper significance of a unique repository will not emerge right away, but in time, you'll see how data, like buildings themselves, can add to the legacy of a place. Make it available to the larger library community and we'll see some very interesting things, indeed.

At any rate, he's speaking again tonight at AADL. If you're in the area and have the evening free, I'd highly recommend his talk. From the AADL website:

What does the UM/Google partnership to digitize the UM Library mean; what significance will this have for libraries, researchers and the public; and why is this so controversial? Through this project, UM hopes to guide more users: to their local libraries; to digital archives of some of the world's greatest research institutions; and to out-of-print books they might not be able to find anywhere else--all while carefully respecting authors' and publishers' copyrights. This event is the Library Director's program for 2006.

Each year, AADL Director Josie Parker chooses a current topic of community, state or national concern to highlight at a special program during this national week of celebrating libraries.

His talk will be at the Downtown branch in the Multi-Purpose Room. Regretfully, I'll be missing it due to prior commitments.

[tags] Google, Wilkin, Books, OCR, Ann Arbor, AADL [/tags]

I've never attended a real open spaces event, so this is going to be a completely new experience for me (I've been saying that about a lot of things lately). I'm confident, however, that the format will be conducive to discussion and I'm expecting that we'll not only learn a lot but accomplish a fair bit as well. A number of great people are planning on attending--be sure to add your name if you plan to come. I believe that space constraints are capping registration at 40.

Some of my topics may overlap what Eli's already tossed out, but that's the idea--to find the overlapping topics and go from there.

I'd like to spend some time looking at Library 2.0. I'm wondering if the term itself is becoming a liability. My concern is that there is so much contention associated with it now that the great intellectual discussion it's intended to represent is being neglected.

I want to talk about ways that techies and non-techies can better communicate. I think there will be a good representation from both camps, if you will, that a candid dialogue could ensue.

I'd like to spend some time talking shop with other techies. Specifically, I'd like to get together with some other III users and see where that goes. I'd also like to cover/learn more about some of the stuff discussed at code4lib. XML/XSLT hacking would be fun too.

I want to talk about OPACs. Specifically, I want to talk about adding social software to the OPAC. I'd like to share some of the work I've been doing in this direction and invite others to do the same. I'd also like to gather ideas on getting buy-in for this sort of thing.

A general talk about trends in library websites. I'd really like to hear from the academic sector on this because the public library perspective can be very different and sometimes the academic library voices don't come through as well as I'd like. I'd like to find out where the similarities end and differences begin.

Because this is not an "in-industry" event, I'd like to talk with library users and get their opinions and perspective on many of the ideas behind Library 2.0.

I'd like to spend some time checking out all the neat gadgets people will be coming with--so bring your cool stuff!

I, too, would like to talk about IM virtual reference. AADL doesn't do it and I really think we should. I'd like to hear from others who may have successfully pulled it off. I'd like to talk about some other alternatives as well, such as web-based IRC clients/bots/etc.

I'd like to talk about networking in libraries. Perhaps cover different Wi-Fi solutions. I have some ideas on using bittorrent as a content delivery system for patrons I'd like to vet. General chat about networking/server infrastructure would be fun.

I think some discussion should go into whether an information clearinghouse should exist for L2 ideas and resources. The Library 2.0 wiki, perhaps? Maybe we can get a start on filling in some information.

I have a feeling I'll be updating this page post, but this is a start. I'll be cross posting this list to the possible session page on the Library 2.0 wiki. Be sure to add yours there as well. Even if you're not going, if you think of a topic, add it because there will be a number of bloggers present who will be interpreting events. Who knows, your topic might get discussed.