blyberg.net » SOPAC 2.0: What to Expect

SOPAC 2.0: What to Expect

On September 1st, Darien Library will go live with a new web site. We will also be going live with SOPAC version 2.0. In case you’ve been wondering why I don’t blog anymore, it’s because of 2.0. It’s been a massive project.

As you probably know, SOPAC 1.0 was written for the Ann Arbor District Library catalog while I was working there. It was, however, a collection of ‘hacks’ that I made in order to get that functionality working within the Drupal framework of aadl.org. I did release the 1.0 code but as I said at the time, it would “definitely not work out-of-the-box”. It was a highly-customized module made available so that other developers could play with it.

When I began work on SOPAC 2.0, I kept two prime directives in mind. First, it had to have the potential to work with any ILS. Second, it had to be a highly-configurable, distributable package. In other words, it has been completely rewritten from the ground-up to be a real software “product”. This will be a project that is managed and maintained by Darien Library. We will make the beta available under GPLv3 once we get our legs back under us after Sept 1st.

How it Works

SOPAC 2.0 itself is a non-trivial project. It is an expansive Drupal module that completely integrates all online catalog and patron activity. It is written for Drupal 6 and takes exclusive advantage of Drupal 6’s hooks and APIs. The result is that it is 100% theme-able and template-able. You will be able to make your catalog look exactly the way you want it to look without exception. Virtually every element of the patron account integration is also configurable.

In order to make SOPAC 2.0 possible, however, I’ve decided to create two software libraries that operate independently of each other. The first, Locum, provides a catalog discovery layer as well as ILS-independent abstraction for all catalog-related activity. It can be used by any PHP application, not just SOPAC. So, for example, you could ostensibly port SOPAC to Joomla using Locum as the sublayer. Locum allows developers to “drop in” connector pieces for their own ILS. Locum (and anything built on top of it) doesn’t care if you are a III library or a SirsiDynix library or a Koha library.

The second software library, Insurge (Independent Social Repository), is in response to my feelings about the inadequacy of local, community-driven social data. On one hand, I recognize the value and importance of highlighting local social data, but for the purposes of search and content discovery, it’s just not a large enough data-set.

My original intent was to store all the social data within the Drupal database (as it is in SOPAC 1), but I was not terribly thrilled with that prospect, so I began to envision a second abstraction that was dedicated just to bibliographic social data and which could be shared, freely, between different institutions. In addition to our own tags and reviews, we can choose which other libraries we want to pull social data from. We would, in turn, sync up our new data so that other libraries could download and use it. In this way,we have the opportunity to “highlight” our own local reviews and tags while taking advantage of a larger pool of data. The best of both worlds.

Naturally, there are many nuances and complexities involved in all this and we will be launching a dedicated site for the product suite that will allow users and developers to start building a support base.

I’m sure there will be a myriad of questions and I will answer them as I can but if I don’t reply to your email or comment right away it’s because September 1st is closing in fast and it’s two weeks and two days to bingo time.

These icons link to social bookmarking sites where readers can share and discover new web pages.

34 Comments so far
Leave a comment

No questions, really, just a comment - “dude, awesome” is about the most I can offer because I cannot really get much more out at this point because I think this is so huge b/c of Insurge and that it will be GPL’ed. I think this could really be the piece that pushes so many people over the OSS edge to actually contribute and work together and see the value of devoting resources to development rather than (and/or in addition to) support.

Can I also be the first to request that you speak at a panel we are holding at ALA Annual in Chicago next year about this building community through use of technology? I can send more details if you are at all interested.

I’ll second the “awesome” sentiment. Projects like this are important because they’re helping libraries open up their “silos” and giving us what a lot of other internet communities already take for granted.

I’m particularly excited about how the SOPAC and things like it might interact with other open source projects such as Evergreen and Koha. With open data, open standards, open API’s, and open source, “disintegrated” library automation is becoming more and more a reality.

John, you are made of awesome.

W00t! I assume you have created an ILS connector for III. Does it depend on the discontinued XML Server product or… ???

Thanks for the support, we are very excited!

Mike, yes the connector piece for III is completely written and because Locum is its own discovery layer, it does not require the III XML server. The III connector piece uses XRecord and the patron API.

John, This looks great. Is there any chance of seeing a beta version of the site before Sept? I’m also curious about how you are thinking of handling the moderation of user generated data. Will there be any distinction between content that a site would use for it’s own pages and content that it would make available for others to use? I could see it being useful to allow an anything goes approach to your own site with data that your own users generate, but vetting the content that is made available to other sites.

Josh, I’m afraid that you’ll have to wait until Sept 1st to see SOPAC2 in action. I’d love to give everyone a sneak peek, but I just don’t have the time to accommodate those requests right now.

As for participation in the Insurge repository, your organization can either choose to or not. You can even start your own repository if you wish because that software, too, will be made available. The system I envision is based on a mutual trust between organizations that they not allow bad data be introduced into the repository. However the fact that you can select which other libraries you want to pull digests from will give you some control over the data you’re using in your own environment. In other words, if you notice that Library X’s data looks to be of questionable quality, you will be able to simply unsubscribe from their data-set. Bear in mind that we’re not talking about “pages” here, but contributed metadata.

I do plan on building in administrative capability to moderate your own content, but that will not be included in the initial beta. It’s a feature that is, quite frankly, unnecessary 99.9% of the time.

No need to repeat all the ‘awesome’ comments but this is exciting, and yes, I have a couple of questions:

You’ve commented before that social tools in your OPAC didn’t live up to your expectations, e.g.M., not enough people tagging. I can see you are trying to address that with the Insurge idea, but how are you defining success this time? How will you measure this service?

Second, you’re obviously putting a lot of your own personal effort and vision into SOPAC. Is it all your work/vision, or were others involved in the describing / envisioning / spec’ing the functionality for SPOAC? Patrons, library staff, the board of trustees?

John, we’re of the mind that the success of this project will be reflected in the improved quality of online customer service we provide. When approaching this project, we did not consider it in terms of how we would measure its success, we based our decision to proceed on whether there was anything else available that we could use that measured up to our standards. But that said, there will be metrics to measure the success of this project (number of records, open-source community participation, etc).

A project of this size cannot exist as an island. This project reflects the sensibilities and philosophy of Darien Library. Long before I even became a Darien Library employee, we had committed to redefining what it means to be a library in the 21st Century. Many talented and smart people have put a lot of their own personal effort and vision into that redefinition, SOPAC is simply a piece of that larger effort.

Congratulations, John-and-team. I can’t wait to see it in action and to look at the code. The Insight repository sounds like a real step forward in aggregating local activity to reach a network effect.

This is great news! Our library would definitely be interested in testing and/or building additional functionality (modules?)… we are already using a Drupal-based “discovery layer” (http://biblioteca.mty.itesm.mx/pasteur/) however building on top of a mature base seems more logical.

We also have a “sort-of” patron API-less Drupal Patron API for III; I guess we could help out libraries that do not have the Patron API product.

Alejandro’s Millennium module is pretty cool and I’m looking forward to SOPAC 2. Would be great if libraries could choose Patron API or another type of authentication for those without Patron API!

Well, you could easily take the packaged III connector that I’ve already written and adapt it to use Alejandro’s patron interface as long as it adheres to all the up-stream expectations.

Singing the praises of the skillz of Mr. Blyberg. Way to go John.

A day-of-thinking-about-it thought, John. In your post, you say “My original intent was to store all the social data within the Drupal database (as it is in SOPAC 1), but I was not terribly thrilled with that prospect…” Storing the user data in Drupal is arguably one of the strengths of using Drupal as a framework. If y’all decided to create a non-Drupal data infrastructure for user data (which it would seem would only help those that decide to join an Insurge collaboration without using Drupal), does SOPAC itself use much of the Drupal infrastructure anymore?

Peter, there is a distinction to be made between user data and social data, and yes, there is some user-specific data that Drupal stores as a means to facilitate SOPAC functionality. But as you point out, the purpose of Insurge is to provide a social bibliographic application framework and repository that is independent of any given application. Insurge is a software library. SOPAC (via Drupal) is an application that takes advantage of it.

Also, Drupal’s use of MySQL is actually a tertiary consideration in my estimation of its potential as a CMS and as a platform on which to build SOPAC. Its real strength is its beautifully conceived API which SOPAC2 takes exclusive advantage of. So yes, SOPAC is a full-blooded Drupal module, written such that you’ll never have to ‘hack’ it in order to customize it. It actually uses the Drupal framework far more extensively than SOPAC 1.

Dizziness! I’m simply dizzy with excitement and awe…and just when I had resigned myself to needing to join something like WorldCat Local to reap the benefits of large, meaningful data sets of social information…I have marked Sept. 1 on my calendar. Can’t wait to see it all in action. Congrats!

[...] Blyberg of Darien Library says that SOPAC 2.0 will be launching September 1. I’ve been watching this “discovery layer” project for some time, in part due to [...]

[...] so that others can take advantage of it. The components, as he describes in his blog ‘SOPAC 2.0: What to Expect’ post on the subject, includes a separate social repository layer (Insurge) which not only could [...]

[...] SOPAC 2.0: What to Expect. OPAC’en flytter sig videre ind i fremtiden. Det bliver muligt i SOPAC 2.0 at dele social viden. [...]

I came here via Panlibus, and just wanted to add another voice to the enthusiastic chorus (”awesomeness;-)”) There are a number of UK library 2.0 initiatives grappling with some of these issues — galvanizing critical mass of usage data content to support effective personalisation/tagging, etc.
Our service is among those beginning to explore this terrain, and so we’ll definitely be looking at how all this develops with real interest.

[...] and functionality of this new discovery layer over the top of an Innovative Interfaces catalogue: SOPAC 2.0 What to expect [...]

Congrats on the new website launch, now get some sleep!

How soon till we get to see some code? =)

[...] looks very good… excellent, in fact. I am already looking forward to playing with this version of the software. What I really like at first [...]

[...] Blyberg 稱它為 SOPAC (Social OPAC) 2.0 版的網站。John Blyberg 在部落格(blog) 裡提到 SOPAC 1.0 與 2.0 的差異，底下是 Keven 的翻譯/介紹： [...]

[...] http://www.blyberg.net/2008/08/16/sopac-20-what-to-expect/ [...]

John, extremely exciting architecture-well done. As I begin to wrap my mind around the concept the potential is huge. Thanks for the herculean effort.

[...] http://www.blyberg.net/2008/08/16/sopac-20-what-to-expect/ [...]

Sounds like you need to go to this panel:
Open source software enjoys the rapt attention of libraries. In practice, however, most institutions are better at consuming than producing open source code. Issues that repeatedly arise include:

redundancy (“we can build it better”)
shyness (“my code isn’t perfect”)
competitiveness (“we have an edge we don’t want to lose”)
quirkiness (“our needs are so unique that we’ll ignore any obvious standards”)

WHERE”S THE CODE?

Anon,
We indeed do love Open Source and we will release the code very soon now. My plan was to release it shortly after September 1st, but I’ve been taking the time to fix some of the more critical bugs that have come up, post-launch. Watch this blog for an announcement this week.

[...] I love the disorganized, sloppy, but wonderful findability-ness of them. I would love to have a SOPAC or LibraryThing in my library catalog, so I could tag away in there, too. So when I read [...]

[...] more detail (and more accuracy than my notes!) available at http://www.blyberg.net/2008/08/16/sopac-20-what-to-expect/ and [...]

[...] a detailed explanation of SOPAC, see http://www.blyberg.net/2008/08/16/sopac-20-what-to-expect/ and [...]

RSS feed for comments on this post. TrackBack URI

Leave a comment
Line and paragraph breaks automatic, e-mail address never displayed, HTML allowed:

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>