Library Network Security

The Library Network (TLN) runs a service called MiLE which handles all of our inter-library loans and does the same for a good part of southeastern Michigan. Recently, they were targeted by a hacker who brought their system down. They were unable to recover from the disaster and restore service. Ed Vielmetti has posted about this with some links to more information.

I want to preface any comments I make by saying that there is no excuse whatsoever for what the hacker did to MiLE. It’s a case of malicious vandalism.

However, that doesn’t change the fact that there are malicious vandals. It also doesn’t change the fact that library systems contain sensitive patron information. TLN is responsible for two major failures.

First, TLN should have made sure that their systems were not vulnerable to an attack of this magnitude. I’m not familiar with their infrastructure, so I can’t comment on this without speculation. There are a good number of simple practical and procedural steps you can take to harden your network and it’s servers. These include using encrypted transmission protocols, segmenting your network behind a firewall, isolating critical servers, intrusion detection systems (an attack like this doesn’t happen without a fair amount of probing beforehand), keeping up-to-date on security patches, doing your own security audits, scans, attacks, etc. I’m fairly certain that there must have been a failure to do much of this on a scheduled or regular basis.

Second, and this is the real shame, is that TLN was unable to recover from this incident because their backup and recovery system either wasn’t in place or failed completely. If you’re operating without a viable disaster recovery plan, you’re essentially operating without a net. Anyone who is responsible for backups knows that part of performing this duty is to change tapes on a regular basis, perform check-sums on the tape data to make sure it’s good, and to backup often enough that your recovery window isn’t so large that it’s useless. At the very least, data that changed every day should have a full backup every week, with a nightly differential backup.

TLN is fully responsible for this failure, but at least they’re picking up and moving on in the best way possible. I give them a lot of credit for being totally transparent about this disaster and making the absolute best from a crumby situation. They will be upgrading to URSA 4.0 and will open with an expanded feature-set. I just hope that they’ve tackled their back-end issues.

My fear is that the problems that plague TLN are systemic in our industry, and I think they probably are. For instance, if you use telnet to access your ILS, your institution is at just as much risk as TLN. There are steps your network administrators must take to ensure the safety of your patrons’ information. When adult patron’s come in to get a new card, most libraries ask for their social security number, and they give it to us, freely and without hesitation. I would certainly think twice given what I know. I’d probably insist that it not be kept on their online system.

So, what are the steps you should take? At the bare minimum, all libraries should do the following:

1) Install a firewall and move every single server, computer, printer, fax machine in the entire organization behind it. If your machines are exposed directly to the internet, they are infected already. Every single windows machine on your network is a “bot”, or soon will be. When you move your machines behind the firewall, they will need to be wiped clean and a fresh operating system installed on them (unless they are macs or Linux boxes).

2) Install virus scanning software on your mail server/gateway. Ideally, you’ll use a mail gateway to vet all incoming mail. Prohibit all attachments with the following extensions: exe com bat vbs mp3 mpg avi pif bas and probably zip and rar.

3) Upgrade all your server software to the latest version. If you can’t do that, at least patch them with the latest patches. Set aside one day a week to check the patch-levels and install new patches where necessary.

4) If you have wireless or patron Ethernet access, separate it from the rest of the network. In addition, create a separate sub-net for all machines the public uses, ie OPAC, internet stations, print stations, etc.

5) Install intrusion detection systems on your firewall. If you have a Cisco router/PIX firewall, forward netflow traffic to Ntop and analyse it daily.

6) Insist on encrypted transmission protocols. Instead of telnet, use SSH. Use SSL on sensitive web pages. I had to insist that our vendors used SSH. This included our ILS vendor who wanted telnet access! Say, “No!” and leave no room for discussion. They can use SSH or nothing. Telnet is the protocol of death and the sooner you acknowledge it, the better.

7) Change your default passwords! You know, after Innovative finished their “implementation”, they were going to walk away and let us continue using the default passwords. Unbelievable. Unbelievable! So as a curiosity, I tried the default passwords on some other III libraries around the country. I tried 5 systems and got into 3. Holy crap. I could have perused patron information to my hearts content. Instead I logged off and emailed those sysadmins immediately.

I’ve heard the remark, “Well, we’re a small library, we only have one IT guy, we don’t have time to do all this”. My response to that is, “Then delete all your patron information now, because you’re not responsible enough to have it in your possession”. Libraries are among the most insecure systems in the country, and that’s shameful.

In my presentation at IL2005, I tried to stress the importance of infrastructure redesign. This is why, right here. All the fun, new things we want to do go right out the window the second a patron’s identity is stolen because we let someone walk right in and take it away.

About this entry