Networks Tackle Cybersecurity with MBLC State Grants

Identity theft, ransomware attacks, phishing and other types of cyber-risks are dangers that have become part of our daily existence, both as library workers and digital citizens.  In response, the websites we use now require multi-step logins, also known as multi-factor authentication or MFA.  Changes are even more striking in the workplace. Many of us log into staff applications via a VPN, or virtual private network, involving multiple steps and a dedicated phone. While our systems providers try to streamline our workflows, our computers and work phones are locked down, requiring more work simply to begin work.  Simple, shared passwords are a thing of the past. Data backup and recovery strategies are important for anyone using the internet, even casual home users.

Two years ago, news of large-scale cyberattacks exploded in the national media. The Colonial Pipeline attack in May of 2021 stood out in particular.  In July of 2021, I was made aware of some new guidance generated by New York State on ransomware attack prevention and response.  I myself had just become a victim of a ransomware attack at home, through a security hole in my backup software; the irony did not escape me. My music files were locked up and held for ransom.  At that time, I asked the nine automated resource sharing networks whether they were prepared. Were they confident with their cybersecurity posture? Were they on top of protecting core library services and patron data? Did they have the ability to quickly recover should they experience an attack?  Should all the networks, possibly with help from the MBLC, work individually or together to improve network resilience in the face of seemingly inevitable cyberattacks?

Three weeks later, on August 25th, 2021, the Boston Public Library (BPL) was hit by a ransomware attack which brought the BPL and Metro Boston Library Network systems down for a full week.  David Leonard, the President of the BPL was kind enough to meet with network administrators a few weeks later to share lessons learned — to describe what had happened, how it might have happened, how the BPL had recovered, and what step the library was taking to protect itself in future.

MBLC Awards State Cybersecurity Grants

The BPL attack showed how broadly disruptive a cyberattack can be on library services.  Networks provide the mission critical, core business functions on which every library operates.  When an attack occurs, patron records, the catalog and circulation system all become unavailable.  Ancillary systems, email, websites, access to electronic resources may all be affected.

The MBLC decided to offer a cybersecurity grant opportunity of up to $25,000 per network using state funds from account 7000-9506, Library Technology and Resource Sharing.  In total, we awarded $181,093 to eight networks.   The program ran from May 2022 through June 2023.

Each network used grant funds to address its own priorities as each was in a different place in its thinking, planning and overall preparedness.  To provide an overall framework, MBLC asked networks to categorize their activities according to the four goals laid out in the Minimum Baseline of Cybersecurity for Municipalities from MassCyberCenter. Though designed for cities and towns, the framework proved equally well suited for a common perspective on network grant activities.

The four goals are:

Not surprisingly, all eight participating addressed Goal 4. Providing technology is a network’s bread and butter. Four networks also identified staff training, and one network focused on response planning.

Minimum Baseline GoalsNetwork
Trained and Cyber-Secure EmployeesCW MARS, FLO, MVLC, SAILS
Improved Threat Sharing 
Cyber Incident Response PlanningSAILS
Secure Technology Environment and Best PracticesCLAMS, CW MARS, FLO, MBLN, MVLC, NOBLE, OCLN, SAILS

Staff Training

It’s almost a truism that human beings are the weakest link in the cybersecurity chain. Therefore, thorough training is essential. Besides a series of instructional sessions or webinars, training often includes a series of phishing tests. A security vendor will send out phishing emails or smishing texts (phishing via SMS) to see whether staff recognize the malicious messages or instead, open the message or message attachment, actions that might in the real world have led to a damaging security breach.  FLO reports that their “phish-prone percentage” came down to 7.8% from a 50% mark (half of FLO staff) at the beginning of the program, and that since January 2023 no FLO staff member has clicked on a phishing email at all, easily surpassing FLO’s objective of 5% originally set out in their grant application.

MVLC experimented with a suite of free security training tools to gauge their effectiveness. Having obtained encouraging results in participation, they will consider making this part of their annual training regime in future.

Cyber Incident Response Planning

SAILS undertook formal planning as part of the grant.  SAILS’ incident response plan, when complete, will cover the steps to be taken should there be a security breach. It will include who will be notified: the network attorney, the system vendor(s), the cybersecurity insurance provider, telecommunications support provider, the network internet service provider, and, of course, member libraries.

The plan will address the following six phases:   preparation, identification, containment, eradication, recovery, and lessons learned.

SAILS recognizes the importance of sharing the plan with member libraries. An incident can start at the library.

The Boston Public Library /MBLN network, which had suffered that significant cybersecurity attack in 2021, hired a consultant to develop a security roadmap to improve its overall security posture. Preliminary direction will have been guided by vulnerability scan and penetration testing. BPL also intends to hire a full-time Cybersecurity Analyst.

Improved Threat Sharing

No network explicitly identified threat sharing as a grant goal.  However, as part of incident response planning, networks recognize that registering with regional and national threat resource centers, such as MS-ISAC, the Multi-State Information Sharing and Analysis Center, and the New England regional office of CISA, the Cybersecurity & Infrastructure Security Agency, is critical.  Networks will proactively hear about threats that might affect them and will know whom to inform should an attack happen to them.  Networks will be better prepared to share threat information with each other in a timely fashion.

Secure Technology Environment and Best Practices

The majority of grant-related work focused on ensuring that networks’ core systems, backups, were secure, and that shared work environments being accessed by both central site staff and library staff were controlled by technologies, policies and procedures to minimize risk.

The Library System Hosting Environment

Two networks, CW MARS and NOBLE, had locally hosted library system servers. Recently, either as part of this grant, or slightly before, both networks had moved their servers into a Google Cloud environment under the management of Mobius Open-Source Solutions (MOSS).  Large-scale cloud hosts such as provided by Google and Mobius, bring assurances of a much more secure environment than any local installation could manage.  This includes physical security, system and software patching, vulnerability testing, standards, access controls, authentication, and backup and restore options.

Through a consultant, NOBLE audited the security of their servers’ new home, and especially the cloud-hosted data backups.  NOBLE’s consultant provided a series of recommendations back to Mobius that should benefit not only NOBLE and CW MARS, but other similarly situated library systems as well. NOBLE also now takes more frequent system backups, housing them in a separate location, a more secure approach.

CLAMS took a hard look at the hosting environment for their new Koha/Aspen Discovery library system from Bywater Systems. Bywater has tested incident response and business continuity plans.  Bywater had several recommendations for CLAMS, including the use of a reverse proxy server, regular vulnerability scans, an intrusion detection and prevention system, and IP access control for all Koha admin interfaces.

Equipment Replacement

OCLN and NOBLE replaced older routers in members libraries with state-of-the-art advanced firewalls that included intrusion prevention features. Intrusion prevention systems proactively check for real-time threats or attacks and take action to stop the activity.  The new routers will better protect not only the network, but also local library LANs, attached equipment and data.

The change to remote or hybrid work environments that we’ve seen over the last three years means that staff are no longer necessarily accessing the library system through library-owned computers on library-managed LANs.  As part of the grant, networks focused on ensuring that secure VPN connections are always used by both central site and library staff.  The newly purchased firewalls have made possible simplified VPN sessions for staff working remotely, and a much more manageable overall VPN environment for central site. As an example, OCLN reports that it now has single-sign-on capabilities through Google, so that staff can sign onto the VPN via a regular browser and using the same credentials that they use for Google Workspace.  And no more shared passwords among library staff!

Staff Applications: Google Workspace and Microsoft 365

Central site and numerous library staff use shared applications.  Several audits found that access to Google Workspace and Microsoft 365 needed better access controls. A clear, and near-term goal is to enforce multi-factor authentication for administrative users, and if possible, extend the requirement to all library staff.  CLAMS purchased MFA “security keys”, a small USB device for all staff to use when working remotely.  Security keys obviate the need for passwords and thereby avoid the danger of phishing attacks designed to capture passwords.

Password Strengthening and Management

CW MARS obtained a business class password management platform which enabled password strength to be audited.  By the end of the grant period, they reported that for central site staff, “Our average password strength was 94%. 0% of staff had a weak master password. 0% of staff had a reused password.”  Based on its security audit, MVLC intends to pursue a similar solution.

Email

Tighter network email attachment policies, better email verification via mailing system standards —DKIM and DMARC in particular– have been identified as ways to improve trust in email messages both coming into libraries and going out.

Penetration Testing and review by 3rd party

FLO was one of several networks that did vulnerability testing. FLO really dug into this issue, using Open Source Intelligence (OSINT) techniques to see whether there was information on potentially harmful attack vectors “out there” on the internet that might impact FLOs systems. As a result, they decommissioned an outdated server using an old operating system along, among other actions.

Not Just the Central Site – Including Member Libraries

Though some projects focused exclusively on central site systems and staff, others had broader reach. For example, MVLC’s security audit included 28 of its member libraries.

Next Steps

Every year, the MBLC provides network infrastructure grants from account 9506.  For FY24, the total grant round was increased by 33% to $400,000.  Cybersecurity investments are now allowable expenditures under this grant.   The initial MBLC cybersecurity grant round kicked off an ongoing process. Networks will take what they learned, and at the very least, invest in training, planning, plugging holes, updating policies, communicating cybersecurity roles and responsibilities to member libraries, and working together with their peers across the state to make Massachusetts libraries, resources, and library patron information safer and more secure.


Main Image: Lightning striking a rural building during a storm: onlookers react in terror. Engraving, 16 –. Weather. Lightning. Work ID: hfz9n5qe : under CC BY 4.0.

Shop ‘Til You Drop

Shopping for Statewide Databases : Chasing the Best Value for the Commonwealth

Tragedy and Comedy masks : From the Baths of Decius on the Aventine Hill, Rome
I’ve got some good news and some bad news…

By Paul Kissman, MBLC Library Information Systems Specialist

We’ve just completed a procurement process for the next set of statewide databases, a fifteen month long odyssey. There were moments that put me in mind of those old shopping-themed TV game shows. Some days we were contestants on Supermarket Sweep, as we frenetically raced the clock to put as much quality content in the cart as possible before the bell rang.  At other times, we were competing on The Price is Right,  guessing at that ineffable figure, the actual dollar value of a database.

But it was no game, and there were no big prize winners at the end.  With a 30% reduction in funding we knew right from the start that the results of our efforts were going to be bittersweet.  We are proud of what we accomplished, and Massachusetts libraries will continue to have a strong core set of databases.  But we also know that we have lost access to some very important products; our shared resources are that much smaller.

Where Do We Stand and How Did We Get Here?

Beginning July 1, Massachusetts libraries will have the same three vendors and a set of database products that looks an awful lot like what we have today — just diminished.

Some of you may wonder, “Why all the sound and fury then? Why the big process?”  Are we complacent, taking the path of least resistance?  Maybe we lack the courage to try something new or maybe we have a hidden bias in favor of the incumbent vendors and familiar products.

Though we heard from many libraries and invited input along the way, including a month-long open trial and vendor demonstrations to representative stakeholders, our decision-making process may look like a black box to many of you.  Without going into the gory details, here is what the procurement looked like from the inside.

Peeking Under the Hood

Who exactly sets the stage and makes the final procurement decisions? MBLC and MLS, with a sprinkling of Library for the Commonwealth.  These three organizations have worked hard to complement each other’s offerings.  With shrinking budgets and other critical priorities we can’t afford not to.  Though I’d like to think we would anyway.

MBLC appointed an advisory committee of ten very smart and knowledgeable librarians from academic, school and public libraries to help guide us through this process.  They were content specialists — the ones doing bibliographic instruction, working with teachers, students and the general public every day. Their contributions were incredibly valuable.  They each represented their own library types’ interests but showed great sensitivity to how different products would be valued by users of all types of libraries.  Not an easy thing to do.  They analyzed product titles to gauge full-text content, overlap, uniqueness and value. I came out of the process with tremendous respect for their skills and experience, and I am grateful that they were there every step of the way.  Thanks guys!

We first began to experiment with databases for all regional members twenty years ago.  Gale/Cengage, then Information Access Company, was our first provider with some general periodical content.  Since that time we have run five procurements and have contemplated many approaches.  We’ve considered targeted solutions for different library types:  school-centric products for schools, more specialized databases for our academics and special libraries, local newspaper products only available to parts of the state.  We’ve tried creating a market basket, where preferential pricing was offered for libraries or groups wishing to supplement what the state could offer.  Five years ago, we managed to expand the subject areas and types of resources, asking for genealogy and language learning products, both general and specialized encyclopedias.  Though the genealogy and language products didn’t pan out, we were able to add a general encyclopedia for the first time.

We have to find products that appeal to all types of libraries.  The scope of the our procurement is determined by usage statistics and surveys.  Usage statistics are necessarily limited to current product offerings. However, when establishing the procurement scope, we only use these statistics to draw inferences about subject coverage, not about particular titles from particular vendors.  The only exception to this rule is The Boston Globe, a specific title.  A large library survey in the spring of 2016 gave us broader insight into library preferences.

Why Do We Always Seem to End Up with the Same Vendors?

The answer is fairly straightforward.  They have consistently provided the best value for the Commonwealth.  It doesn’t mean that this will always be the case.

In the past we’ve disqualified vendors because they could not demonstrate the capability to roll out services statewide, work with our statewide login process (geolocation for users in Massachusetts) or set up 1,600+ library accounts. They couldn’t provide interoperability with library discovery systems and knowledgebases, provide granular usage statistics and related management tools. Not this time.  All six vendors were sufficiently qualified.

We try really hard to be objective and open to new solutions.  I know that I get enthused about new products, new platforms, new vendors. I also like to see the progress that familiar companies have made with their user interfaces.  From one procurement cycle to the next, the three big periodical vendors, EBSCO, ProQuest and Gale seem to leapfrog past each other in user interface design and usability .  This time around all three main platforms were really solid, with contemporary interfaces providing excellent user experience.  That hasn’t always been the case.

From the library community we hear competing interests.  Some academics have urged us to license EBSCO so they can repurpose their limited budgets. Public libraries and schools may not want us to change vendors because then they would have to extensively retool and retrain patrons.

There is no alchemical mixture of intangibles at work here. As with any rigorous procurement, we use weighted score sheets to evaluate the various components of each proposal.  Content (which is weighted most heavily), organizational qualifications, technical qualifications, ability to license to all our users, all are evaluated and quantified.  Cost is the last factor we look at.

What’s on Offer

It is important to remember, we can only evaluate what the vendors propose. Sometimes librarians will ask why we didn’t license a particular product.  Often the answer is, “It wasn’t proposed”.  Sometimes products are simply out of scope. Sometimes proposed packages don’t provide enough valuable content to schools, or academics, or even to public libraries.

Sometimes there is not a good business case from the vendor perspective.  We can’t afford to replace the large base of existing academic contracts for products like Academic Search Premier from EBSCO.  EBSCO has indicated that a statewide offering this comprehensive would be way beyond our means and so they don’t propose it.  Thus, academic libraries see they will need to keep their EBSCO contracts, but they also find tremendous value in Gale Academic OneFile as a complement to their own locally-licensed content.

The Globe is the Globe.  We reached out directly to both the Boston Globe and New York Times, but they declined to bid.  For the Globe, ProQuest was the only game in town.

Encyclopedias – World Book and Britannica were both highly esteemed products. Britannica appealed more to public and academic libraries, as World Book seemed more targeted to K-9. At the end of the day, Britannica had the broadest appeal, and was the product that we could afford.

So here we are, entering a new fiscal year with old friends.  We ended up here for good reasons. Maybe next time around things will turn out differently.