Inside the NLA's make-or-break digital transformation

Archiving printed materials is one thing, but keeping up with a deluge of digital materials was proving to be a difficult challenge for the National Library of Australia (NLA).

The agency was facing an inability to fulfil its basic mandates until it commenced a major overhaul that is exceeding financial and process expectations.

The digital library infrastructure replacement (DLIR) project emerged after years of effort to modernise the NLA’s traditional archival capabilities for the digital era.

Commercial off-the-shelf (COTS) systems had provided a stopgap measure but they were, according to NLA development branch director Kavinga Hubert, “rudimentary in nature, had very limited capacity and functionality, and were no longer able to support Library’s future requirements".

A working group convened and recognised the business risks posed by the systems’ inability to scale, process inefficiency, and “unsustainable” maintenance requirements. The six-year DLIR emerged as a broad initiative to modernise the library’s systems and facilitate new collaborative approaches.

Users were well aware of the system’s limitations, and so were engaged from the start of the project - joining business representatives, business analysts, developers, testers, and stakeholders in a development cycle that elicited and addressed real-world user stories.

Staff from business areas were embedded within the ICT team, ensuring that delivery was addressing their requirements, and that users took ownership of the workflows as DLIR evolved. Regular releases of new functionality, end-of-sprint demonstrations, and user acceptance testing ensured the project stayed on track.

Collaboration amongst the 30-strong team proved critical for a bespoke development process that combined collection management systems, collection and preservation workflows, digital delivery systems such as Trove, web archiving tools, a digital deposit service (DDS) enabling outside contributions, and integration with internal library systems.

The DLIR platform was based on several commercial tools including Preservica (digital preservation and management), Compact Computer System GmbH DocWorks (optical character recognition), Kakadu Software (JPEG2000 image conversion), and ArchivesSpace (collection management capabilities).

These were integrated with back-end NLA systems and user-facing interfaces using approximately 30 open-source software tools and libraries including the Spring digital library framework, Apache utility libraries including the Solr enterprise-search platform, Oracle MySQL database, and others.

Since it went live in June, the project has exceeded financial and performance expectations, driving a dramatically faster workflow that means a 200-page book can be digitised 80 percent faster than in the past. By November, the library had digitised 1.53 million book and journal pages – up from just 15,000 three years ago.

The digital deposit service (DDS) particularly has proven to be an invaluable efficiency tool, allowing donors to upload digital objects for analysis, indexing, archiving, and delivery to the public. This workflow used to take weeks, but within the new framework a new object can be processed within a single day.

The NLA’s original business plan for DLIR projected a 20-year return of $14.30 per dollar invested, but after six months it says the actual benefits being realised are far greater than initially hoped.

Strategic, operational, managerial, organisational and overall IT infrastructure improvements have allowed NLA to boost externally-funded digitisation revenue by 55 percent.

By engaging different stakeholders and facilitating new forms of project delivery, the DLIR initiative has fundamentally transformed the NLA and positioned it to be able to meet its archiving mandate far into the future.

The Library describes the "major program" as fundamental to its strategic direction. It has been the IT team's highest priority since the initiative began in 2011.

“This was the largest co-ordinated software development the NLA has attempted to date and we learned a lot along the way,” Hubert says.

“The workforce is much more engaged with improving processes as a result, and there is a great deal of pride in managing constructive change.”

This project has been named a finalist in the federal government category of the iTnews Benchmark Awards 2017/18. The full list of finalists can be found here.