Central Michigan University Digital Collections convert from CONTENTdm to Veridian

Author: Kuan-Yu (Jeffrey) Ke, Project Leader, DL Consulting Ltd.
Date: 2017-07-12

Central Michigan University began their digital library project with OCLC’s CONTENTdm in 2008. After working with the system for several years, and after some consideration, CMU decided in 2010 to seek an alternative solution. They had four primary requirements of the change – a) a better user experience, b) more easily scalable to large newspaper collections, c) easier system maintenance, and d) lower cost. After two RFP rounds CMU decided to migrate to Veridian at the end of 2016. While changing to Veridian would not significantly lower costs, Veridian was seen as a good solution to achieve their other three key goals.

The Conversion Process

The migration began in early 2016 with the goal to consolidate several dozen small CONTENTdm collections into four Veridian instances: Digital Michigan Newspapers, CMU History, CMU Scholarly & Creative Works, and Clarke Digital Collections.

There were around 430,000 pages of documents stored across a number of separate CONTENTdm collections. For documents without significant metadata, such as newspapers, CMU exported the images from CONTENTdm and sent them to Veridian staff via FTP, along with spreadsheets containing title-level metadata. For documents with more extensive metadata, such as research theses and dissertations, Veridian engineers developed a process to harvest the images as well as their associated metadata directly from the CONTENTdm collections.

Build Collection

The exported documents were then further tidied up by CMU and Veridian engineers, into a more consistent and structured format. Finally, they were processed with Veridian’s MAPS (METS/ALTO Processing System) system, to generate consistent and standards compliant METS/ALTO objects for each document. The data conversion process was completed in batches, with the first batches appearing online approximately 3 months after the project began.

While the conversion of CONTENTdm data to METS/ALTO was in progress, Veridian engineers worked with CMU on the user interface and feature customization of the four Veridian software instances. The overall process (including data conversion) took approximately 10 months, and all four Veridian instances went live in early 2017.

According to the CMU staff involved in the project the conversion to Veridian was “surprisingly painless”, and things moved much faster than they had expected. Something they found very positive in working with Veridian was the level of empathy that the Veridian engineers showed towards the challenges they faced in making such a significant change. The Veridian engineers were able to cope with CMU’s tempo, and were extremely understanding throughout the entire conversion process.

The Improvements After Converting to Veridian

All four instances were launched in early 2017 and have received positive feedback from users and other Michigan libraries. It has been a great experience for both CMU and Veridian. The upgrade to Veridian has brought a number of improvements to the CMU collections, including:

Better User Experience

Veridian has improved the user experience in many ways. For instance, Veridian’s search result snippets (see image below) give users a small paragraph around the matched query to help them quickly determine the relevance of the matched records, making the content of the collections significantly more discoverable.

Michigan Search Page

Veridian also has many user-oriented features to make the collections more useful and interesting, and to encourage users to engage more with the collections, their content, and CMU. For example “My account” (see image below) allows researchers to create their own collection of records and share it with others. Veridian’s “User Text Correction” feature, as well as comments and tags, encourages users to contribute to improving the content of the collections, which has been shown to considerably increase the amount of time many users spend using the site.

MichiganUserPage

User’s personal space – private lists, recent search history (click to enlarge the image)
Michigan User Text Correction

User text correction (click to enlarge the image)

Easier to Maintain the System

The migration to Veridian has greatly reduced the complexity of the system by reducing the number of collections from several dozen to only four Veridian instances. And with the ability to set metadata at different “levels” in Veridian (i.e. publication-level, document-level and page-level), CMU can now manage the collection much more efficiently. For example, tasks which were previously difficult and prone to error (e.g. managing the metadata associated with a large collection of newspapers) are now very simple.

Veridian is also a fully-supported service, so CMU can rely on Veridian engineers to ingest and update batches. In addition, the Veridian instances are hosted by Veridian, so CMU has no need to purchase and maintain their own hosting platform. That all allows CMU library staff to spend only a small fraction of the time they did previously on the technical management and maintenance of their collections, leaving more time for more important tasks, like promoting their collections and adding new content.

More Scalable for Large Collections

The Digital Michigan Newspaper Collection currently has over 230,000 newspaper pages from 26 publications, and will continue to grow into the future. While in the previous system the newspapers had to be split up into multiple “collections” to prevent a loss of performance, in Veridian they are in a single instance. With Veridian as the base system the collection can easily grow to several million pages with no loss of performance.

Branded for CMU

The focus for CMU has always been on growing their digital content rather than building “the system” that serves the content. The overhead for employing engineers to utilize an open source system like Open ONI or Hydra is just too high, as is the cost of purchasing and maintaining their own hosting platform. Veridian provides the flexibility of customization that gives CMU the opportunity to have a system with CMU’s look-and-feel as well as the features that fit CMU users’ needs.

More Discoverable by the World

Veridian’s Search Engine Optimization (SEO) is making the content of these collections more discoverable by search engines, e.g. Google and Bing, etc… The number of visitors increased significantly shortly after the sites were launched and prestige sites, such as the Library of Michigan, that didn’t previously have a link to these collections, is now linking to them – http://www.michigan.gov/libraryofmichigan/0,2351,7-160-50206_18643-53657–,00.html

The upgrade to Veridian allows CMU to be more creative with their collections, as well as making them look more professional. Veridian has opened up more doors and more possibilities for these Michigan collections, and especially for the Digital Michigan Newspapers collection, for which it provides the capability to grow towards CMU’s ultimate goal of being the hub for historic digitized newspapers for the state of Michigan.

Michigan Digital Collections (Powered by Veridian)

Digital Michigan Newspaper Portal

Digital Michigan Newspapers
For nearly 50 years, the Clarke Historical Library has been actively preserving Michigan’s historic newspapers through microfilm. More than 239,000 pages of Michigan newspapers.
Central Michigan University Scholarly & Creative Works

Central Michigan University Scholarly & Creative Works
The faculty, staff, and students of Central Michigan University regularly create significant contributions to scholarship.
Central Michigan University History

Central Michigan University History
Documentation about the history of CMU is found here. More than 118,800 pages of Central Michigan University historical newspapers/documents.
Clarke Digital Collections

Clarke Digital Collections
The Clarke Historical Library holds a large body of material documenting the history of Michigan as well as a substantial collection of children’s books.