Samvera Tech 101
We are both Software Engineers at Notch8, but we are still relatively new to working with Samvera applications. For that reason, we thought our insights would be valuable in building a beginner-friendly presentation because the learning curve can feel a little steep and overwhelming at times. There are 5 categories we'll be discussing: Samvera, Data Stores, Background Jobs, User Interfaces and Samvera Applications.
If you'd prefer to watch the presentation, you can do so here. Otherwise, let's talk Samvera!
Samvera
When we hear the term “Samvera” it may refer to one of two things:- It is the name of the community that maintains a group of software
- It also often refers to the actual pieces of code and technologies that, when combined, make up the “Samvera stack"
At the foundation level, everything is built on top of Ruby on Rails, which is an open-source full stack web application framework built with the Ruby programming language.
But what's a framework??
A framework is a collection of code, tools & utilities that adhere to a specific structure when writing software. Frameworks can help to make code more organized and it allows developers to be more productive, instead of reinventing the wheel. We can think of a framework like a stencil. If we were given the task to cut out 1,000 pumpkin shaped holiday cards, we could manually draw and cut each and every one of them, or we could use a pumpkin-shaped stencil or tool to punch them out all at once, for example. From there we can color and customize them as we wish, but they are fundamentally the same since they were cut from the same stencil.
Likewise, frameworks like Ruby on Rails emphasize “convention over configuration”, which increases efficiency for developers and makes it easier to collaborate with others as well, because the foundation of the applications are the same. From there, it can be fully customized to meet specific needs.
Data Stores
A data store is a repository for continuously storing and managing collections of data. We can think of it as a physical library, but for digital objects in our context.
Solr is an indexed based data storage used to power the search functionality of our Samvera applications. It’s quick because it allows for indexing records by ID as well as by other metadata, like ‘author’, which means a record can be linked to multiple pointers and referenced in many different ways. It answers the question: What items have metadata that match? And returns the ids to look up the actual data in a database.
Although this analogy is not as flexible as Solr, a Library Index Card is similar to how Solr works. Index cards contain metadata of a book you are trying to find in a library, but it is not the book itself. It has a numerical index which points to the actual location of the book, which is where a data store, like Fedora, comes in.
We can think of Fedora as the bookshelves in a library. It is used as the persistence layer and is where the actual content and its associated metadata are stored. Thanks to integration with a gem called Valkyrie though, Fedora is no longer a hard requirement for the Samvera Stack.
Valkyrie is a data mapper that is used with data stores. It’s a gem for enabling multiple backends for the storage of files and metadata in Samvera. In other words, it’s how we manage and save objects to a database. You may hear of another gem called Active Fedora, but there’s currently ongoing efforts to replace it with Valkyrie because it offers much more flexibility. Active Fedora is tightly coupled with Postgres or MySQL database drivers. It can only talk to one data source. Valkyrie can talk to various versions of Fedora and other storage engines as well. So we will likely continue seeing a phase out of Active Fedora in favor of Valkyrie.
Background Jobs
Because some web requests take a long time to process, we use background jobs (an asynchronous process that runs outside of an application's request cycle).
In a real world Samvera-related example, if we're bulk importing and processing 500 records in an application, we're (probably) impatient! We don’t want to wait for the process to complete before we're able to do something else. So bulk importing and processing would be the perfect tasks to hand off to a background job. We can continue working while the records process outside of our view, versus waiting around for what would feel like an unresponsive web page to load.
Some popular choices for this are: Delayed::Job and Sidekiq. (Sidekiq requires Redis to store all of its jobs and operational data, as shown in the side diagram).
User Interfaces
How our users interact with our app.
Blacklight is a user interface option maintained by Project Blacklight. While it was adopted as one of the core components of the Samvera Tech Stack, it pre-dates Samvera and therefore is used by those in and out of the Samvera community. The Blacklight gem provides our users with the ability to search and or browse the metadata that we’ve indexed in Solr, right out of the box. We can use facets to browse items that are grouped according to predefined metadata keywords like format, language and author; as we see in the image above. Or we can do an open ended search that returns results based on whatever term(s) we've searched for.
Although the out of the box solution, once configured with our own metadata standards, is fully deployable... it is also highly customizable. One of the most obvious changes is adding our own repository style guidelines so that our app reflects our larger brand identity. In addition to styling however, we have the ability to use several plugins. Some of these plugins allow us to tailor our apps for the specific type of metadata we have, which we’ll discuss next. While some plugins allow us to do things like add a gallery view or slide show view.
In maintaining our library analogy, Blacklight is like the catalog cabinet that holds a library’s index cards.
The Spotlight plugin is the first of 3 plugins we’ll discuss. It’s an extension of the Blacklight gem so it too offers full metadata searching and faceted browsing plus display capabilities. The difference between the two, as the name suggests, is that the Spotlight plugin allows us to create attractive, feature-rich websites that spotlight a particular digital collection. Similar to how museums have lots of items, but showcase them in groups. As a librarian, curator, cultural institution, etc., we can have a large dataset, but present a curated exhibit.
One of the most notable things about the gem is that it’s self serviced. That means that we don’t need a developer to update our interface. Instead of writing code, site administrators would have access to a dashboard that allows us to update the collection, navigation headers, search result behaviors and more through the use of forms, select boxes and other easy to use methods.
GeoBlacklight was created with the goal of building a better way to discover, share, and access geospatial data. Which simply means data that has locational information tied to it like coordinates, an address, city, or ZIP code.
It is also based on Blacklight, but it extends the Blacklight functionality for geographical purposes, like adding map views of our search results.
ArcLight -
By now we’re catching on that each of these plugins are built on top of Blacklight. So just like Spotlight, and GeoBlacklight, ArcLight has its own niche. It’s specifically tailored to support the discovery and digital delivery of information in archives.
Each institution can have one or more repositories and each repository can have one or more collections. Like Indiana University’s Archive of African American Music and Culture. There are 3 collections in that repository, but this specific one is for Reclaiming the Right to Rock: Black Experiences in Rock Music Collection, 2008-2010.
Samvera Applications
Hyrax -
Hyrax is its own full stack application that allows us to better manage the content that you have in your digital repository, If you’re familiar at all with Sufia or CurationConcerns then a lot of how Hyrax operates will be familiar to you because Hyrax “descended”, if you will, from these two systems. It’s primed to be used with Fedora, Solr, Blacklight, a relational database and Rails of course. One benefit of starting your application with Hyrax instead of starting your application with rails is that Hyrax has a rich and growing set of features built in that are especially useful for repository owners.
For instance: Hyrax provides account administrators with a user friendly dashboard. That gives us the ability to create and edit user profiles, configure workflows, generate work types and work type images, upload multiple files and folders, set user level control over metadata and more.
Another advantage to using Hyrax is that it’s supported and maintained by the Samvera community. Therefore, as time goes on you receive the benefits of the upgrades, bug fixes and new features that come with Hyrax instead of having to maintain these things separately. However, it may not be the best solution for every project.
Hyku -
<img src="../images/blog/hyku-300x172.png"alt="Hyku">
Hyku is what’s known as a solution bundle. A repository app that’s been bundled in such a way to deliver functionality for a specific set of use cases. The use case for Hyku is multi tenancy. It’s built on top of Hyrax so it comes with all of the features of Hyrax, but being multi tenant means that there’s a single repository owner that can create multiple Hyrax instances for that repository. For example, the Pennsylvania Academic Library Consortium, Inc. and the Private Academic Library Network of Indiana came together to form Hyku Commons and share a single Hyku application. Here we see 4, but eventually dozens of libraries that are a part of the “super consortia” will have their own fully customizable tenant under Hyku Commons.
In addition to multi tenancy, we also get IIIF Image & Presentation API support, the Universal Viewer, and bulk import scripts. We also get greater customization options like adding fonts and custom CSS. While Hyrax alone is typically deployed locally and requires system administration and Rails development knowledge, Hyku is easier to deploy and maintain.
Hyku as a service is also something that vendors in the community provide. As a repository owner we wouldn’t need our own development team, but we would still get a lot of say in the development of the product. This may be especially useful for a library consortia. As of today there are two companies that provide Hyku as a service. Ubiquity Press provides
Ubiquity Repositories and Notch8 provides Hyku Up,Avalon -
Avalon is the second solution bundle provided by the Samvera community. Its use case is for managing and providing access to large collections of digital audio and video materials. While currently built on the Samvera core components, a future version will be built on Hyrax specifically, with this work resuming in early 2021. With version 7, released January 2020, came two new ways to explore and display collections, a new transcoding dashboard that provides administrators with a way to manage jobs directly within Avalon, a redesigned homepage, and easier configuration of featured collections that are displayed on that homepage.
In conclusion, we wanted to leave you with a visual image of the various technologies we just mentioned. We hope that you now have a better understanding of the pieces in the Samvera Tech Stack and how they all work together.
If you have any questions, feel feel to reach out to Notch8 at connect@notch8.com!