Adding blacklight_advanced_search to Hyku

Adding blacklight_advanced_search to Hyku

I was recently asked to add Blacklight Advanced Search to a Hyku app for the US Department of Transportation. It was a little tricky, so I’m documenting the process in the hopes of making life easier for the next person who has to do this.

Many thanks to Dean Farrell at UNC Libraries for pointing me at UNC’s hy-c implementation, where blacklight advanced search is installed into Hyrax: https://github.com/UNC-Libraries/hy-c/pull/421/files

David Kinzer’s Blacklight Search Notes are also excellent background reading: https://gist.github.com/dkinzer/4f6dbb4634dbbdc99255dbea6305ccae

Write a feature spec first

As always, start with a test. I like to write a high-level feature test as a way of keeping myself focused on what I’m trying to deliver, and then I fill in unit tests for specific methods as needed. Here is my nearly-empty high level feature test. Note that I don’t really know what advanced search is going to look like yet, so I’m not spending lots of time on detail yet. However, it’s still helpful to have this test first. It gives me a way to quickly iterate and prove to myself that the changes I’m making are moving me in the right direction, and it lets me build the test up over time as I’m implementing the feature.

# frozen_string_literal: true

require 'rails_helper' include Warden::Test::Helpers

RSpec.feature 'Advanced Search', type: :feature, js: true, clean: true do context 'an unauthenticated user' do scenario 'advanced search basic sanity check' do visit '/advanced' fill_in('Title', with: 'ambitious aardvark') find('#advanced-search-submit').click expect(page).to have_content('ambitious aardvark') expect(page).to have_content('No results found for your search') end end end

Installing blacklight_advanced_search

I follow the blacklight_advanced_search basic instructions:

Add to your application's Gemfile:

gem "blacklight_advanced_search"

then run 'bundle install'. Then run:

rails generate blacklight_advanced_search

Note: It will offer to generate a new basic search partial for you. You do NOT want it. It will break the basic search in Hyku. If you end up with a new app/views/catalog/_search_form.html.erb just delete it and add by hand anywhere you want a link to advanced search.

So, I follow the installation instructions, and then I run my feature spec. The first time I do, I get this error:

undefined method facets_for_advanced_search_form for Hyrax::CatalogSearchBuilder (NoMethodError)`

So, clearly, I’m missing some configuration.

Tracing the error

I drop a byebug in at the line indicated by the stack trace. In this case: blacklight-6.23.0/lib/blacklight/search_builder.rb:147

[142, 151] in /usr/local/rvm/gems/ruby-2.5.8/gems/blacklight-6.23.0/lib/blacklight/search_builder.rb
   142:     #
   143:     # @return a params hash for searching solr.
   144:     def processed_parameters
   145:       request.tap do |request_parameters|
   146:     byebug
=> 147:         processor_chain.each do |method_name|
   148:           send(method_name, request_parameters)
   149:         end
   150:       end
   151:     end
(byebug) request_parameters
{"facet.field"=>[], "facet.query"=>[], "facet.pivot"=>[], "fq"=>[], "hl.fl"=>[]}
(byebug) processor_chain
[:default_solr_parameters, :add_query_to_solr, :add_facet_fq_to_solr, :add_facetting_to_solr, :add_solr_fields_to_query, :add_paging_to_solr, :add_sorting_to_solr, :add_group_config_to_solr, :add_facet_paging_to_solr, :add_access_controls_to_solr_params, :filter_models, :only_active_works, :add_access_controls_to_solr_params, :show_works_or_works_that_contain_files, :show_only_active_records, :filter_collection_facet_for_access, :facets_for_advanced_search_form]

That : facets_for_advanced_search_form at the end of the processor chain seems like a likely culprit, so I’m going to remove it and see if that gets my form to render. Initially, I’m just going to add a next right in place, to see if this fixes the problem, before I go to the trouble of figuring out the right way to do it:

    def processed_parameters
      request.tap do |request_parameters|
        processor_chain.each do |method_name|
          next if method_name == :facets_for_advanced_search_form
          send(method_name, request_parameters)

And now my form renders!

So I’ve proven to myself that the problem is this missing facets_for_advanced_search_form method, but how to fix that in a maintainable way?

Override as little as possible to keep local code maintainable

Let’s go look at the class mentioned in the error message: Hyrax::CatalogSearchBuilder

Here is the source of that method: https://github.com/samvera/hyrax/blob/130c4e600318a9194725b35dd0fd6e19e5108dd9/wp-content/search_builders/hyrax/catalog_search_builder.rb

Notice the excellent guidance provided here:

# If you plan to customize the base catalog search builder behavior (e.g. by
# adding a mixin module provided by a blacklight extension gem), inheriting this
#  class, customizing behavior, and reconfiguring `CatalogController` is the
# preferred mechanism.

Sounds good to me. Let’s try that!

So, I’m going to start by writing a test, of course. I go look at how the search builders are set up in hyrax, and I make this very basic test in spec/search_builders/ntl_search_builder_spec.rb (note that this is adapted from the test setup at https://github.com/samvera/hyrax/blob/130c4e600318a9194725b35dd0fd6e19e5108dd9/spec/search_builders/hyrax/collection_search_builder_spec.rb... I did not attempt to write this off the top of my head.)

# frozen_string_literal: true
RSpec.describe NtlSearchBuilder do
  let(:scope) do
    double(blacklight_config: CatalogController.blacklight_config,
           current_ability: ability)
  let(:user) { create(:user) }
  let(:ability) { ::Ability.new(user) }
  let(:access) { :read }
  let(:builder) { described_class.new(scope).with_access(access) }
  it "can be instantiated" do
    expect(builder).to be_instance_of(described_class)

This test doesn’t do anything except verify that the class is set up correctly, but that is better than nothing, and it gives us a place where we can flesh out behavior as we need it. I run it, and of course it fails, for the expected reasons: NameError: uninitialized constant NtlSearchBuilder

I then define the class, by making this file at app/search_builders/ntl_search_builder.rb:

class NtlSearchBuilder < Hyrax::CatalogSearchBuilder


And now my test passes!

Now, we set CatalogController to use this new class:

    # Use locally customized NtlSearchBuilder so we can enable blacklight_advanced_search
    config.search_builder_class = NtlSearchBuilder

And I add the facets_for_advanced_search_form method to my NtlSearchBuilder class, just copying the method (https://github.com/projectblacklight/blacklight_advanced_search/blob/master/lib/blacklight_advanced_search/advanced_search_builder.rb#L77-L96)

I also add the lines that the Blacklight advanced search generator installed into app/models/search_builder.rb. This is the bit that actually combines the Advanced Search query capabilities with the Hyrax provided search capabilities (e.g., search gated by permissions, workflow, group membership, etc.)

Now my class looks like this:

# A locally defined search builder, which will allow us to customize the search
# behavior of this application. In particular, this is needed to allow us to
# use blacklight_advanced_search.
class NtlSearchBuilder < Hyrax::CatalogSearchBuilder
  include Blacklight::Solr::SearchBuilderBehavior
  include BlacklightAdvancedSearch::AdvancedSearchBuilder
  self.default_processor_chain += [:add_advanced_parse_q_to_solr, :add_advanced_search_to_solr]

  # A Solr param filter that is NOT included by default in the chain,
   # but is appended by AdvancedController#index, to do a search
   # for facets _ignoring_ the current query, we want the facets
   # as if the current query weren't there.
   # Also adds any solr params set in blacklight_config.advanced_search[:form_solr_parameters]
   def facets_for_advanced_search_form(solr_p)
     # ensure empty query is all records, to fetch available facets on entire corpus
     solr_p["q"]            = '{!lucene}*:*'
     # explicitly use lucene defType since we are passing a lucene query above (and appears to be required for solr 7)
     solr_p["defType"]      = 'lucene'
     # We only care about facets, we don't need any rows.
     solr_p["rows"]         = "0"

     # Anything set in config as a literal
     if blacklight_config.advanced_search[:form_solr_parameters]

And now my advanced search page renders and my feature test for advanced search passes.

One last step: a more robust feature test with real data

One last step: I flesh out my advanced search feature spec a bit, including loading some actual client provided data, ensuring the advanced search feature really does work as expected. Here is the feature spec with some of that added:

# frozen_string_literal: true

require 'rails_helper'
include Warden::Test::Helpers

RSpec.feature 'Advanced Search', type: :feature, js: true, clean: true do
  context 'empty solr index' do
    scenario 'basic search sanity check' do
      visit '/'
      fill_in('q', with: 'ambitious aardvark')
      expect(page).to have_content('ambitious aardvark')
      expect(page).to have_content('No results found for your search')
    scenario 'advanced search basic sanity check' do
      visit '/advanced'
      fill_in('Title', with: 'ambitious aardvark')
      expect(page).to have_content('ambitious aardvark')
      expect(page).to have_content('No results found for your search')

  # Load solr data from Department of Transportation import testing and ensure advanced
  # search works reasonably well with this data.
  context 'with data' do
    let!(:admin_set_collection_type) { FactoryBot.create(:admin_set_collection_type) }
    let!(:user_collection_type) { FactoryBot.create(:user_collection_type) }

    before do
      solr = Blacklight.default_index.connection
      sample_records = JSON.parse(File.open(File.join(fixture_path, 'solr', 'dot-sample-data.json')).read)
      docs = sample_records["response"]["docs"]
      docs.each do |doc|
    scenario 'basic search sanity check' do
      visit '/'
      number_of_search_results = find('span.page_entries').find_all('strong').last.text
      expect(number_of_search_results).to eq('17')
    it "searches by title" do
      visit '/advanced'
      fill_in('Title', with: 'habitat')
      number_of_search_results = find('span.page_entries').find_all('strong').last.text
      expect(number_of_search_results).to eq('1')
Bess Sadler
Bess Sadler