Freelancing Gods 2014

God
09 Jul 2013

Rewriting Thinking Sphinx: Middleware, Glazes and Panes

Time to discuss more changes to Thinking Sphinx with the v3 releases – this time, the much improved extensibility.

There have been a huge number of contributors to Thinking Sphinx over the years, and each of their commits are greatly appreciated. Sometimes, though, the pull requests that come in cover extreme edge cases, or features that are perhaps only useful to the committer. But running your own hacked version of Thinking Sphinx is not cool, and then you’ve got to keep an especially close eye on new commits, and merge them in manually, and… blergh.

So instead, we now have middleware, glazes and panes.

Middleware

The middleware pattern is pretty well-established in the Ruby community, thanks to Rack – but it’s started to crop up in other libraries too (such as Mike Perham’s excellent Sidekiq).

In Thinking Sphinx, middleware classes are used to process search requests. The default set of middleware are as follows:

  • ThinkingSphinx::Middlewares::StaleIdFilter adding an attribute filter to hide search results that are known to not match any ActiveRecord objects.
  • ThinkingSphinx::Middlewares::SphinxQL generates the SphinxQL query to send to Sphinx.
  • ThinkingSphinx::Middlewares::Geographer modifies the SphinxQL query with geographic co-ordinates if they’re provided via the :geo option.
  • ThinkingSphinx::Middlewares::Inquirer sends the constructed SphinxQL query through to Sphinx itself.
  • ThinkingSphinx::Middlewares::UTF8 ensures all string values returned by Sphinx are encoded as UTF-8.
  • ThinkingSphinx::Middlewares::ActiveRecordTranslator translates Sphinx results into their corresponding ActiveRecord objects.
  • ThinkingSphinx::Middlewares::StaleIdChecker notes any Sphinx results that don’t have corresponding ActiveRecord objects, and retries the search if they exist.
  • ThinkingSphinx::Middlewares::Glazier wraps each search result in a glaze if there’s any panes set for the search (read below for an explanation on this).

Each middleware does its thing, and then passes control through to the next one in the chain. If you want to create your own middleware, your class must respond to two instance methods: initialize(app) and call(contexts).

If you subclass from ThinkingSphinx::Middlewares::Middleware you’ll get the first for free. contexts is an array of search context objects, which provide access to each search object along with the raw search results and other pieces of information to note between middleware objects. Middleware are written to handle multiple search requests, hence why contexts is an array.

If you’re looking for inspiration on how to write your own middleware, have a look through the source – and here’s an extra example I put together when considering approaches to multi-tenancy.

Glazes and Panes

Sometimes it’s useful to have pieces of metadata associated with each search result – and it could be argued the cleanest way to do this is to attach methods directly to each ActiveRecord instance that’s returned by the search.

But inserting methods on objects on the fly is, let’s face it, pretty damn ugly. But that’s precisely what older versions of Thinking Sphinx do. I’ve never liked it, but I’d never spent the time to restructure things to work around that… until now.

There are now a few panes available to provide these helper methods:

  • ThinkingSphinx::Panes::AttributesPane provides a method called sphinx_attributes which is a hash of the raw Sphinx attribute values. This is useful when your Sphinx attributes hold complex values that you don’t want to re-calcuate.
  • ThinkingSphinx::Panes::DistancePane provides the identical distance and geodist methods returning the calculated distance between lat/lng geographical points (and is added automatically if the :geo option is present).
  • ThinkingSphinx::Panes::ExcerptsPane provides access to an excerpts method which you can then chain any call to a method on the search result – and get an excerpted value returned.
  • ThinkingSphinx::Panes::WeightPane provides the weight method, returning Sphinx’s calculated relevance score.

None of these panes are loaded by default – and so the search results you’ll get are the actual ActiveRecord objects. You can add specific panes like so:

# For every search
ThinkingSphinx::Configuration::Defaults::PANES << ThinkingSphinx::Panes::WeightPane

# Or for specific searches:
search = ThinkingSphinx.search('pancakes')
search.context[:panes] << ThinkingSphinx::Panes::WeightPane

When you do add at least pane into the mix, though, the search result gets wrapped in a glaze object. These glaze objects direct any methods called upon themselves with the following logic:

  • If the search result responds to the given method, send it to that search result.
  • Else if any pane responds to the given method, send it to the pane.
  • Otherwise, send it to the search result anyway.

This means that your ActiveRecord instances take priority – so pane methods don’t overwrite your own code. It also allows for method_missing metaprogramming in your models (and ActiveRecord itself) – but otherwise, you can get access to the useful metadata Sphinx can provide, without monkeypatching objects on the fly.

If you’re writing your own panes, the only requirement is that the initializer must accept three arguments: the search context, the underlying search result object, and a hash of the raw values from Sphinx. Again, the source code for the panes is not overly complex – so have a read through that for inspiration.

I’m always keen to hear about any middleware or panes other people write – so please, if you do make use of either of these approaches, let me know!

Comments

10 responses to this article

18 Oct 2013
lazylacquerista.com said:

Heyy there! I know this is somewhat off topic but I wass wondering iif you knew where I cohld get a captcha plugin for my comment form?
I’m using the same blog platform as yours and I’m having problems finding one?
Thanks a lot!

29 Oct 2013
likes for facebook said:

hi!,I like your writing so much! percentage we keep in
touch extra about your post on AOL? I need an expert in this house to
solve my problem. May be that is you! Looking ahead to see you.

. likes for facebook Absolutely no Security password and also Without Next.

buy us facebook likes it can be an exceptionally smart financial commitment
for your company.
Clients can easily buy facebook fans through a multitude of inexpensive expert services.

buy likes on facebook Safe and straightforward from The major facebook endorsements lending
institution.

04 Nov 2013
webpage said:

I think this is one of the most significant information for me.
And i am glad reading your article. But wanna remark on
few general things, The website style is perfect, the articles is really great : D.
Good job, cheers

16 Nov 2013
youtube views said:

Hmm it seems like your website ate my first
comment (it was super long) so I guess I’ll just sum it up what I wrote and say,
I’m thoroughly enjoying your blog. I as well am an aspiring blog writer but I’m still new to everything.
Do you have any suggestions for newbie blog writers? I’d certainly appreciate it.
You are able to youtube views via several dealers from very affordable rates.

you always can buy youtube views cheap from legitimate suppliers for cheap and secure
increase youtube views along with quickly to your youtube Account through finest Providers.

buy youtube likes, pick the deal this best suits your needs.

16 Feb 2014
Triton Tank said:

Hi there, just became aware of your blog through Google,
and found that it’s truly informative. I’m gonna watch out for brussels.
I will be grateful if you continue this in future.

Lots of people will be benefited from your writing.
Cheers!

17 Feb 2014
piano lessons said:

Howdy would you mind letting me know which web host you’re working with?
I’ve loaded your blog in 3 different browsers and I must say this blog
loads a lot quicker then most. Can you recommend a good hosting provider at a fair price?
Many thanks, I appreciate it!

04 Mar 2014
Virtual Router said:

The whole of the BBC network available on your phone, laptop,
or internet TV. With the capabilities to run on multiple host operating systems, Sun’s Virtual – Box runs many
former and future operating systems. STP ensures the loop free transportation of packets from source to destination.

26 Mar 2014
free google advertising said:

Hi, its pleasant paragraph on the topic of media
print, we all know media is a wonderful source of information.

04 Apr 2014
SEO Company said:

Very nice post. I just stumbled upon your blog and wanted to say that I’ve truly enjoyed browsing your blog posts.
In any case I’ll be subscribing to your rss feed and I hope you write again very soon!

17 Apr 2014
how to increase penis girth said:

Heya! I just wanted to ask if you ever have any
trouble with hackers? My last blog (wordpress) was hacked and I ended up losing months of hard work
due to no backup. Do you have any solutions to stop hackers?

Leave a Comment

Comments are formatted using Textile. Please be respectful of others when posting comments. Be nice.

RssSubscribe to the RSS feed

Related Links

Related Posts

About Freelancing Gods

Freelancing Gods is written by , who works on the web as a web developer in Melbourne, Australia, specialising in Ruby on Rails.

In case you're wondering what the likely content here will be about (besides code), keep in mind that Pat is passionate about the internet, music, politics, comedy, bringing people together, and making a difference. And pancakes.

His ego isn't as bad as you may think. Honest.

Here's more than you ever wanted to know.

Ruby on Rails Projects

Other Sites

Creative Commons Logo All original content on this site is available through a Creative Commons by-nc-sa licence.