Freelancing Gods 2015

09 Jul 2013

Rewriting Thinking Sphinx: Middleware, Glazes and Panes

Time to discuss more changes to Thinking Sphinx with the v3 releases – this time, the much improved extensibility.

There have been a huge number of contributors to Thinking Sphinx over the years, and each of their commits are greatly appreciated. Sometimes, though, the pull requests that come in cover extreme edge cases, or features that are perhaps only useful to the committer. But running your own hacked version of Thinking Sphinx is not cool, and then you’ve got to keep an especially close eye on new commits, and merge them in manually, and… blergh.

So instead, we now have middleware, glazes and panes.


The middleware pattern is pretty well-established in the Ruby community, thanks to Rack – but it’s started to crop up in other libraries too (such as Mike Perham’s excellent Sidekiq).

In Thinking Sphinx, middleware classes are used to process search requests. The default set of middleware are as follows:

  • ThinkingSphinx::Middlewares::StaleIdFilter adding an attribute filter to hide search results that are known to not match any ActiveRecord objects.
  • ThinkingSphinx::Middlewares::SphinxQL generates the SphinxQL query to send to Sphinx.
  • ThinkingSphinx::Middlewares::Geographer modifies the SphinxQL query with geographic co-ordinates if they’re provided via the :geo option.
  • ThinkingSphinx::Middlewares::Inquirer sends the constructed SphinxQL query through to Sphinx itself.
  • ThinkingSphinx::Middlewares::UTF8 ensures all string values returned by Sphinx are encoded as UTF-8.
  • ThinkingSphinx::Middlewares::ActiveRecordTranslator translates Sphinx results into their corresponding ActiveRecord objects.
  • ThinkingSphinx::Middlewares::StaleIdChecker notes any Sphinx results that don’t have corresponding ActiveRecord objects, and retries the search if they exist.
  • ThinkingSphinx::Middlewares::Glazier wraps each search result in a glaze if there’s any panes set for the search (read below for an explanation on this).

Each middleware does its thing, and then passes control through to the next one in the chain. If you want to create your own middleware, your class must respond to two instance methods: initialize(app) and call(contexts).

If you subclass from ThinkingSphinx::Middlewares::Middleware you’ll get the first for free. contexts is an array of search context objects, which provide access to each search object along with the raw search results and other pieces of information to note between middleware objects. Middleware are written to handle multiple search requests, hence why contexts is an array.

If you’re looking for inspiration on how to write your own middleware, have a look through the source – and here’s an extra example I put together when considering approaches to multi-tenancy.

Glazes and Panes

Sometimes it’s useful to have pieces of metadata associated with each search result – and it could be argued the cleanest way to do this is to attach methods directly to each ActiveRecord instance that’s returned by the search.

But inserting methods on objects on the fly is, let’s face it, pretty damn ugly. But that’s precisely what older versions of Thinking Sphinx do. I’ve never liked it, but I’d never spent the time to restructure things to work around that… until now.

There are now a few panes available to provide these helper methods:

  • ThinkingSphinx::Panes::AttributesPane provides a method called sphinx_attributes which is a hash of the raw Sphinx attribute values. This is useful when your Sphinx attributes hold complex values that you don’t want to re-calcuate.
  • ThinkingSphinx::Panes::DistancePane provides the identical distance and geodist methods returning the calculated distance between lat/lng geographical points (and is added automatically if the :geo option is present).
  • ThinkingSphinx::Panes::ExcerptsPane provides access to an excerpts method which you can then chain any call to a method on the search result – and get an excerpted value returned.
  • ThinkingSphinx::Panes::WeightPane provides the weight method, returning Sphinx’s calculated relevance score.

None of these panes are loaded by default – and so the search results you’ll get are the actual ActiveRecord objects. You can add specific panes like so:

# For every search
ThinkingSphinx::Configuration::Defaults::PANES << ThinkingSphinx::Panes::WeightPane

# Or for specific searches:
search ='pancakes')
search.context[:panes] << ThinkingSphinx::Panes::WeightPane

When you do add at least pane into the mix, though, the search result gets wrapped in a glaze object. These glaze objects direct any methods called upon themselves with the following logic:

  • If the search result responds to the given method, send it to that search result.
  • Else if any pane responds to the given method, send it to the pane.
  • Otherwise, send it to the search result anyway.

This means that your ActiveRecord instances take priority – so pane methods don’t overwrite your own code. It also allows for method_missing metaprogramming in your models (and ActiveRecord itself) – but otherwise, you can get access to the useful metadata Sphinx can provide, without monkeypatching objects on the fly.

If you’re writing your own panes, the only requirement is that the initializer must accept three arguments: the search context, the underlying search result object, and a hash of the raw values from Sphinx. Again, the source code for the panes is not overly complex – so have a read through that for inspiration.

I’m always keen to hear about any middleware or panes other people write – so please, if you do make use of either of these approaches, let me know!


19 responses to this article

04 Jun 2014
web hosting promo codes said:

If you truly want individuals to buy your product then creating your USP will
let you get there so as to truly improve your
business. I thought we were looking at the strangest sounding words when I first heard
them. You go for to use the very best features like innovative site building tools, customized email accounts, blog and podcast tools, photo
albums and also a host of those kinds of services, completely targeted at getting your online business over the internet inside the best
possible way.

18 Jun 2014
Winnie said:

Most enough time when someone’s laptop stops working
or becomes unusable due to viruses along with other software problems they will simply dump it and get a
whole new one. Visit PC Repair Shack to support small business and
acquire exactly what you need in computer repair. You can certainly produce a call
and order assistance in addition to their agents will visit you at your home and repair
your systems.

23 Sep 2014
Stefanie said:

Asking questions are actually nice thing if you are not understanding something completely, except this article offers good understanding yet.

14 Feb 2015
youtube said:

Hi, this weekend is nice for me, since this time i am reading this
great educational paragraph here at my residence.

01 Sep 2014
westerville dentist said:

Whɑt’s up, I reaɗ your Ƅlog on a regular basis.
Yourr writing style is awesome, keep uup thhe
good work!

20 Jun 2014
Tonia said:

Try to give the list from the features which can be present in your device.
If you might be living in Jaipur, used cars in Jaipur
is a perfect selection for you. Obviously, a trade
in on your used car may be the astest option.

28 Sep 2014
internet marketing tactic said:

You should be a part of a contest for one of the greatest sites online.
I will recommend this web site!

12 Aug 2014
Lynchburg Realtors said:

Great article! Thhat is the type of info that are mean to be shareed around the internet.
Disgrace on Gogle for now not positioning this publish upper!
Come onn over and seek advice from my website .
Thank you =)

08 Jun 2014
how To get rid Of acne scars said:

I just now discovered your blog post and now I’m among your followers.

23 Jun 2014
E Cigarette information said:

Majority of big companies have minimal, full and half strength.
For those not familiar, an e – Cigarette (or ecig) is definitely an electronic
vaporizer which looks, feels, and functions almost like a normal tobacco cigarette.
There can be a 75-day comment period that permits the public
to discuss the proposed regulations, but some e-cigarette users are looking forward to
the CASAA to organize their response before they leave a comment.

26 Feb 2015 has stopped said:

Appreciating the dedication you put into your site and detailed information you offer.
It’s great to come across a blog every once in a while that isn’t the same old rehashed information. Fantastic read!
I’ve saved your site and I’m adding your RSS feeds to my Google account.

07 Nov 2014 said:

Hello very nice blog!! Man .. Excellent .. Amazing .. I’ll bookmark
your website and take the feeds additionally?
I’m glad to search out a lot of useful information right here within the submit,
we want work out more techniques on this regard, thank you
for sharing. . . . . .

20 Sep 2014
Damon said:

I have read so many articles or reviews about the blogger lovers except this
article is actually a pleasant paragraph, keep it up.

16 Jun 2014
post_54421 said:

Article writing is also a fun, if you know afterward you can write
or else it is complicated to write.

06 Sep 2014
facebook said:

Hi! Do you know if they make any plugins to help with SEO?
I’m trying to get my blog to rank for some targeted keywords
but I’m not seeing very good success. If you
know of any please share. Cheers! facebook

22 Sep 2014
วางแผนเขียน sop said:

Top universities in Norway proffer high level
of flexibility and excellent quality courses for students who yearn to study in Norway.
Just keep in mind that these differences can be both good or bad, it actually depends on your perspective and also your upbringing.
It is inevitable that you will build life-long friendships
as you study in London with people from all over the UK, Europe
and the rest of the World.

22 Sep 2014
Enid said:

Hmm is anyone else experiencing problems with the images on this blog loading? I’m trying to figure out if its a problem on my end or if it’s the blog. Any feed-back would be greatly appreciated.

14 Sep 2014
ครีมหน้าใส said:

For most recent information you have to pay a visit internet and on the web
I found this web site as a best web site for newest updates.

05 Oct 2014
articles rewrite said:

Do you mind if I quote a couple of your articles as long as I provide credit and sources back to your website?
My blog is in the very same niche as yours and my users would certainly benefit
from some of the information you provide here. Please let me
know if this okay with you. Thank you!

Leave a Comment

Comments are formatted using Textile. Please be respectful of others when posting comments. Be nice.

RssSubscribe to the RSS feed

Related Links

Related Posts

About Freelancing Gods

Freelancing Gods is written by , who works on the web as a web developer in Melbourne, Australia, specialising in Ruby on Rails.

In case you're wondering what the likely content here will be about (besides code), keep in mind that Pat is passionate about the internet, music, politics, comedy, bringing people together, and making a difference. And pancakes.

His ego isn't as bad as you may think. Honest.

Here's more than you ever wanted to know.

Open Source Projects

Other Sites

Creative Commons Logo All original content on this site is available through a Creative Commons by-nc-sa licence.