Freelancing Gods 2013

In one of the projects I’ve been working on lately, I’ve needed to implement a decent search – and so I looked at both Ferret and Sphinx. I ended up choosing the latter, although I’m not sure why – perhaps just to be different (most people I spoke to are using ferret), or perhaps because the setup seemed easier.

The next step was to pick a Sphinx plugin to work with. Ultrasphinx seemed to have a good set of features (particularly pagination), and supported fields from associations within indexes – something critical for what we were doing.

Unfortunately, grabbing fields from associations wasn’t that easy – and the SQL generated for the Sphinx configuration file was overly complex. I could (and did) change the config file manually, but that makes half the usefulness of the plugin worthless.

So, since I had some spare time, I wrote my own plugin. Much like Rails, it favours convention over configuration – perhaps a little too much so at this point, but I do plan to make it more flexible at some point. Installation is the same as any other plugin:

script/plugin install
  http://rails-oceania.googlecode.com/svn/patallan/thinking_sphinx

An example of defining indexes (within a model class):

define_index do |index|
  index.includes.email
  index.includes(:first_name, :last_name).as.name
  index.includes.tags.key.as.tags
  index.includes.articles.content.as.article_content
end

To index the data, just use the thinkingsphinx:index rake task (aliased to ts:in) – which will also generate the configuration file on the fly. My goal is to make changing the configuration file manually unnecessary – making the index task build the configuration file helps enforce this.

And to search:

# Searching particular fields
User.search(:conditions => {:name => "Pat"})
# Or searching all fields
User.search("Pat")
# Pagination is built in
User.search("Pat", :page => (params[:page] || 1))

Paginated results can also be used by the will_paginate helper from the plugin of the same name. Current documentation can be found on this site.

I managed to use ActiveRecord’s join and associations code, which kept my plugin reasonably lean. For interactions with Sphinx’s searchd daemon, I did look at Dmytro Shteflyuk’s Ruby Sphinx Client API, but the non-ruby-like syntax irritated me, so again, I coded my own – heavily influenced by the original though (ie: he did all the hard work, not me).

There’s no support for some way to update the index pseudo-incrementally (something that is a limitation of Sphinx). If I don’t feel like the incremental updating works well enough, then I may switch to Ferret – which might lead to a Thinking Ferret plugin, perhaps. We’ll just have to wait and see.

Nov 14th 2007 – Update: I’ve just released the internal Sphinx client as its own library – Riddle.

Whenever I’ve described caching in Rails to anyone who isn’t familiar with it, I have made clear the limitations of each method:

• Page caching is only useful when the output is exactly the same for every visitor, and you don’t need to confirm user authentication
• Action caching allows you to run filters for every request – thus can be used to check if users are authenticated – but the rendered output has the same limitations as page caching
• Fragment caching, while flexible, is definitely slower than the other two options.

Obviously, it’s best to use the fastest caching possible that fits your pages. That’s rarely page caching for the sites I code. Even action caching hasn’t been viable too often. Or so I thought.

I used to use fragment caching in most of my views – generally with extra parameters to indicate user role, so it would store different versions of the fragment for each role (ie: admin user, normal user, no user). This worked reasonably well, but often I was wrapping an entire view in a <% cache do %> block – almost action caching!

In the site I was coding at the time, ausdwcon.org, I realised the best time to cache would be when no user was logged in – as that would cover the majority of requests. So, to simplify: what I wanted was caching only when a certain condition was true.

Getting methods coded for conditional caching at a fragment level was a piece of cake. At the action caching level? Well, that was trickier, but with some help from the RORO crew I got it working, and into a plugin. You can find the code in the RORO svn repository and the documentation on this site. To install:

script/plugin install
  http://rails-oceania.googlecode.com/svn/patallan/conditional_caching

One brief example so you know what to expect:

# a controller
caches_action :index, :if => :no_user?

# application.rb
def no_user?
  session[:user_id].empty?
end

Obviously the :if parameter is the key – it can be a symbol pointer to an instance method on the controller, or it can be a Proc which is evaluated in the scope of the controller.

Now, the no-user condition is the only example I can think of where my plugin is useful – if you think of others, please let me know. Keep reading though, because I’ve got another helpful hint or two to share.

I mentioned above my usual method of using fragment caching – liberal use of extra parameters. It’s actually possible to do this with action caching too – which I only found out recently, so I’m assuming there are other people out there who aren’t aware of it either.

It’s not quite as easy as the equivalent fragment caching code, as you’re using both class and object level methods, but here’s an example:

# a controller
caches_action :index, :show, :cache_path => cache_params

# application.rb
def self.cache_params
  @cache_params ||= Proc.new { |controller|
    controller.params.merge(:role => controller.current_role)
  }
end

This has actually made me cut back on usage of my plugin – because most of my pages don’t have user-specific content.

Oh, and one more thing – to cut down on user-specific content in my views, I’ve been mapping my users controller as both a normal resource and a singleton resource. This means instead of a “Your Profile” link being /users/34, it’s just /user. Makes the controller code a little tricky, and the named routes get confused, but nothing a few clever helper methods can’t fix.