Deployment

Once we’ve installed Sphinx and installed ThinkingSphinx on your production server, as well as setting up all of our indexes, we can deploy it to our production servers.

Heroku

If you’re using Heroku, then you cannot install Sphinx directly within your application. There is an add-on that allows you to use Sphinx with your Heroku apps though: Flying Sphinx.

Basics

Essentially, the following steps need to be performed for a deployment:

  • stop Sphinx searchd (if it’s running)
  • generate Sphinx configuration
  • start Sphinx searchd
  • ensure index is regularly rebuilt

Configuring Sphinx for our production environment includes setting where the PID file and the index files are stored. In your config/thinking_sphinx.yml file, set up the following additional parameters:

production:
  pid_file: /path/to/app/shared/tmp/searchd.pid
  indices_location: /path/to/app/shared/db/sphinx
  configuration_file: /path/to/app/shared/production.sphinx.conf
  binlog_path: /path/to/app/shared/binlog

Please make sure all of the above files (configuration file, pid file, index files, binlog path) are located in a shared directory (instead of a directory tied to a specific deployed release). Otherwise, running rake tasks will become difficult and unreliable.

Symlinked directories are strongly discouraged as an alternative to (or in combination with) shared paths. Symlinked paths can be translated to release-specific paths when generating configuration, and this can lead to data inconsistency problems.

Thinking Sphinx v1/v2

Note: If you are using an older version of Thinking Sphinx, then the file is config/sphinx.yml, and the second setting is searchd_file_path (and the third and fourth can be skipped):

production:
  pid_file: /path/to/app/shared/tmp/searchd.pid
  searchd_file_path: /path/to/app/shared/db/sphinx

You’ll want to make sure that the application shared folder has db and tmp subdirectories (or whatever is appropriate for your settings). You’ll also want to double check the permissions of these folders so that the user the application and searchd both runs as can write to both folders.

Before deploying, we generally assume that the Sphinx searchd search daemon is running. You may need to manually configure and run the daemon the first deployment with ThinkingSphinx support added.

Deploying With Capistrano

Deploying via Capistrano is simplified by the included recipe file that comes with the ThinkingSphinx plugin.

The first step is to include the recipe in order to define the necessary tasks for us:

# If you're using Thinking Sphinx 3.x or newer:
require 'thinking_sphinx/capistrano'
# If you're using Thinking Sphinx 2.x as a gem (Rails 3 way):
require 'thinking_sphinx/deploy/capistrano'
# If you're using Thinking Sphinx 1.x or 2.x as a plugin:
require 'vendor/plugins/thinking-sphinx/recipes/thinking_sphinx'

Thinking Sphinx v1/v2

Note: If you are using an older version of Thinking Sphinx and you've not set your paths up to be outside of a specific deployed release directory, then you'll need to add some extra code to your deploy.rb file to make sure that Sphinx is properly configured, indexed, and started on each deploy.

It is far better, though, to use the configuration options mentioned above and avoid release-specific paths.

before 'deploy:update_code', 'thinking_sphinx:stop'
after  'deploy:update_code', 'thinking_sphinx:start'

namespace :sphinx do
  desc "Symlink Sphinx indexes"
  task :symlink_indexes, :roles => [:app] do
    run "ln -nfs #{shared_path}/db/sphinx #{release_path}/db/sphinx"
  end
end

after 'deploy:finalize_update', 'sphinx:symlink_indexes'

The above makes sure we stop the Sphinx searchd search daemon before we update the code. After the code is updated, we reconfigure Sphinx and then restart. You should also setup a `cron` job to keep the indexes up-to-date.

Regularly Processing the Indices

One of the side effects of the Sphinx indexing methodology is that it is necessary to regularly process your indices in order to be able to search with recent changes. In order to do this, we set up a cron job to run the appropriate command.

In your /etc/crontab file, add the following line to the bottom:

0 * * * * deploy  cd /path/to/app/current && /usr/local/bin/rake ts:index RAILS_ENV=production

Also, you can use the whenever gem to control your Cron tasks from the Rails app. The same job for whenever in config/schedule.rb looks like this:

every 60.minutes do
  rake "ts:index"
end