Caching Your Photographs

If you have at some point followed my now fairly ancient rFlickr tutorial you may have noticed that your photo page loads quite slowly, and that my photo page loads fairly quickly. To get my page to load as quickly as it does required a small custom caching method and a willingness on my part to sacrifice some bandwidth. Here’s how I did it.

This tutorial assumes that you have already worked through the previously mentioned rFlickr tutorial and have something similar to it set up. It also assumes that you have some knowledge of Rails, not that my knowledge was particularly wide ranging at the point I wrote this caching method.

First of all, you will need a table in your database to store the information about your photographs, I suggest the structure illustrated in the migration below;

class CreatePhotos < ActiveRecord::Migration
    def self.up
        create_table :photos, :id => false do |t|
            t.column "flickr_id", :string, :limit => 25, :null => false
            t.column "title", :string, :limit => 250
            t.column "description", :text
            t.column "url", :string, :limit => 250
        end
    
        add_index :photos, :flickr_id
    end
  
    def self.down
        drop_table :photos
    end
end

Once you have created this table you will need to create some folders to store the cached images, I created the following folders and will be using them throughout this tutorial;

#{RAILS_ROOT}/public/images/flickr_cache/small/
#{RAILS_ROOT}/public/images/flickr_cache/large/

Then generate the model for this photos table;

$ cd /your/rails/application
$ ./script/generate model Photo

Your view from the first tutorial can remain almost the same (details at the end of the post), however, to see the greatest speed improvement I suggest caching it, i.e;

<% cache do %>
... Your view code here. ...
<% end %>

Then modify your view method in your photography controller to read something like;

def view
    unless read_fragment({})
        check_cache
        @photos = Photo.find(:all)
    end
end

The above code will make sure that a cached photography page doesn’t already exist, if it doesn’t, then and only then will it check that the photograph cache is up to date and query the database.

We have not yet created a ‘check_cache’ method, this method is the core method to make the photography page load much, much faster, even when the photography page’s cache does not exist. The method should be placed as the last method in your photography controller, the code is as follows;

private
def check_cache
    if ENV['RAILS_ENV'] == 'production'
        flickr = Flickr.new(RAILS_ROOT + "/config/flickr.cache", FLICKR_API_KEY, FLICKR_SHARED_SECRET)
        @photos = flickr.people.getPublicPhotos(flickr.people.findByUsername(FLICKR_USERNAME))
      
        @db_photos = Array.new
        Photo.find(:all).each { |p| @db_photos.push(p.flickr_id) }

        for photo in @photos.reverse
            if !@db_photos.include?(photo.id)
       
                db_photo = Photo.new
                db_photo.flickr_id = photo.id.to_i
                db_photo.title = photo.flickr.photos.getInfo(photo.id).title
                db_photo.description = photo.flickr.photos.getInfo(photo.id).description
                db_photo.url = photo.flickr.photos.getInfo(photo.id).urls.values[0]
          
                db_photo.save
          
                open(File.expand_path("#{RAILS_ROOT}/public/images/flickr_cache/small/" + photo.id + ".jpg"),"w").write(open(photo.url('s')).read)
                open(File.expand_path("#{RAILS_ROOT}/public/images/flickr_cache/large/" + photo.id + ".jpg"),"w").write(open(photo.url).read)
            end
        end
    end
end

The following code will only run if you are in production mode (and probably test mode too). It will then load the necessary information from Flickr using the methods outlined in the rFlickr tutorial post. The method then iterates through the collection of photos from Flickr in reverse, so they appear in the same order in the database as the order they appear on Flickr.

It will then check if the photo already exists in the database, if it does not it will store a copy of the photograph’s information in the database and download previews of the images from Flickr to your server, previews of images can then be loaded from your server rather than Flickr’s slow servers .

To take advantage of the cache you will also need to modify your view to access the thumbnails from your newly created local repository rather than from Flickr’s servers, i.e;

<% for photo in @photos.reverse %>
    <%= image_tag("/images/flickr_cache/small/" + photo.flickr_id + ".jpg", :alt => photo.title) %>
<% end %>

Hope this goes some way to helping you improve the speed of your website.

Check back soon.

Update: As I have just been reminded in the comments, I forgot a piece of code to make this tutorial work correctly. You should put the line;

require 'open-uri'

Either in the bottom of you ‘environment.rb’ file or just under the ‘class’ line in the controller that is responsible for your photography page.

Update (3rd May 09): The code in the ‘check_cache’ method has been updated slightly, it now makes less round trips to the database, should speed things up if your DB server is in a different location to your application server. Also, any redirection problems you may have faced before should be solved by the new rFlickr ruby gem that has support for ‘farm’ URLs built in.