Caching Your Photographs

If you have at some point followed my now fairly ancient rFlickr tutorial you may have noticed that your photo page loads quite slowly, and that my photo page loads fairly quickly. To get my page to load as quickly as it does required a small custom caching method and a willingness on my part to sacrifice some bandwidth. Here’s how I did it.

This tutorial assumes that you have already worked through the previously mentioned rFlickr tutorial and have something similar to it set up. It also assumes that you have some knowledge of Rails, not that my knowledge was particularly wide ranging at the point I wrote this caching method.

First of all, you will need a table in your database to store the information about your photographs, I suggest the structure illustrated in the migration below;

class CreatePhotos < ActiveRecord::Migration
    def self.up
        create_table :photos, :id => false do |t|
            t.column "flickr_id", :string, :limit => 25, :null => false
            t.column "title", :string, :limit => 250
            t.column "description", :text
            t.column "url", :string, :limit => 250
        end
    
        add_index :photos, :flickr_id
    end
  
    def self.down
        drop_table :photos
    end
end

Once you have created this table you will need to create some folders to store the cached images, I created the following folders and will be using them throughout this tutorial;

#{RAILS_ROOT}/public/images/flickr_cache/small/
#{RAILS_ROOT}/public/images/flickr_cache/large/

Then generate the model for this photos table;

$ cd /your/rails/application
$ ./script/generate model Photo

Your view from the first tutorial can remain almost the same (details at the end of the post), however, to see the greatest speed improvement I suggest caching it, i.e;

<% cache do %>
... Your view code here. ...
<% end %>

Then modify your view method in your photography controller to read something like;

def view
    unless read_fragment({})
        check_cache
        @photos = Photo.find(:all)
    end
end

The above code will make sure that a cached photography page doesn’t already exist, if it doesn’t, then and only then will it check that the photograph cache is up to date and query the database.

We have not yet created a ‘check_cache’ method, this method is the core method to make the photography page load much, much faster, even when the photography page’s cache does not exist. The method should be placed as the last method in your photography controller, the code is as follows;

private
def check_cache
    if ENV['RAILS_ENV'] == 'production'
        flickr = Flickr.new(RAILS_ROOT + "/config/flickr.cache", FLICKR_API_KEY, FLICKR_SHARED_SECRET)
        @photos = flickr.people.getPublicPhotos(flickr.people.findByUsername(FLICKR_USERNAME))
      
        @db_photos = Array.new
        Photo.find(:all).each { |p| @db_photos.push(p.flickr_id) }

        for photo in @photos.reverse
            if !@db_photos.include?(photo.id)
       
                db_photo = Photo.new
                db_photo.flickr_id = photo.id.to_i
                db_photo.title = photo.flickr.photos.getInfo(photo.id).title
                db_photo.description = photo.flickr.photos.getInfo(photo.id).description
                db_photo.url = photo.flickr.photos.getInfo(photo.id).urls.values[0]
          
                db_photo.save
          
                open(File.expand_path("#{RAILS_ROOT}/public/images/flickr_cache/small/" + photo.id + ".jpg"),"w").write(open(photo.url('s')).read)
                open(File.expand_path("#{RAILS_ROOT}/public/images/flickr_cache/large/" + photo.id + ".jpg"),"w").write(open(photo.url).read)
            end
        end
    end
end

The following code will only run if you are in production mode (and probably test mode too). It will then load the necessary information from Flickr using the methods outlined in the rFlickr tutorial post. The method then iterates through the collection of photos from Flickr in reverse, so they appear in the same order in the database as the order they appear on Flickr.

It will then check if the photo already exists in the database, if it does not it will store a copy of the photograph’s information in the database and download previews of the images from Flickr to your server, previews of images can then be loaded from your server rather than Flickr’s slow servers .

To take advantage of the cache you will also need to modify your view to access the thumbnails from your newly created local repository rather than from Flickr’s servers, i.e;

<% for photo in @photos.reverse %>
    <%= image_tag("/images/flickr_cache/small/" + photo.flickr_id + ".jpg", :alt => photo.title) %>
<% end %>

Hope this goes some way to helping you improve the speed of your website.

Check back soon.

Update: As I have just been reminded in the comments, I forgot a piece of code to make this tutorial work correctly. You should put the line;

require 'open-uri'

Either in the bottom of you ‘environment.rb’ file or just under the ‘class’ line in the controller that is responsible for your photography page.

Update (3rd May 09): The code in the ‘check_cache’ method has been updated slightly, it now makes less round trips to the database, should speed things up if your DB server is in a different location to your application server. Also, any redirection problems you may have faced before should be solved by the new rFlickr ruby gem that has support for ‘farm’ URLs built in.

Oops

Due to a massive oversight on my part at the point I implemented a user groups system on the website it would appear, for standard users at least, that the comments system has not been functioning correctly. The problem has now been solved and the comments system should be working correctly. Apologies to anyone that have ever tried to post a comment on the website for it not to work.

Check back soon.

About Time Too

It’s been well over a month since I managed to write a post now so I thought it was about time I started the cogs whirring again, especially as all my assignments are now finished and handed in.

Even though there has been a lack of posts in the recent month there hasn’t been a lack of feverish activity. First on the books was the release of Set Icon 0.2, this release just bought basic bug fixes and a much improved authentication system. This was followed in recent weeks by the release of Set Icon 0.3, in this release I fixed yet more bugs and introduce an option to remove a custom icon from a hard drive. You can, as always, download Set Icon from it’s download page.

On a more website orientated note, I have moved the website to new servers. Prime Hosting weren’t terribly Ruby on Rails centric in the end, hence the continual website downtime. The website is now hosted with media72. They have proven themselves over the last month to be far more reliable hosts than Prime, even though I am hosted on their beta testing server.

Along with the change of host I have modified the way in which the website is run, updates to the website are now performed via Capistrano, I hope to write a little more on this system in a later post to explain how it can help improve your Ruby on Rails development.

Don’t expect too many posts or software updates over the next two weeks, unfortunately I have a series of exams that require some serious revision, however, if I get really bored of revising you might see some posts / software appearing.

I’ve also recently acquired a new lens for my camera, a Tokina 12-24mm f/4. As part of my endeavor to expand what I write about I will hopefully (assuming I remember) write a small review about the lens as it took me a long time to find any concrete opinions on it before I purchased it.

That’s all I can think of to write for now, no doubt I will think of something else eventually.

Check back soon.