Posted
03 May 2009 @ 18:31

Categories
,

Comments
2 Comments

Author
Alex

Clearing Out Old Sessions

Rails

A while ago I started setting up my websites to use ActiveRecord as a session store, this means that the session information for all the visitors to my website is placed in a table in my database. It may or may not be the best way to store sessions but it’s certainly faster than the filesystem and my VPS doesn’t really have the memory capacity for an ‘in memory’ store.

Anyway, one day I decided to perform some DB maintenance, check tables where okay etc, upon logging into the DB I noticed that my sessions table had grown quite large, almost 125,000 records, little did I realize that the sessions are persisted forever in the DB.

I didn’t think it was the best idea to keep all the session data so wrote the following script and put it in ‘lib/tasks/session.rake’;

namespace :session do
    desc "Prune old session data"
    task :prune => :environment do
        sql = ActiveRecord::Base.connection()
        sql.execute "SET autocommit=0"
        sql.begin_db_transaction
            response = sql.execute("DELETE FROM sessions WHERE `updated_at` < DATE_SUB(CURDATE(), INTERVAL 1 DAY)");
        sql.commit_db_transaction
    end
end

This gave me a ‘session:prune’ rake task. The task removes all sessions older than 1 day from the sessions table. I then added a CRON job for in the following format;

0 0 * * * cd /home/user/railsapp && rake RAILS_ENV=production session:prune > /dev/null 2>&1

The job above basically calls the ‘session:prune’ rake task at midnight every night.

The code in the task in MySQL specific but without a model representing the session table I couldn’t (or at least couldn’t think of a way) to make the code any more ruby-fied. In the event that you do have, or decide to create a session model the following code may work in your task (warning: untested);

Session.destroy_all("created_at" < (Time.now - 1.day))

Hope the above snippet solves at least one of your ActiveRecord session woes.

Check back soon.

Posted
03 May 2009 @ 17:50

Categories
,

Comments
None

Author
Alex

Clearing Out The rFlickr Cache

rFlickr

Assuming you’ve followed the Caching Your Photographs tutorial at some point, you’ll probably have had a lot of fun either deleting the cache every time you upload a new photo or you’ve written your own automated method by now. For those of you that haven’t written your own method of dumping the cache yet, here’s how I do it.

First of all, I created a ‘lib/actions’ folder in the root of my rails project. Inside this folder I created the file ‘photography_action.rb’ with the following contents;

class PhotographyAction
    def self.clear_cache
        ActionController::Base.new.expire_fragment(%r{photography.cache})
    end
end

The above fragment naming assumes that your photos are on a page called ‘photography’ if they are elsewhere, change the fragment to expire that page instead.

Fairly simple I think you’ll agree, you may also be asking yourself ‘why the extra file?’, the main reason for the new file is so that the cache clearing can be executed from a new rake task that doesn’t remove all your cached pages or from an admin page on the website.

You’ll also need to update your ‘config.load_paths’ in ‘environment.rb’. After updating, mine looks like this;

config.load_paths += %W( #{RAILS_ROOT}/app/sweepers #{RAILS_ROOT}/lib/actions )

Inside some action in some, preferably protected, controller somewhere, add the following (redirecting to anywhere you fancy);

PhotographyAction.clear_cache
redirect_to :action => 'index'

Now for the rake task. Inside the directory ‘lib/tasks’ (create it if it doesn’t exists) create the file ‘photography.rake’ then put the following code inside the file;

namespace :photo do
    namespace :cache do
        desc "Clear out photography cache"
        task :clear => :environment do
            PhotographyAction.clear_cache
        end
    end
end

You should then be able to run;

rake photo:cache:clear

From the base of your project in order to clear the cache.

Bear in mind, the code above is literally just a convenient way of clearing out the fragment cache so new photos show up on your photo page, it does not delete photos, nor does it perform a refresh automatically, although, you could add it to a CRON job.

When I get chance, I intend to automate this process and build it into rFlickr along with a new, improved, caching mechanism. I’m sure the above will tide you over for now though.

Check back soon.

Posted
12 Mar 2009 @ 21:22

Categories
, ,

Comments
None

Author
Alex

New rFlickr Ruby Gem

rFlickr

After I started to use the rFlickr gem it didn’t take me long to realize that development of the gem had all but halted, yes it worked, which was more than the original Flickr gem did, but it was still a little bit out of date and in the end, a little bit broken.

In one of my older posts I documented a fix for the gem and provided a download to unzip into your plugins folder, however, with the advent of the wonderful GitHub and it’s marvelous gem support I’ve decided to move the project onto GitHub.

I have preserved the original gem’s GPL license and copied the source code from it’s original repository on RubyForge to a new, public, GitHub repository. In the process of the move I have dropped old code from the project, updated the readme & license information and generally performed a little house-keeping.

You can find the project at: http://github.com/digitalpardoe/rflickr/. You can install the gem using one of the following methods. First involves adding GitHub as a gem source (always a good idea) and installing the gem:

$ gem sources -a http://gems.github.com
$ sudo gem install digitalpardoe-rflickr

The second method it to add the gem as a gem dependency to the environment.rb of your Rails project:

config.gem 'digitalpardoe-rflickr', :lib => 'flickr', :source => 'http://gems.github.com'

And run a rake task to install the gem:

$ sudo rake gems:install

Whilst performing the code migration I also added the fix that was documented in my original post and implemented support for the (not so) new ‘farm’ based Flickr URLs for images (which should make things easier to implement).

The future plans for rFlickr include new tests, improved usage examples, updated readme / documentation and implementation of missing API methods, time permitting of course.

Until the readme is updated please refer to the original post for information on how to use rFlickr.

That’s all for now, enjoy the new gem and as they say, if you don’t like it, fork it.

Posted
08 Mar 2009 @ 17:19

Categories
, ,

Comments
None

Author
Alex

Guess Who's Back

As you may have noticed, it’s been a long time since my last post. There isn’t really any good reason for this. Plenty has happened, I just haven’t got round to writing any of it down.

First off I’d like to mention the website, it went through a fairly radical redesign a few months ago and I mentioned nothing about it. For some reason it’s not in my nature to be happy with what I make hence the many faces and iterations of the website. This website, whilst being my home on the internet, is also the test bed for my RoR programming, you may get tired of hearing about it’s re-designs and re-codes but that’s part of the reason I created it. Anyway, another re-design is coming, this time it’s not visual but all back end, the main difference you will notice is that I am doing away with user accounts and having a more open comment system (I could be shooting myself in the foot with this decision, we’ll have to see how the spam bots take it). To the people that have commented on the blog already, your comments will be preserved and, when I roll out the changes, I intend to reply to all the comments I haven’t yet replied to.

The second thing I wanted to mention, again website related, is my hosting. A good proportion of my posts seem to be apologizing for the downtime of the website. I was actually getting pretty bored of this so decided to, quite literally, take matters into my own hands. The website is now hosted on a virtual private server set up and maintained by me. This again, may be a case where I’ve shot myself in the foot. For those of you interested, the VPS is provided by the wonderful folks at Bytemark Hosting.

Number three. Many of the posts of my website relate to the use of the ‘rflickr’ RubyGem. Development of this gem seems to have been at a stand still for a good while now, I’ve therefore taken the decision to clone it and try to continue development in my spare time. More on this in a later post.

Four. Any of you interested in my photography will have noticed a lack of it over the past few months, it’s not that I haven’t been taking any photographs, it’s just that I’ve not published any. To try and remedy this I uploaded a batch of photos today that have been sitting on my computer for a while. You can take a look at them on the photo page of the website or on my Flickr page.

Posted
30 May 2008 @ 23:03

Categories
,

Comments
8 Comments

Author
Alex

Caching Your Photographs

rFlickr

If you have at some point followed my now fairly ancient rFlickr tutorial you may have noticed that your photo page loads quite slowly, and that my photo page loads fairly quickly. To get my page to load as quickly as it does required a small custom caching method and a willingness on my part to sacrifice some bandwidth. Here’s how I did it.

This tutorial assumes that you have already worked through the previously mentioned rFlickr tutorial and have something similar to it set up. It also assumes that you have some knowledge of Rails, not that my knowledge was particularly wide ranging at the point I wrote this caching method.

First of all, you will need a table in your database to store the information about your photographs, I suggest the structure illustrated in the migration below;

class CreatePhotos < ActiveRecord::Migration
    def self.up
        create_table :photos, :id => false do |t|
            t.column "flickr_id", :string, :limit => 25, :null => false
            t.column "title", :string, :limit => 250
            t.column "description", :text
            t.column "url", :string, :limit => 250
        end
    
        add_index :photos, :flickr_id
    end
  
    def self.down
        drop_table :photos
    end
end

Once you have created this table you will need to create some folders to store the cached images, I created the following folders and will be using them throughout this tutorial;

#{RAILS_ROOT}/public/images/flickr_cache/small/
#{RAILS_ROOT}/public/images/flickr_cache/large/

Then generate the model for this photos table;

$ cd /your/rails/application
$ ./script/generate model Photo

Your view from the first tutorial can remain almost the same (details at the end of the post), however, to see the greatest speed improvement I suggest caching it, i.e;

<% cache do %>
... Your view code here. ...
<% end %>

Then modify your view method in your photography controller to read something like;

def view
    unless read_fragment({})
        check_cache
        @photos = Photo.find(:all)
    end
end

The above code will make sure that a cached photography page doesn’t already exist, if it doesn’t, then and only then will it check that the photograph cache is up to date and query the database.

We have not yet created a ‘check_cache’ method, this method is the core method to make the photography page load much, much faster, even when the photography page’s cache does not exist. The method should be placed as the last method in your photography controller, the code is as follows;

private
def check_cache
    if ENV['RAILS_ENV'] == 'production'
        flickr = Flickr.new(RAILS_ROOT + "/config/flickr.cache", FLICKR_API_KEY, FLICKR_SHARED_SECRET)
        @photos = flickr.people.getPublicPhotos(flickr.people.findByUsername(FLICKR_USERNAME))
      
        @db_photos = Array.new
        Photo.find(:all).each { |p| @db_photos.push(p.flickr_id) }

        for photo in @photos.reverse
            if !@db_photos.include?(photo.id)
       
                db_photo = Photo.new
                db_photo.flickr_id = photo.id.to_i
                db_photo.title = photo.flickr.photos.getInfo(photo.id).title
                db_photo.description = photo.flickr.photos.getInfo(photo.id).description
                db_photo.url = photo.flickr.photos.getInfo(photo.id).urls.values[0]
          
                db_photo.save
          
                open(File.expand_path("#{RAILS_ROOT}/public/images/flickr_cache/small/" + photo.id + ".jpg"),"w").write(open(photo.url('s')).read)
                open(File.expand_path("#{RAILS_ROOT}/public/images/flickr_cache/large/" + photo.id + ".jpg"),"w").write(open(photo.url).read)
            end
        end
    end
end

The following code will only run if you are in production mode (and probably test mode too). It will then load the necessary information from Flickr using the methods outlined in the rFlickr tutorial post. The method then iterates through the collection of photos from Flickr in reverse, so they appear in the same order in the database as the order they appear on Flickr.

It will then check if the photo already exists in the database, if it does not it will store a copy of the photograph’s information in the database and download previews of the images from Flickr to your server, previews of images can then be loaded from your server rather than Flickr’s slow servers .

To take advantage of the cache you will also need to modify your view to access the thumbnails from your newly created local repository rather than from Flickr’s servers, i.e;

<% for photo in @photos.reverse %>
    <%= image_tag("/images/flickr_cache/small/" + photo.flickr_id + ".jpg", :alt => photo.title) %>
<% end %>

Hope this goes some way to helping you improve the speed of your website.

Check back soon.

Update: As I have just been reminded in the comments, I forgot a piece of code to make this tutorial work correctly. You should put the line;

require 'open-uri'

Either in the bottom of you ‘environment.rb’ file or just under the ‘class’ line in the controller that is responsible for your photography page.

Update (3rd May 09): The code in the ‘check_cache’ method has been updated slightly, it now makes less round trips to the database, should speed things up if your DB server is in a different location to your application server. Also, any redirection problems you may have faced before should be solved by the new rFlickr ruby gem that has support for ‘farm’ URLs built in.

Posted
03 May 2008 @ 13:06

Categories
, , , ,

Comments
None

Author
Alex

About Time Too

It’s been well over a month since I managed to write a post now so I thought it was about time I started the cogs whirring again, especially as all my assignments are now finished and handed in.

Even though there has been a lack of posts in the recent month there hasn’t been a lack of feverish activity. First on the books was the release of Set Icon 0.2, this release just bought basic bug fixes and a much improved authentication system. This was followed in recent weeks by the release of Set Icon 0.3, in this release I fixed yet more bugs and introduce an option to remove a custom icon from a hard drive. You can, as always, download Set Icon from it’s download page.

On a more website orientated note, I have moved the website to new servers. Prime Hosting weren’t terribly Ruby on Rails centric in the end, hence the continual website downtime. The website is now hosted with media72. They have proven themselves over the last month to be far more reliable hosts than Prime, even though I am hosted on their beta testing server.

Along with the change of host I have modified the way in which the website is run, updates to the website are now performed via Capistrano, I hope to write a little more on this system in a later post to explain how it can help improve your Ruby on Rails development.

Don’t expect too many posts or software updates over the next two weeks, unfortunately I have a series of exams that require some serious revision, however, if I get really bored of revising you might see some posts / software appearing.

I’ve also recently acquired a new lens for my camera, a Tokina 12-24mm f/4. As part of my endeavor to expand what I write about I will hopefully (assuming I remember) write a small review about the lens as it took me a long time to find any concrete opinions on it before I purchased it.

That’s all I can think of to write for now, no doubt I will think of something else eventually.

Check back soon.

Posted
19 Feb 2008 @ 21:13

Categories
,

Comments
None

Author
Alex

rFlickr And Rails 2.0

rFlickr

Any of you that followed my tutorial on setting up rFlickr and have subsequently upgraded to Ruby on Rails 2.0 may have noticed some server errors on your photo pages, this is due to the fact that a single line of code in rFlickr does not work correctly with Rails 2.0.

You can download the hopefully fixed rFlickr from digitalpardoe.co.uk/downloads/rflickr/rflickr.zip. Simply unzip this file into your rails applications ‘vendor/plugins’ directory and restart the application, everything should start working again as expected.

For those of you that are interested line 644 of ‘lib/base.rb’ had to be modified from:

def from_xml(xml,photo=nil)

to:

def self.from_xml(xml,photo=nil)

Hope this gets you back on your feet, so to speak. Check back soon.

Update: You may also want to install the updated version of ‘actionwebservice’ because it is no longer bundled with the Rails distribution. Other people have reported rFlickr not working because of a lack of this gem, however, the lack of this gem didn’t affect my setup at all.

Posted
19 Feb 2008 @ 20:58

Categories
,

Comments
None

Author
Alex

Ruby On Rails 2.0

Rails 2.0 has been out for a couple of months now, I had refrained from upgrading to it because I wasn’t sure what it would break. After putting the website under version control I decided it was too easy not to upgrade to Rails 2.0, so here it is, a website that looks and runs exactly like it did and only a few problems to be found.

To go along with the upgrade I have also added a couple of new features to the website, the photography page now links to my Flickr photostream so if you want to post comments on my photos, feel free. I have also added the ability for me to add images from other sources to the blog, not that important but no doubt you’ll notice the change. The largest change at the moment is the addition of social networking sites on the sidebar. If you like what you read to can add the page to Delicious, Digg, reddit or StumbleUpon.

Mainly to test the upgrade, you can admire one of my flickr photos, the most popular of my photos, or check back soon.

Update: Oops, no image. Remember, if you update something to allow longer links, update the database too.

Update: Generally a broken site is a sign of bad testing, that’s true in this case, the archives were broken, now they’re fixed.

Posted
28 Jul 2007 @ 21:51

Categories
,

Comments
4 Comments

Author
Alex

Using rFlickr

rFlickr

I said it was coming and here it is, my little tutorial on how to use the rFlickr Ruby on Rails gem to create a photograph section like the one on my own website. The first thing to note is that pretty much all of the options available in the Flickr API (here) are available for use in rFlickr due to the fact it is all based around XML. There is a laborious process of configuration to go through, however, to make everything work, but once this is done you should have no problems.

Firstly install the rFlickr gem, I should at this juncture note the fact I am primarily a UNIX user so will aim these instructions at other UNIX users, mainly because I don’t know the specifics for Rails installations on Windows. So lets dive in (‘$’ denotes the terminal prompt and ‘\’ denotes line continues below):

$ sudo gem install rflickr --include-dependencies

The second thing you will need to do is make sure you have a Flickr account with some photos on it then pay a visit to http://www.flickr.com/services/api/keys/ and sign yourself up for an API key, once you have generated the key make a note of the key itself and the ‘secret’ that you are given, you will be needing these quite a bit.

The next thing to do is to basically follow the tutorial here, albeit with a few modifications, I have re-written the tutorial in full below.

$ cd /your/rails/application
$ ./script/console

To make differences clear the Rails console prompt will be shown as ‘>>’, don’t forget to replace the x’s with your information.

>> require 'flickr'
>> API_KEY = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
>> SHARED_SECRET = "xxxxxxxxxxxxxxxx"
>> flickr = Flickr.new("/tmp/flickr.cache", API_KEY, SHARED_SECRET)
>> flickr.auth.token
=> nil

The above just sets up your rFlickr object and makes sure that you don’t already have a key.

>> flickr.auth.getFrob

This returns a value that you will need to save somewhere.

>> flickr.auth.login_link
=> "http://some.link.flickr.com"

Click or copy the link you are given into a browser and authorize the API for usage, don’t worry, we’re almost there.

>> flickr.auth.getToken('that_frob_number_we_saved')
>> flickr.auth.cache_token
>> exit

Right, this is as far as the tutorial online goes, but there are some other useful steps we need to take to make everything more useable, mainly the moving of the token as the ‘/tmp’ directory may get cleared by our host.

$ cp /tmp/flickr.cache /your/rails/application/config/flickr.cache
$ rm /tmp/flickr.cache

Now we can get onto the actual programming and leave the authentication business behind.

We’re going to need a controller to use, for the purposes of this tutorial I will use a controller named ‘Photography’, it should save me some time as that’s what mine is called, the page to be rendered will be called ‘view’.

In the file ‘photography_controller.rb’ we will need the following information, rename as necessary to your application.

class PhotographyController < ApplicationController
    API_KEY = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
    SHARED_SECRET = "xxxxxxxxxxxxxxxx"

    def index
        view
        render :action => 'view'
    end
  
    def view
        flickr = Flickr.new(RAILS_ROOT + "/config/flickr.cache", \
            API_KEY, SHARED_SECRET)
        @photos = flickr.people.getPublicPhotos( \
            flickr.people.findByUsername("YOUR_FLICKR_NAME"))
    end
end

Then in the ‘views.rhtml’ that you will have created in your ‘views’ folder (or at least, are creating now) paste the following code.

<% for photo in @photos %>
    <a href="<%= photo.flickr.photos.getInfo(photo.id).urls.values %>" \
        ><img src="<%= photo.url('s') %>" /></a>
<% end %>

And that’s it, your basically done, all the thumbnails will link directly to your Flickr page, easy wasn’t it? The main problem occurs when you load your newly created page, it’s very, very slow, due to the speed of the Flickr API (I think). In order to improve the situation I would recommend using either page or fragment caching, but that will be covered in a future tutorial.

Hopefully this will have given you a few pointers in using the rFlickr gem, read through the Flickr API for more inspiration if you are feeling adventurous. Check back soon.

Update: If you are having problems with Rails 2.0, take a look at this fix.

Update: I have written a new tutorial on caching your photos page, that should speed it up a lot, assuming you are having problems.

Posted
19 Aug 2006 @ 22:15

Categories
,

Comments
None

Author
Alex

Textilize Alternative

The textilize methods in Rails has never really been what I wanted. When it wasn’t adding paragraphs it was messing with my line brakes but it made such a good job of making HTML and XHTML that validates. In order to make it work better I re-wrote the method to be more suitable.

Put the following in the ‘application_helper.rhtml’ file and call it using ‘custom_text(“String”)’.

def custom_text(text)
    if text.blank?
        ""
    else
        text = RedCloth.new(text).to_html
        if text[0..2] == "<p>" then text = text[3..-1] end
        if text[-4..-1] == "</p>" then text = text[0..-5] end
        text = text.gsub("<p>", "")
        text = text.gsub("</p>", "")
        return text
    end
end

Job done, you should get correct HTML and XHTML but no paragraphs and no messing with line brakes.