James Smith - Main

Mocking Kernel#require

2008-08-31T10:15:00Z

The other day, I remembered to add coverage analysis to my AMEE-Ruby tests, using rcov. The results were pretty good - most of the code was already tested, except a few failure cases, and I quickly wrote some new tests to make sure those were working properly. One little bit of code stood out though. Because AMEE talks XML and JSON, my gem can use JSON, but only if the JSON gem is available on the system (XML support is built into core Ruby). Problem is, require calls throw errors if they fail, so I had to write some code to handle the load failure and carry on regardless. This is the code inside amee.rb:

begin
  require 'json'
rescue LoadError
  nil
end

Problem with this is that the gem is installed on my system, so the require never fails, and the rescue is never used. Rcov notices, and I can't get to 100% coverage, which is annoying.

So, mocking to the rescue. We have to make Kernel#rescue throw an error if we try to require 'json', even if the gem is installed. At first, I tried to use flexmock to do this, but couldn't make it work - if anyone knows how, please tell me. My final approach was to monkeypatch Kernel#require inside my test so that require 'json' (and only json) would fail:

it "should cope if json gem isn't available" do
  # Monkeypatch Kernel#require to make sure that require 'json'
  # raises a LoadError
  module Kernel
    def require_with_mock(string)
      raise LoadError.new if string == 'json' 
      require_without_mock(string)
    end
    alias_method :require_without_mock, :require 
    alias_method :require, :require_with_mock
  end
  # Remove amee.rb from required file list so we can load it again
  $".delete_if{|x| x.include? 'amee.rb'}
  # Require file - require 'json' should throw a LoadError,
  # but we should cope with it OK.
  lambda {
    require 'amee'
  }.should_not raise_error
end

We have to make a new require function, then use alias_method to rename the old one and install our new one in it's place. Rails includes alias_method_chain to make this easier, but that's not available in pure Ruby. Never mind. Once we've done the monkeypatch, we take amee.rb out of the $" array, which lists all the currently-required files, to make sure we can require it again, then simply run the require and make sure it catches the error that is thrown by our patched require. And the most satisfying part? The rescue is executed, the test works, and we get to 100% coverage. I've got a nice warm fuzzy feeling inside... mmmmmm.

CurrentCost data live on Pachube

2008-08-29T09:56:00Z

So, the other day I got a nice little tray icon working for my CurrentCost power monitor. That's great, but data is only really gets fun when it's mashable, so the next step was to get it online somehow.

Pachube is a site which aggregates data feeds from real-world (and virtual-world) devices, shows them on a map, makes graphs, things like that, so it seemed like a good first attempt at putting my power data online. My first thought was to get my app to post data at regular intervals to the service, but unfortunately Pachube doesn't work like that. Instead, it acts more like a news reader, not a publishing platform - Google Reader instead of Blogger, if that makes sense. So, I had to publish my feed live on the web and point Pachube at the URL.

First step: EEML. This is an XML-based format which Pachube reads which can contain not only multiple data feeds, but tags and other metadata. So (as seems to be the fashion), I wrapped it up in a Ruby gem, available from GitHub as ever. The gem simply provides utility classes to build an EEML feed and convert it to the XML-based format for delivery over the web.

Then, the final step was to publish the data on the web. For that, we need a web server. However, having a full web server for just one feed seemed overkill, and I didn't want to publish to yet another intermediate server, so we need to serve the data directly. Ruby to the rescue once again. WEBrick is a simple web server which is part of the core Ruby libraries. You create a server, write simple servlet classes, and mount them at particular locations. For instance:

# Create WEBrick server
s = WEBrick::HTTPServer.new( :Port => 50000 )
# A simple "hello world" servlet 
class HelloServlet < WEBrick::HTTPServlet::AbstractServlet
  def do_GET(request, response)
    response.status = 200
    response['Content-Type'] = "text/xml"
    response.body = "hello world"
  end
end
s.mount("/", HelloServlet)

Now, http://localhost:50000/ will say "hello world". From here it's a simple modification to publish the EEML feed. EEML-Ruby includes a simple EEML server script as an example. So, after building this into the tray app, now whenever my CurrentCost is connected and the app is running, it serves up EEML data to the web. You can see the data feed (fairly intermittent, as obviously the meter isn't always connected to my PC at the moment) live on Pachube.

Some successful CurrentCost hacking

2008-08-22T09:25:00Z

After a bit of work, I've finally got my CurrentCost meter working in Ruby, and I now have a power monitor sitting in my system tray! There were a few stages involved...

Serial comms: The ruby-serialport library that already existed for Ruby was no good to me. Firstly, it didn't seem to be in a working state, but more importantly, the license it is under (GPL) is no good to me. So, I had to write my own. I've created a nice simple serial library (including a gem) for Ruby called RB232, which is available on GitHub. It only supports reading at the moment, and only works on Linux systems, but it's a start. Next!

Reading CurrentCost data: Once RB232 was in place, this was pretty easy. Just create a couple of classes to wrap up the process of getting data from the meter, and away you go. Easy. Also released as a Ruby gem on Github.

User interface: Last step was to make a simple user interface for the meter, which you can see in the picture above. It's a simple tray icon that changes colour based on power usage. It's based heavily on another very useful tool called cctrayrb, so many thanks to Daniel Parnell for doing the heavy GUI lifting there. The app is included as part of the currentcost-ruby gem mentioned above.

Anyway, it's all freely available, so if you have a CurrentCost meter and a serial cable for it, you can grab the code and get going. Enjoy :)

Twitter support in JabberStatus

2008-08-07T21:10:00Z

JabberStatus was originally inspired by Twitter's facility to update your status via XMPP/Jabber. Unfortunately, Twitter's Jabber interface has been down for months now, which is rather sucky.

The solution to this? Extend JabberStatus to work for Twitter as well as Facebook. It turned out to be very easy, so it's online now. Just add twitterstatus@jabber.org to your contact list and it will talk you through the rest.

The JabberStatus code has moved to GitHub now, so you can grab it from there. The code for the CO2Updates app has also moved there.

One Hundred Months

2008-08-06T09:56:00Z

I just saw the One Hundred Months campaign, and decided it was ripe for a bit of automated Twittering. So, 5 minutes hacking and we have One Hundred Months on Twitter. Code (as ever these days) is available from GitHub.

I'm amazed by what computers can do sometimes. This one seriously took me longer to publish to the world than to write.

Hacking your energy usage with the CurrentCost

2008-08-06T00:09:00Z

The other day, I managed to get hold of a CurrentCost energy monitor (available to buy from here, or maybe from your electricity supplier). Now, the nice thing about this particular monitor (apart from the ton of information on-screen) is the fact that it has a serial output on the bottom, which you can (with a bit of hacking) plug into your PC, and - bingo - lovely XML data!

However, once you have it connected and spewing XML at you, you really need something to do with all that data. I don't have anything big written yet, but my first step towards making something useful is a Ruby gem, which is available from GitHub. So far it can only parse the XML data from the meter - direct access to the serial port is hopefully coming soon.

In other news, the AMEE gem I started a while ago is still coming on. It can now use the XML and JSON interfaces, parse the whole Data API, and retrieve a list of Profiles. Not a bad start.

AMEE for Ruby

2008-07-10T21:29:00Z

In a fit of why-the-hell-not, I've just started writing my first gem for Ruby, which is going to be a wrapper around the AMEE carbon calculation engine. It's still in a very early incarnation, but more will be forthcoming soon. At the moment it can authenticate, and parse DataCategory nodes. DataItems and DataItemValues will be following after a short (UK-based) holiday. Source code and installation instructions are available from GitHub.

Filtering Twitter with Pipes

2008-07-10T17:29:00Z

I like Twitter. I'm not sure why, but I do. However, I don't like the fact that I can't view my friends timeline via RSS without seeing my own updates in there as well. I already know what I've written, I just want to see what other people wrote. To scratch that particular itch, over lunchtime I built a Yahoo! Pipe to filter my own updates out of my feed, and you can use it too. Check it out here.

Announcing JabberStatus

2008-05-30T20:07:00Z

Twitter, while it has its problems, does one thing very well. The IM interface allows you to update your status via a Jabber account, which just makes it so easy to do. Facebook, on the other hand, has no such facility, which is unfortunate because although I like the Facebook service, I hate using the website itself. Yes, OK, you can update your Facebook status via Twitter, but I write different things on Facebook and Twitter, so I didn't want to go down that route. So, my solution? Write my own!

JabberStatus is a simple Ruby script which listens on a Jabber account and updates your Facebook status for you. You add facebook@jabber.org to your Jabber contact list, then it says hello and gives you instructions on what to do. You click a couple of URLs to give it access to your Facebook account, and that's it! Then you just send it status updates, and it posts them on Facebook for you. It's free for anyone to use, so give it a go!

Technically, it's pretty cunning though I say so myself. It stores Facebook session data for each person that talks to it in it's own Jabber contact list, so no backend database is required. Anyway, it's all open source so you can take a look yourself. You can even run your own instance of it. It's still under development (needs code tidying up, as well as better error handling), but it's out there working now, so have a go.

geeKyoto 08

2008-05-19T10:05:00Z

Well, Saturday was geeKyoto 08 in London, and I finally had to get round to writing my presentation. Had to finish it on the train on the way in, hoping I wouldn't run the laptop battery down too much in case I had to use it during the talk.

It was a great day though! The speakers were incredibly varied, from Alex Haw, an architect/artist who has more ideas in a second than I do in a year, to a great presentation by Gavin Starks about AMEE, to a design student who hung a swing in a bus stop (which got the day's most spontaneous round of applause). The whole thing was then rounded off by Ben Saunders, an arctic explorer who has really seen the ice melting up there. Add onto that a fascinating post-conference pub session, and you can't lose. The day left me so energised and full of ideas. Now I just need to decide which to attack first!

Hopefully my talk ("Can software save the world?" - pretentious title intentional) will be up online soon. For those of you who saw my talk but didn't get the links down, here they are again:

Adventures with GitHub

2008-05-09T15:31:00Z

When the Linux kernel moved to using git for version control, I didn't really pay much attention, to be honest. When Rails moved, I only really paid attention because it made my life harder, as Piston doesn't support git fully yet. However, I've now had a reason to use it myself, and I have to say I'm actually rather more impressed than I thought I'd be.

While moving this site over to Mephisto, I had a problem with the asset system, in that while assets have a "title" field in the database, it's not editable in the admin interface, or accessible in liquid templates. The fix was simple, just a couple of lines (once I found where to put them), but patching my local copy of Mephisto was somehow unsatisfying. Well, Mephisto is hosted on GitHub these days, so it was time to dive in!

Normally, the process of sending a patch to an open source project is just too laborious to bother with for small stuff. Check out the code, editing it, creating a patchfile, subscribing to the mailing list, introducing yourself, explaining the patch, and so on. However, following the Rails on the Run GitHub tutorial, I was done in 5 minutes, even accounting for learning a new version control tool!

The way it works is just great - press the button to get your own fork, check it out, change the code, commit, and then press another button to send a "pull request" back to the main trunk. It really does make quite a difference - and if they don't like my change, who cares? I now have my own version-controlled branch of Mephisto which I can use to manage my local copy, and make any other changes I feel like.

Thanks, git (and specifically GitHub's web toolset), for solving problems I never even really knew I had.

A whole new look

2008-05-09T10:55:00Z

Well, it was about time for an overhaul... my old site was looking increasingly outdated, and it didn't really fulfill what I wanted from it any more, so I've moved over to a new one, using Mephisto. It's deployed on my own virtual server, and is running under Phusion Passenger, the new mod_rails solution for Apache. It's definitely been the easiest Rails installation I've ever done! Not sure how well it would stand up to heavy use, but this site certainly doesn't need to be high-capacity, so it does the job admirably.

The hardest thing about getting the new site up and running was deciphering Mephisto's liquid code. While there is *some* documentation, most of it seems to be out of date and scattered across the net, so it was a bit awkward. I'm sure you can expect a couple of blog posts detailing exactly what I had to do to get the site into a good shape at some point...

CO2 on Twitter

2008-05-07T13:08:00Z

I read this article last night, about using Twitter to make machines talk, specifically Tower Bridge. "What a fantastic idea", I thought to myself. "Maybe I can do the same thing for climate change". Also, it would be a good way to flex my Ruby muscles a little and get some "fun" coding in for the first time since Amelia was born.

So, a couple of hours of Ruby later, and we have atmospheric CO2 updates on Twitter. The code is dead simple, and can be grabbed from GitHub.

Sanitizing :limit and :offset

2008-02-01T09:54:00Z

I have a need in the Green Thing Rails app to pass in limit and offset parameters to a find query, and these parameters come from the URL. For example:

@examples = Example.find(:all, :limit => params[:limit], :offset => params[:offset])

This works fine, but is vulnerable to dodgy things making their way in through the parameters. Normally, to defend against SQL injection, you would use the array form of :conditions to sanitize the parameters, like so:

@examples = Example.find(:all, :conditions => ["name LIKE ?", params[:name])

This is great for :conditions. However, you can't use it with :limit and :offset, despite the fact that they are just as vulnerable to SQL injection. The trick I came up with is this. :limit and :offset are both integers, so simply force the conversion to an integer before using them (checking for nil first, of course):

@examples = Example.find(:all, :limit => (params[:limit].nil? ? nil : params[:limit].to_i),
                               :offset => (params[:offset].nil? ? nil : params[:offset].to_i))

This means that no nasty SQL can get in, because it won't get through the integer conversion. Simple and effective. Probably blindingly obvious, but I didn't find anything online to point me to this solution while I was looking around for :conditions-style sanitizing, so I thought I'd share.

Of course, if I've missed something glaringly obvious, please let me know ;)

Testing in Trac

2008-01-09T01:39:00Z

Hurrah, I've just contributed my first code back to the Trac project. I've created a workflow for the 0.11 release which adds a simple "testing" state to tickets before they are closed. For the Trac addicts amongst you, it's available at Trac Hacks. I'm rather proud of it, because it's actually the first workflow hack on the site!