James Smith

Well, I finally got round to it. I started developing The Carbon Diet 2 years ago (had the idea 3 years ago!), and it's finally spread it's wings and joined the world of open source.

After I'd been developing it for a few months, it became obvious that it was never going to be a commercial proposition, so it's been my intention to open it up for a very long time now. However, I was putting it off until "after I get it finished", or "after I tidy up the code", or any number of other reasons. Having a small child puts paid to all those plans, so I've finally bitten the bullet and released my dodgy code to the world as it is, warts and all.

The site is built in Rails, but it's pretty creaky. It was originally written in version 1.2 as I was learning, and a lot of the old-style controller and route code is still hanging around, waiting to be improved. I've started, but there is a long way to go to bring it up to a decent standard.

Anyway, if you use the site (or not!) and fancy improving it, now you can. Head on over to github to grab the code and join in. I hope to see you there!

A while ago, I started hacking around with a CurrentCost real-time energy monitor. This is a very nice little device that measures your electricity use, but more importantly has a serial output on the bottom so you can get data out of it.

Well, eventually I decided that while being able to get data off the meter was nice, it would be better if I could publish it somewhere. So, I wrote a program in Ruby to do exactly that. It reads data from the meter, and sticks it on the web. I'll go into a bit more detail in a minute, but first, you can download it (and get the source code) from github.

The program operates as a system daemon, which runs in the background on your Linux box (Mac should work as well) and uploads to various places. Adding new publishers is dead easy, so if you want to publish your data somewhere else, you can easily add it.

The Carbon Diet

First up, my own site, The Carbon Diet. This uploads your daily usage history from the meter into your carbon diet account so that you don't have to take so many meter readings to get an accurate graph. This is still pretty experimental and needs more work, but it's pretty useful already. You can see how it looks on my profile.

AMEE

Next, what I think of as the important one. AMEE, if you're not aware, AMEE is a neutral aggregation platform for sharing energy data and carbon calculations (disclaimer: I work for them these days). Think OpenID for your energy identity. Anyway, we have a nice "smart meter" demo which uses my currentcost app as the data source. Every minute, it uploads into an AMEE profile, and then another app makes a nice graph of the CO2 produced. In theory, the Carbon Diet could pull that data from AMEE instead of me publishing it directly, but that's still to come.

Pachube

Another energy data sharing service is Pachube (pronounced "patch bay"). As far as I can tell, this is more geared at art and design than rigorous data, but it's fun to play with. They've done a bunch of stuff with the CurrentCost, and now my app joins the throng. My pachube feed updates every 6 seconds - every time the meter sends data out.

Twitter

Finally, what would be the point of a web energy publisher if it couldn't tweet? If you really want to, you can see my minute-by-minute energy usage on Twitter by visiting @james_energy.

My server runs Ubuntu, and seeing as 8.10 (Intrepid Ibex) is out, I thought "woo, let's upgrade". Bad idea. Well, partly. Most of the upgrade went fine, but there was one problem. Intrepid comes with Ruby 1,8.7, which works fine with Rails >=2.1, but doesn't work with anything lower. Mephisto, on the other hand, doesn't work with Rails 2.1 yet, and is only usable with Rails <=2.0.2. So, you get an error starting Mephisto that looks like this:

So, I was faced with having to downgrade to Ruby 1.8.6 (a royal pain as Intrepid doesn't provide packages for it), or "fix" Mephisto to work with 2.1 (oh my god, don't even get me started). Fortunately, another solution presented itself.

I came across this blog post which describes the problem I was having. The gist (it's in Spanish) is that the main problem with Rails 2.0.2 and Ruby 1.8.7 is that Ruby 1.8.7 has added a String#chars function to the core language. Rails also adds this function, but the two have different return types, hence causing much breakage. The solution? Remove Ruby's String#chars and let Rails do its thing. Add this code to an initializer in your Rails 2.0.2 app, and it will work (better, at least) with Ruby 1.8.7.

begin
  String.class_eval { remove_method :chars }
rescue NameError
end

As you can tell by the fact that you're reading this blog, it works a treat. Hurrah!

EDIT: Another solution, which I only just worked out, would be to install the very badly-named Ruby Enterprise Edition, which sits alongside the system's Ruby and is call-compatible with 1.8.6. This makes a lot of sense if you're using passenger (mod_rails), as well.

I've been crazy busy for a while, but what little spare time I have grabbed here and there has been used investigating some areas of Rails I hadn't played with before, like route globbing and running without a database. The results are a couple of tools, one useful, one most definitely not.

The first (and most useful) is Has My Gem Built Yet?. I have a bunch of ruby gems on GitHub and get annoyed that I have to keep running "gem update" while waiting for GitHub to build the gem once I've pushed the code. Well, no more. Just put in the details, and the page checks for you and pops up a message when the gem is done. Nice. It got picked up on the GitHub blog, and I've even had some patches sent in. It's a nice feeling :)

The second app is somewhat less useful (though still entertaining). After a suggestion on Twitter by Tom Taylor, I built a website which lets you bookmark sandwich combinations. There are a couple of interesting bits in this though - first is route globbing, which is the Rails way of using arbitrary bits of the URL as parameters to an action. Not something you use in every project, but still. More importantly, I've used it to test out rubymarks, a Rails plugin for generating links for social bookmarking services.

I love Rails for this sort of thing. I can't think of other system I've used that would let me create such fun things in the tiny snippets of spare time I have at the moment. And of course, everything above is open source and available on Github.

The other day, I remembered to add coverage analysis to my AMEE-Ruby tests, using rcov. The results were pretty good - most of the code was already tested, except a few failure cases, and I quickly wrote some new tests to make sure those were working properly. One little bit of code stood out though. Because AMEE talks XML and JSON, my gem can use JSON, but only if the JSON gem is available on the system (XML support is built into core Ruby). Problem is, require calls throw errors if they fail, so I had to write some code to handle the load failure and carry on regardless. This is the code inside amee.rb:

begin
  require 'json'
rescue LoadError
  nil
end

Problem with this is that the gem is installed on my system, so the require never fails, and the rescue is never used. Rcov notices, and I can't get to 100% coverage, which is annoying.

So, mocking to the rescue. We have to make Kernel#rescue throw an error if we try to require 'json', even if the gem is installed. At first, I tried to use flexmock to do this, but couldn't make it work - if anyone knows how, please tell me. My final approach was to monkeypatch Kernel#require inside my test so that require 'json' (and only json) would fail:

it "should cope if json gem isn't available" do
  # Monkeypatch Kernel#require to make sure that require 'json'
  # raises a LoadError
  module Kernel
    def require_with_mock(string)
      raise LoadError.new if string == 'json' 
      require_without_mock(string)
    end
    alias_method :require_without_mock, :require 
    alias_method :require, :require_with_mock
  end
  # Remove amee.rb from required file list so we can load it again
  $".delete_if{|x| x.include? 'amee.rb'}
  # Require file - require 'json' should throw a LoadError,
  # but we should cope with it OK.
  lambda {
    require 'amee'
  }.should_not raise_error
end

We have to make a new require function, then use alias_method to rename the old one and install our new one in it's place. Rails includes alias_method_chain to make this easier, but that's not available in pure Ruby. Never mind. Once we've done the monkeypatch, we take amee.rb out of the $" array, which lists all the currently-required files, to make sure we can require it again, then simply run the require and make sure it catches the error that is thrown by our patched require. And the most satisfying part? The rescue is executed, the test works, and we get to 100% coverage. I've got a nice warm fuzzy feeling inside... mmmmmm.

So, the other day I got a nice little tray icon working for my CurrentCost power monitor. That's great, but data is only really gets fun when it's mashable, so the next step was to get it online somehow.

Pachube is a site which aggregates data feeds from real-world (and virtual-world) devices, shows them on a map, makes graphs, things like that, so it seemed like a good first attempt at putting my power data online. My first thought was to get my app to post data at regular intervals to the service, but unfortunately Pachube doesn't work like that. Instead, it acts more like a news reader, not a publishing platform - Google Reader instead of Blogger, if that makes sense. So, I had to publish my feed live on the web and point Pachube at the URL.

First step: EEML. This is an XML-based format which Pachube reads which can contain not only multiple data feeds, but tags and other metadata. So (as seems to be the fashion), I wrapped it up in a Ruby gem, available from GitHub as ever. The gem simply provides utility classes to build an EEML feed and convert it to the XML-based format for delivery over the web.

Then, the final step was to publish the data on the web. For that, we need a web server. However, having a full web server for just one feed seemed overkill, and I didn't want to publish to yet another intermediate server, so we need to serve the data directly. Ruby to the rescue once again. WEBrick is a simple web server which is part of the core Ruby libraries. You create a server, write simple servlet classes, and mount them at particular locations. For instance:

# Create WEBrick server
s = WEBrick::HTTPServer.new( :Port => 50000 )
# A simple "hello world" servlet 
class HelloServlet < WEBrick::HTTPServlet::AbstractServlet
  def do_GET(request, response)
    response.status = 200
    response['Content-Type'] = "text/xml"
    response.body = "hello world"
  end
end
s.mount("/", HelloServlet)

Now, http://localhost:50000/ will say "hello world". From here it's a simple modification to publish the EEML feed. EEML-Ruby includes a simple EEML server script as an example. So, after building this into the tray app, now whenever my CurrentCost is connected and the app is running, it serves up EEML data to the web. You can see the data feed (fairly intermittent, as obviously the meter isn't always connected to my PC at the moment) live on Pachube.

After a bit of work, I've finally got my CurrentCost meter working in Ruby, and I now have a power monitor sitting in my system tray! There were a few stages involved...

Serial comms: The ruby-serialport library that already existed for Ruby was no good to me. Firstly, it didn't seem to be in a working state, but more importantly, the license it is under (GPL) is no good to me. So, I had to write my own. I've created a nice simple serial library (including a gem) for Ruby called RB232, which is available on GitHub. It only supports reading at the moment, and only works on Linux systems, but it's a start. Next!

Reading CurrentCost data: Once RB232 was in place, this was pretty easy. Just create a couple of classes to wrap up the process of getting data from the meter, and away you go. Easy. Also released as a Ruby gem on Github.

User interface: Last step was to make a simple user interface for the meter, which you can see in the picture above. It's a simple tray icon that changes colour based on power usage. It's based heavily on another very useful tool called cctrayrb, so many thanks to Daniel Parnell for doing the heavy GUI lifting there. The app is included as part of the currentcost-ruby gem mentioned above.

Anyway, it's all freely available, so if you have a CurrentCost meter and a serial cable for it, you can grab the code and get going. Enjoy :)

JabberStatus was originally inspired by Twitter's facility to update your status via XMPP/Jabber. Unfortunately, Twitter's Jabber interface has been down for months now, which is rather sucky.

The solution to this? Extend JabberStatus to work for Twitter as well as Facebook. It turned out to be very easy, so it's online now. Just add twitterstatus@jabber.org to your contact list and it will talk you through the rest.

The JabberStatus code has moved to GitHub now, so you can grab it from there. The code for the CO2Updates app has also moved there.

I just saw the One Hundred Months campaign, and decided it was ripe for a bit of automated Twittering. So, 5 minutes hacking and we have One Hundred Months on Twitter. Code (as ever these days) is available from GitHub.

I'm amazed by what computers can do sometimes. This one seriously took me longer to publish to the world than to write.

The other day, I managed to get hold of a CurrentCost energy monitor (available to buy from here, or maybe from your electricity supplier). Now, the nice thing about this particular monitor (apart from the ton of information on-screen) is the fact that it has a serial output on the bottom, which you can (with a bit of hacking) plug into your PC, and - bingo - lovely XML data!

However, once you have it connected and spewing XML at you, you really need something to do with all that data. I don't have anything big written yet, but my first step towards making something useful is a Ruby gem, which is available from GitHub. So far it can only parse the XML data from the meter - direct access to the serial port is hopefully coming soon.

In other news, the AMEE gem I started a while ago is still coming on. It can now use the XML and JSON interfaces, parse the whole Data API, and retrieve a list of Profiles. Not a bad start.

In a fit of why-the-hell-not, I've just started writing my first gem for Ruby, which is going to be a wrapper around the AMEE carbon calculation engine. It's still in a very early incarnation, but more will be forthcoming soon. At the moment it can authenticate, and parse DataCategory nodes. DataItems and DataItemValues will be following after a short (UK-based) holiday. Source code and installation instructions are available from GitHub.

I like Twitter. I'm not sure why, but I do. However, I don't like the fact that I can't view my friends timeline via RSS without seeing my own updates in there as well. I already know what I've written, I just want to see what other people wrote. To scratch that particular itch, over lunchtime I built a Yahoo! Pipe to filter my own updates out of my feed, and you can use it too. Check it out here.

Twitter, while it has its problems, does one thing very well. The IM interface allows you to update your status via a Jabber account, which just makes it so easy to do. Facebook, on the other hand, has no such facility, which is unfortunate because although I like the Facebook service, I hate using the website itself. Yes, OK, you can update your Facebook status via Twitter, but I write different things on Facebook and Twitter, so I didn't want to go down that route. So, my solution? Write my own!

JabberStatus is a simple Ruby script which listens on a Jabber account and updates your Facebook status for you. You add facebook@jabber.org to your Jabber contact list, then it says hello and gives you instructions on what to do. You click a couple of URLs to give it access to your Facebook account, and that's it! Then you just send it status updates, and it posts them on Facebook for you. It's free for anyone to use, so give it a go!

Technically, it's pretty cunning though I say so myself. It stores Facebook session data for each person that talks to it in it's own Jabber contact list, so no backend database is required. Anyway, it's all open source so you can take a look yourself. You can even run your own instance of it. It's still under development (needs code tidying up, as well as better error handling), but it's out there working now, so have a go.

Well, Saturday was geeKyoto 08 in London, and I finally had to get round to writing my presentation. Had to finish it on the train on the way in, hoping I wouldn't run the laptop battery down too much in case I had to use it during the talk.

It was a great day though! The speakers were incredibly varied, from Alex Haw, an architect/artist who has more ideas in a second than I do in a year, to a great presentation by Gavin Starks about AMEE, to a design student who hung a swing in a bus stop (which got the day's most spontaneous round of applause). The whole thing was then rounded off by Ben Saunders, an arctic explorer who has really seen the ice melting up there. Add onto that a fascinating post-conference pub session, and you can't lose. The day left me so energised and full of ideas. Now I just need to decide which to attack first!

Hopefully my talk ("Can software save the world?" - pretentious title intentional) will be up online soon. For those of you who saw my talk but didn't get the links down, here they are again:

• The Carbon Diet
• Green Thing
• CO2 Updates on Twitter
• Green Web UK mailing list

When the Linux kernel moved to using git for version control, I didn't really pay much attention, to be honest. When Rails moved, I only really paid attention because it made my life harder, as Piston doesn't support git fully yet. However, I've now had a reason to use it myself, and I have to say I'm actually rather more impressed than I thought I'd be.