Rails Metal: a micro-framework with the power of Rails: \m/

Updates:

  • Clarified the distinction between Rails Metal and Rack middleware after more information from @Josh in the comments. Thanks!
  • Read more about metal from DHH on the Official Rails Blog.
  • Demonstrate Testing Metal end points
  • Update Poller example to match new style
  • Cover Sinatra Integration
  • Correct benchmarks

Josh Peek committed a new feature to Edge Rails today: Rails Metal. After the recent work to replace Rails’ crufty request processing code with Rack and integrate its middleware support, Rails Metal is a logical progression that allows Rails apps to use the power of Rack middleware to create super-fast actions.

For example, here’s a sample “Hello World” Metal:

  class Poller < Rails::Rack::Metal
    def call(env)
      if env["PATH_INFO"] =~ /^\/poller/
        [200, {"Content-Type" => "text/html"}, "Hello, World!"]
      else
        [404, {"Content-Type" => "text/html"}, "Not Found"]
      end
    end
  end

And for comparison, a “Hello World” controller:

    class OldPollerController < ApplicationController
      def poller
        render :text => "Hello World!"
      end
    end

So, let’s fire up ruby script/server and see what this gives us:

  # traditional Controller
  $ curl 127.0.0.1:3000/old_poller/poller
    Hello World!

  # the new Metal
  $ curl 127.0.0.1:3000/poller
    Hello World!

So, the point of all of these other “micro-frameworks” is that they’re faster than Rails, right? Let’s benchmark this new “Hello World” Metal:

  # first, let's benchmark the traditional controller
  $ ab -n 1000 http://127.0.0.1:3000/old_poller/poller
  ... snip ...
  Requests per second:    408.45 [#/sec] (mean)
  Time per request:       2.448 [ms] (mean)

  # now for the Metal middleware
  $ ab -n 1000 http://127.0.0.1:3000/poller
  ... snip ...
  Requests per second:    1154.66 [#/sec] (mean)
  Time per request:       0.866 [ms] (mean)

For this trivial “Hello World” benchmark, Rails Metal is 2.8x faster than a Controller. Awesome. Have a couple actions of your app you need to optimize? Instead of breaking them out into a separate application using a micro-framework, add a Metal inside your existing app. You get the performance benefits of processing requests outside of ActionPack, and it’s all integrated as a part of your Rails app. Easy!

Sinatra Metal

You can now also use Sinatra to create Metal end points:

  Sinatra::Application.default_options.merge!(:run => false, :env => 
  :production)
  Api = Sinatra.application unless defined? Api

  get '/interesting/new/ideas' do
    'Hello Sinatra!'
  end

First person to show the use of a Merb app as a Metal end point wins a prize.

Standalone Execution

Additionaly, Rails Metal are able to be executed in a separate process from your Rails application using rackup:

  rackup -s mongrel app/metal/poller.rb

This runs the Poller Metal separeately from Rails, on it’s own port (rackup defaults to 9292). This is perfect if you have an action that’s taking a very long time (for example a file upload) that you’d like to split out from the normal Rails request processing queue.

Testing Metal

Update: After several people commented asking how to test metal, DHH chimed in and recommend Integration Testing for Metal end points, as they hit the whole stack, and I submitted a patch cleaning up the Integration Testing behavior of Metal. Testing Metal end points now works just like any other Integration test:

      class PollerTest < ActionController::IntegrationTest
        test "poller returns hello world" do
          get "/poller"
          assert_response 200
          assert_response :success
          assert_response :ok
          assert_equal "Hello World!", response.body
        end
      end

Fun With Middleware

So, essentially, Rails Metal is a thin wrapper around Rails’ new Rack middleware support. Rack middleware is pretty powerful stuff: framework-independent components that process requests independently or in concert with other middleware. For example, here’s a simple piece of Rack middleware that runs a regex on responses:

class RegexMiddleware
  def initialize(app)
    @app = app
  end

  def call(env)
    status, headers, response = @app.call(env)
    new_response = []
    response.each do |part|
      new_response << part.gsub(/World/, 'Middleware')
    end
    [status, headers, new_response]
  end
end

To use this rack middleware in Rails, add this line to your environment.rb

Rails::Initializer.run do |config|
  ...
  config.middleware.use RegexMiddleware
end 

Restart your server, and check out what happens:

  $ curl 127.0.0.1:3000/poller
    Hello Middleware!

The Rack middleware filtered the output of the Metal we created before. This works with output generated by normal controllers and everything too. The possible uses of this pattern are endless:

  • Single Sign On
  • Request/Response Signing (think OAuth)
  • Asset Compression

rack-contrib is a nice collection of Rack middleware if you’re interested in more examples.

Rails Metal is a simple wrapper around the existing (yet undocumented) Rack middleware support in Edge Rails that attempts to DRY the process of using middleware to create endpoints (like a poller) as opposed to filters (which are better implemented as traditional middleware, like the examples above). For example, Rails Metal might be used:

  • To speed up a ‘poller’ action called by all active users of a popular web-based chat application every 3 seconds (hint: Campfire).
  • To improve the performance of any API endpoint
  • To process file uploads outside of the Rails request queue
  • To authorize delivery of cached content

We don’t need no stinking micro-frameworks

With the additional of Metal and Rack middleware support, Rails effectively includes a micro-framework of its own; one that either tighly integrates with Rails or is executed separately – whichever the need dictates.

This is a great response by the Rails team to all of the buzz surrouding micro-frameworks: a micro-framework with the power of Rails. I’m definitely going to try this approach to squeeze a couple extra requests per second out of a heavily trafficked API call – let me know in the comments if you find a use for it.

Read up a bit more on Rack and then take a look at Josh’s commit Introducing Rails Metal (and the ensuing comments) if you’re interested for more information.

resource_this - DRY Rails Resource Controllers

I’ve always been annoyed at the lack of maintainability that comes with using multiple resource controllers in my Rails apps. Each generated resource controller clocks in at 85 lines, and most of mine only differ from each other by a line or two – an added before_filter or a change in the url that the users is redirected to after the creation of a new Widget. Not very DRY. When coming back to each one of these controllers to add or adjust features, it takes me entirely too much time to sift through the stock 85 lines and find my application-specific behavior.

Enter resource_this

git clone git://github.com/jnewland/resource_this.git

resource_this aims to solve this maintainability problem by making your stock resource controllers look like this:

  class PostsController < ActionController::Base
   resource_this
  end

Behind the scenes, this code is generated:

  class PostsController < ActionController::Base
    before_filter :load_post, :only => [ :show, :edit, :update, :destroy ]
    before_filter :load_posts, :only => [ :index ]
    before_filter :new_post, :only => [ :new ]
    before_filter :create_post, :only => [ :create ]
    before_filter :update_post, :only => [ :update ]
    before_filter :destroy_post, :only => [ :destroy ]

  protected
    def load_post
      @post = Post.find(params[:id])
    end

    def new_post
      @post = Post.new
    end

    def create_post
      @post = Post.new(params[:post])
      @created = @post.save
    end

    def update_post
      @updated = @post.update_attributes(params[:post])
    end

    def destroy_post
      @post = @post.destroy
    end

    def load_posts
      @posts = Post.find(:all)
    end

  public
    def index
      respond_to do |format|
        format.html
        format.xml  { render :xml => @posts }
        format.js
      end
    end

    def show          
      respond_to do |format|
        format.html
        format.xml  { render :xml => @post }
        format.js
      end
    end

    def new          
      respond_to do |format|
        format.html { render :action => :edit }
        format.xml  { render :xml => @post }
        format.js
      end
    end

    def create
      respond_to do |format|
        if @created
          flash[:notice] = 'Post was successfully created.'
          format.html { redirect_to @post }
          format.xml  { render :xml => @post, :status => :created, :location => @post }
          format.js
        else
          format.html { render :action => :new }
          format.xml  { render :xml => @post.errors, :status => :unprocessable_entity }
          format.js
        end
      end
    end 

    def edit
      respond_to do |format|
        format.html
        format.js
      end
    end

    def update
      respond_to do |format|
        if @updated
          flash[:notice] = 'Post was successfully updated.'
          format.html { redirect_to @post }
          format.xml  { head :ok }
          format.js
        else
          format.html { render :action => :edit }
          format.xml  { render :xml => @post.errors, :status => :unprocessable_entity }
          format.js
        end
      end
    end

    def destroy          
      respond_to do |format|
        format.html { redirect_to :action => posts_url }
        format.xml  { head :ok }
        format.js
      end
    end
  end

Nested resources like so:

  class CommentsController < ActionController::Base
    resource_this :nested => [:posts]
  end

This generates a very similar controller to the one above with adjusted redirects and one additional before_filter / loader method pair to grab the parent resource. In this case:

  before_filter :load_post

  def load_post
    @post = Post.find(params[:post_id])
  end

The separation of logic – DB operations in before_filters, rendering in the standard resource controller methods – makes this approach ridiculously easy to customize. Need to load an additional object for the :show action? Slap another before_filter on it. Need to change the path that the :update action redirects to? Override the :update action with your new rendering behavior. And this customized behavior sticks out like a sore thumb – making it infinitely easier to maintain.

Oh, there’s also a generator:

./script/generate resource_this FooKlass [title:string body:text]

This works just like the resource generator, with the addition of the resource_this line to your controller and a functional test. No views are generated, so the test focuses on the XML behavior of this controller.

Contributing

resource_this is hosted on GitHub, so feel free to fork it and send a pull request with your changes.

RailsConf '07 Roundup

David Heinemeier Hansson, by James Duncan Davidson. Creative Commons BY-NC-ND

I spent the past several days in Portland, OR, for RailsConf, the yearly gathering of the vibrant Ruby on Rails community. O’Reilly Media and Ruby Central put on an incredible conference. My only disappointment was that I couldn’t attend all of the presentations. Luckily, most of the presentation slides are online (some with accompanying code!!):

But by far, the most valuable part of the whole event was the time I spent in the hallways and around Portland with other Rails developers. In the two years I’ve been working with Rails, I’ve networked and collaborated with dozens if not hundreds of Rails developers online. It was great to finally be able to associate faces and voices with their respective names, blogs, and chat handles.

I also wrangled Erik Kastner and Charles Brian Quinn into the Capazon project while in Portland – look for some updates on that front in the near future.

A special thanks to the following folks for making my RailsConf an especially great time:

Joyent Slingshot Demo Notes

Eric Wagoner gave a demo of the Joyent’s newly released offline Rails application toolkit, Slinghot. He’s had access to Slingshot for a couple weeks ahead of the public release, and shared his early impressions with us tonight. These are rough notes from his presentation.

  • A local Slingshot app is distributed as a DMG on OS X, which is a full-stack Ruby VM plus your Rails apps. Lots of files. I mean, lots.: $ find Radiant.app | wc -l -> 8087
  • Server side, Slingshot uses a plugin that generates a ‘sync’ controller. This controller has several actions:
    • sync.up
    • sync.down
    • sync.log
  • These toss XML back and forth between the server and client app, containing the changed Models and some metadata.
  • Synchronization conflicts are handled in your application’s domain. This is important to note – Slingshot is not a silver bullet for the age old offline/online synchronization problem. However, it lets you solve this problem however your application needs.

MyConfPlan Javascript Widget

I’ve hacked up a simple Javascript widget for everyone out there using myconfplan, Dr Nic Williams’ wonderful conference session planning tool. Nic recently added JSON and XML feeds to myconfplan, so this is a natural progression.

The XHTML generated by this widget is in hCalendar format, with separate events for each session. If you’re using a Microformats aware browser (or the wonderful microformats bookmarklet), you’ll be able to add my RailsConf sessions to your local calendaring app.

To display your conference session selections on your blog or website, insert these two lines of code:

<script src="http://files.jnewland.com/display_myconfplan.js" type="text/javascript"></script>
<script src="JSON_URL?callback=display_myconfplan" type="text/javascript"></script>

Replace JSON_URL with the URL to the JSON feed for one of your conferences. For example, to display my RailsConf 2007 sessions in my sidebar, I used this:

<script src="http://files.jnewland.com/display_myconfplan.js" type="text/javascript"></script>
<script src="http://myconfplan.com/users/jnewland/conferences/RailsConf2007.js?callback=display_myconfplan" type="text/javascript"></script>

Now seems like a good time to mention that I’ll be at RailsConf 2007 later this month in Portland, OR, and I couldn’t be more excited about it. You can view my session selections in the sidebar, on on myconfplan. Also, I’m staying at the Jupiter Hotel, which looks like a blast. Let me know if you’ll be there too!

Capazon - Capistrano Meets Amazon EC2

UPDATE: For those looking for Capistrano 2.0 support, check out Capazon 0.2.0

Just a quick note to announce Capazon 0.1.0, a Capistrano extension library to manage Amazon EC2 instances. If you are familiar with Capistrano and have an Amazon EC2 account, give it a whirl:

  • gem install capazon
  • Edit your your config/deploy.rb:
require 'capazon'

#AWS login info
set :aws_access_key_id, 'XXX'
set :aws_secret_access_key, 'X'

# Name of the keypair used to spawn and connect to the Amazon EC2 Instance
# Defaults to one created by the setup_keypair task
set :aws_keypair_name, "#{application}-capazon"

# Path to the private key for the Amazon EC2 Instance mentioned above
# Detaults to one created by setup_keypair task
set :aws_private_key_path, "#{Dir.pwd}/#{aws_keypair_name}-key"

#defaults to an ubuntu image
#set :aws_ami_id, "ami-e4b6538d"

#defaults to, um, default
#set :aws_security_group, "default"
  • $ cap describe_images
  * executing task describe_images
IMAGE   ami-0386636a    rbuilder-online/nuxleus-1.3-x86_9327.img.manifest.xml   099034111737    available       true
IMAGE   ami-08866361    rbuilder-online/test1-1.0-x86_9326.img.manifest.xml     099034111737    available       true
IMAGE   ami-1281647b    rbuilder-online/mw-tour-1.6.8-x86_9458.img.manifest.xml 099034111737    available       true
IMAGE   ami-1681647f    rbuilder-online/mw-tour-1.6.8-x86_9459.img.manifest.xml 099034111737    available       true
  • $ AWS_AMI_ID=XXXX cap run_instance

This release just scratches the surface of what I hope to accomplish with Capazon – my end goal is to provide a shared AMI as a companion to Capazon which will encapsulate some Rails deployment best practices.

Please report any bugs you may come across, and stay tuned for updates!

Soylentfoo on Rocketboom, Amazon EC2

Soylentfoo and Tweet, a Twitter action for Quicksilver that I made some updates to, were shown for around 15 milliseconds in today’s episode of Rocketboom. This is both nothing and everything at the same time. Watch it here.

In other meta-news, earlier this morning I moved Soylentfoo from my Textdrive Mixed Grill account to a Amazon EC2 server (domu-12-31-34-00-02-4e.usma2.compute.amazonaws.com, to be exact). For those thinking of using EC2 as a Ruby on Rails host in the future, stay tuned for a release of Capazon, a Capistrano rubygem plugin that aims to make deployment of a Rails application to an Amazon EC2 instance a ridiculously simple process. I’m getting very very close to a release.

Xservs are up!

Today, Lexblog took a big leap in the right direction. Our current server setup has been causing us headaches in recent months, so we decided to go with an in-house hosting solution. Two shiny Apple Xservs were ordered, shipped, and upgraded. Today, they were booted up for the first time, and with the help of Apple Remote Desktop, I’ve been configuring, updating, and generally massaging our new workhorses.

These servers don’t know what’s about to hit them: I’ve prepared a tasty cocktail of Movable Type, LightTPD, and FastCGI to power our blogs, with a sprinkling of Ruby on Rails on the top to make management of our blogs easier.

Watch out for some new features and upgrades from Lexblog in the next few months, as well as drastic speed improvements. They don’t call it FastCGI for nothing ;)

Rails and Subversion

Woah, I missed the addition of the -c flag to the ./script/generate command. (When did this happen?) From the help:

-c, --svn    Modify files with subversion. (Note: svn must be in path)

Let’s try it:

jnewland@slash trunk $ ./script/generate controller dashboard -c
      exists  app/controllers/
      exists  app/helpers/
      create  app/views/dashboard
A         app/views/dashboard
      exists  test/functional/
      create  app/controllers/dashboard_controller.rb
A         app/controllers/dashboard_controller.rb
      create  test/functional/dashboard_controller_test.rb
A         test/functional/dashboard_controller_test.rb
      create  app/helpers/dashboard_helper.rb
A         app/helpers/dashboard_helper.rb

Whoo! No more need to run svn --force add . or anything else after generating a model, controller, or anything. There hasn’t been a day in months where I haven’t found out something like this in rails – a small touch that saves a big headache on my part. Nice work, guys :).

Learning Ruby on Rails

I just responded to the third e-mail in a week from friends asking me “How do I learn Ruby on Rails?” I’ve amassed a good collection of resources that are invaluable to anyone wanting to get on the proverbial train, and I figured it was wrong not to share it with everyone…...so, here’s your ticket:

NOTE: All of the following links assume some programming knowledge. If you’re new to that too, check out Learn To Program, a wonderful intro to programming with Ruby

  • Step two, buy the book. I bought the PDF, and have gone through two home-printed copies while I’m waiting for the final verision to ship (should get here very soon!). The Depot example application will help you understand the underlying structure of Rails.
  • If you’re new to Ruby as well, check out the pickaxe, a great free Ruby book.
  • Check out the Ruby on Rails Wiki for tons of HowTo guides and community documentation. The search feature is helpful.
  • The API reference is the epitome of technical, but it sure is COMPREHENSIVE. Want to know if ActiveRecord::Base##Find returns an Array or a Hash? This is the place to go.
  • Snippets tagged as rails – Random rails code written by random people showcased on a website that was written on rails in less than 2 days. Lots of great code to steal, or use as inspiration.
  • Check out the source code of typo the weblog engine I’m using on my blog. This is an excellent application, as well as excellent code to use for inspiration.
  • Better yet, get a Textdrive hosting account and setup a Typo blog for yourself! Textdrive is the best webhost for Ruby on Rails. Seriously, $12 a month is cheap, and the support is great.
  • #rubyonrails on irc.freenode.net is a great place to real people about Rails. Have a question that’s been bugging you? Ask one of the developers.
  • Sign up for the Rails mailing list. It’s high traffic, but very good, and people share code on there frequently.

That should keep anyone who’s interested in Ruby on Rails occupied for at least, oh, say, a couple months.

NOTE: If you feel that any links should be added to or removed from this list, post a message in the comments.

Older posts: 1 2