New Bamboo Web Development

Bamboo blog. Our thoughts on web technology.

Stubbing and Setting Expectations on HTTP Requests Is Now Easy With WebMock

by Bartosz Blimke

I've been working recently on a Ruby application which was making HTTP calls to a remote service. We chose rest-client as a client HTTP library. We needed some way of testing the behaviour we were going to implement.

We wanted to run our tests in isolation, without making any real requests over the Internet.
One obvious option was to use some mocking library to stub rest-client methods and set expectations on them. Doing this is always a pain. You usually end up testing the implementation instead of behaviour.

If you use Net::HTTP directly to spec this code:

1 res = Net::HTTP.start("www.google.com", 80) {|http|
2   http.get("/")
3 }

you have to write the following code in RSpec:

1 @mock_http = mock("http")
2 Net::HTTP.stub!(:start).and_yield @mock_http
3 @mock_http.should_receive(:get).with("/")

If you change your HTTP library, even if both libraries are based on Net::HTTP and behaviour of the application won't change, you still need to fix all your tests where you stubbed methods specific to HTTP library.

Fakeweb as a good alternative. It allows stubbing HTTP requests at low Net::HTTP level so it works with any library built on top of Net::HTTP.
Dispite Fakeweb's excellent solution to the problem, it has unfortunately couple of limitations we were missing. Requests can be matched only by HTTP method or URI and we needed to match POST requests based on body and headers. Fakeweb also didn't support matching of escaped and unescaped URIs.
Another feature I was missing was setting expectations on request invocations. Pat Allan's fakeweb-matcher is some solution but it has the same issues as Fakeweb and I also needed to set some more advanced expectations.

At the beginning I planned to take Fakeweb's source code and extend it but I soon realised that Fakeweb's architecture will make it quite difficult.

During our Hackday I started working on a new library for stubbing HTTP requests. That's how WebMock was born.

Here are some of the main WebMock features:

  • Stubbing HTTP requests at low Net::HTTP level (no need to change tests when you change HTTP lib interface)
  • Setting and verifying expectations on HTTP requests
  • Matching requests based on method, URI, headers and body
  • Smart matching of the same URIs in different representations (also encoded and non encoded forms)
  • Smart matching of the same headers in different representations.
  • Support for Test::Unit and RSpec (and can be easily extended to other frameworks)
  • Support for Net::HTTP and other http libraries based on Net::HTTP (i.e RightHttpConnection, rest-client, HTTParty)
  • Easy to extend to other HTTP libraries apart from Net::HTTP

Here is an example code using WebMock in Test::Unit

 1 stub_request(:post, "www.google.com").
 2     with(:headers => { 'Content-Length' => 3 }).to_return(:body => "abc")
 3 
 4 #Actual request
 5 req = Net::HTTP::Post.new('/')
 6 req['Content-Length'] = 3
 7 Net::HTTP.start('http://www.google.com/', 80) {|http|
 8     http.request(req, 'abc')
 9 }    # ===> Success
10 
11 assert_requested :post, "http://www.google.com", 
12     :headers => { 'Content-Length' => 3 }, :body => "abc", :times => 1    # ===> Success
13     
14 assert_not_requested :get, "http://www.something.com"    # ===> Success

The same functionality in RSpec

 1 stub_request(:post, "www.google.com").
 2     with(:headers => { 'Content-Length' => 3 }).to_return(:body => "abc")
 3 
 4 #Actual request
 5 req = Net::HTTP::Post.new('/')
 6 req['Content-Length'] = 3
 7 Net::HTTP.start('http://www.google.com/', 80) {|http|
 8     http.request(req, 'abc')
 9 }    # ===> Success
10 
11 WebMock.should have_requested(:get, "www.google.com").
12     with(:body => "abc", :headers => { 'Content-Length' => 3 }).once     # ===> Success
13     
14 WebMock.should_not have_requested(:get, "www.something.com")    # ===> Success

You can also choose the following syntax in RSpec:

 1 stub_request(:post, "www.google.com").
 2     with(:headers => { 'Content-Length' => 3 }).to_return(:body => "abc")
 3 
 4 #Actual request
 5 req = Net::HTTP::Post.new('/')
 6 req['Content-Length'] = 3
 7 Net::HTTP.start('http://www.google.com/', 80) {|http|
 8     http.request(req, 'abc')
 9 }    # ===> Success
10 
11 request(:post, "www.google.com").
12   with(:body => "abc", :headers => { 'Content-Length' => 3 }).should have_been_made.once
13 
14 request(:get, "www.something.com").should_not have_been_made    # ===> Success

You can install it with

1 gem install webmock --source http://gemcutter.org

Now in your test/test_helper.rb add the following lines:

1 require 'webmock/test_unit'
2 include WebMock

or if you use RSpec add these lines to spec/spec_helper.rb:

1 require 'webmock/rspec'
2 include WebMock

To find more usage example and information, check WebMock's site on Github.