We’re Hiring!

New Bamboo Web Development

Bamboo blog. Our thoughts on web technology.

Refinements under the knife

by Lee Machin

Surgeon Simulator 2013, courtesy of Edge Online

Refinements, made official in Ruby 2.1, are a way of 'monkey patching' objects without exposing those changes to other parts of the code. You would use them like this:

 1 module RefinedString
 2 
 3   refine String do
 4     def to_thing
 5       # ...
 6     end
 7   end
 8 
 9 end
10 
11 class ThingWithRefinedString
12 
13   using RefinedString
14 
15 end

Like most developers, I find the opportunity to abstract a language feature is often irresistible, and is a great way to learn about how something actually works. The question of whether or not the result is usable or useful is often irrelevant.

My hypothesis for such an abstraction was based on the idea of writing one-time refinements that you never intend to re-use, for which doing the module dance feels like introducing unnecessary boilerplate. This is what I thought I could do:

 1 class ThingWithRefinedString
 2 
 3   patch String do
 4     def to_thing
 5       # ...
 6     end
 7   end
 8 
 9   def refined_string(str)
10     str.to_thing
11   end
12 
13 end

Easy enough, right?

Wrong.

Refining refinements

Ruby's an excellent language. It's rarely the case that it doesn't give you something to help you achieve what you want to do, no matter how awful or contrived that idea is. "You want to shoot yourself in the foot? Here's all the rope you need," says Ruby, rubbing its hands together in glee, knowing you can do a lot of things with a rope without shooting yourself. One of my previous quixotic experiments, node_module, is an excellent example of this.

When you call refine, you pass it a block containing all the methods you want to add or override for a certain class. It's rare that you can't substitute that for a proc, since the ability to pass blocks around as objects is what makes Ruby so wonderful for writing expressive and convenient DSLs, like we did ourselves with static_association and configurable_search.

Additionally, you have the ability to create anonymous modules. Combine these two together and you can theoretically create refinement modules at runtime.

 1 module Patch
 2 
 3   def self.included(receiver)
 4     receiver.send :include, ClassMethods
 5   end
 6 
 7   module ClassMethods
 8     def patch(klass, &block)
 9       self.send :using, Module.new { refine klass, &block }
10     end
11   end
12 
13 end

This mixin would presumably allow me to do exactly what I wanted, by calling using with an anonymous module containing my refinements. There are two reasons why this, or any attempt to do it any other sensible way, doesn't work, and to understand that we can look at the Ruby source to see how refinements are implemented.

1. A block can't be passed as a proc

"Wut? But it works everywhere else!" you might think. And there's probably a reason why it doesn't in this case (performance? subtle bugs?). A quick dive into eval.c shows us exactly why:

 1 // ...
 2 
 3 rb_block_t *block = rb_vm_control_frame_block_ptr(th->cfp);
 4 
 5 // ...
 6 
 7 if (block->proc) {
 8     rb_raise(rb_eArgError,
 9              "can't pass a Proc as a block to Module#refine");
10 }

Ruby is fetching the block that is currently being evaluated, and is actually making sure some clever clogs isn't trying to pass in a proc. That at least means that doing this in pure Ruby isn't gonna work, at least not that way.

My second attempt was in C, and after many hours poring through guides on how to write native extensions, and even more hours hopping from file to file in the Ruby source (thanks to GitHub's search tool), I realised that none of the things I needed were exposed in the public API. Of course, after an excursion like that you never leave empty handed, and the insight was fascinating.

At the point of giving up, I remembered what I did with node_module: I could get the source of a block as a string (using sourcify) and then I could transform it into something that would work. Take THAT, interpreter.

 1 require 'sourcify'
 2 
 3 module Patch
 4 
 5   def self.included(receiver)
 6     receiver.send :include, ClassMethods
 7   end
 8 
 9   def self.refinements
10     @refinements ||= {}
11   end
12 
13   def self.new_refinement(receiver, class_to_refine, &block)
14     refinement = Module.new
15     refinement.module_eval <<-RB, __FILE__, __LINE__
16       # Ruby doesn't complain when a refinement looks like this
17       refine #{class_to_refine} do
18         # get the source of the block, without the curly braces, and place it inline
19         #{block.to_source(strip_enclosure: true, ignore_nested: true)}
20       end
21     RB
22 
23     module_name = "#{receiver}::#{class_to_refine}"
24     self.refinements[module_name] = refinement
25   end
26 
27   module ClassMethods
28     def patch(klass, &block)
29       class_exec Patch.new_refinement(self, klass, &block) do |mod|
30         # mod here is our dynamically created refinement
31         using mod
32       end
33     end
34   end
35 
36 end

This actually worked, and I was able to create refinements on the fly. But what about using them?

2. using cannot be called from within a method

They really don't want you taking shortcuts with this feature. Even if you can find a way to create a refinement, the mere fact that the call to using occurs inside a method means it just won't work. Check this out in eval.c:

1 if (prev_frame_func()) {
2   rb_raise(rb_eRuntimeError,
3            "Module#using is not permitted in methods");
4 }

Ruby is checking if the previous control frame is from inside a method, and if it is, you're all out of luck. Since the key entry-point for this idea is a single patch method, you can't really escape the fact you have to call it to do what you want.

Back to the drawing board

The original idea has been proven to be unworkable, but enough work has been done to try a different approach instead. Or a compromise, if you will. We can still hack a refinement into place, and maybe we can adapt it a little bit.

The blocker for the previous attempt was being unable to use using within a method. What happens if we build the abstraction around that instead?

Imagine that we're now testing the ability to do this:

 1 class ThingWithRefinedString
 2 
 3   using patched String do
 4     def to_thing
 5       # ...
 6     end
 7   end
 8 
 9   def refined_string(str)
10     str.to_thing
11   end
12 
13 end

Seems doable, right?

Well... sort of.

Refactoring

Let's first tweak the patch method and simplify it a little:

1 def patched(klass, &block) # was `patch`
2   Patch.new_refinement(self, klass, &block)
3 end

We don't care about refining and using in one fell swoop now, but we do have to worry about precedence. Will we actually receive a block, or will Ruby think we're trying to pass it to using?

In the new example, the latter will be true. We don't want that to happen, so we have to play with the syntax a bit and compromise on the prettiness to get what we want.

 1 class ThingWithRefinedString
 2 
 3   using patched(String) {
 4     def to_thing
 5       # ...
 6     end
 7   }
 8 
 9   def refined_string(str)
10     str.to_thing
11   end
12 
13 end

IT WORKS! Using the do ... end syntax for blocks doesn't work, because Ruby thinks it's being passed to using. Wrapping the first argument to patched in parentheses and using the alternative block syntax ensures the block is going exactly where we want it to.

And there we have it. Success! If you feel brave enough to use this in your own project, you can give the patched gem a shot.

Why so difficult?

I don't know what the reasons are for going to such lengths to make this really hard to achieve. I'd love to be enlightened by any obliging reader, and to understand the reasoning behind restricting things so much, because for all the guns Ruby gives you to hang yourself, this is the first time I've not been able to shoot myself in the foot.