Two weeks ago, Antonio Cangiano compared the performance of different ruby implementations using Ruby 1.9 (YARV)'s benchmark suite. His numbers got me thinking: all alternative implementations performed badly -- most are even way slower than ruby 1.8.5. Does it signal that JVM and .NET are bad platform for Ruby language?
With this doubt I tried the benchmark with XRuby. XRuby is a ruby compiler. Unlike other implementations, it generates Java bytecode that run directly on JVM. But at first the numbers are not impressive: the 0.1.2 version is still slower than Ruby 1.8.5 in most of the cases.
Maybe I should mention that the XRuby team had done virtually nothing for performance before, and we would avoid optimization as long as possible if it makes our code complicated. But after doing some measurements, it turns out our bad performance are largely caused by a logic 'error': as we know Ruby Fixnum can not have singleton methods, but in 0.1.2 it still lookup an empty method table. And along with some bad code practices (iterating an empty ArrayList without checking if it is empty first etc), it makes method lookup much slower than it should be.
I fixed the problem by adding about 10 lines of code, and got great result: In most benchmarks, XRuby 0.1.3 is faster than Ruby 1.8.5. For some, faster in a significant way. There are still some tests in which we are slower, but it looks like caused by poorly implemented builtin.
The following table shows the benchmark result for XRuby 0.1.3. The best part is: we did it without a method cache. YARV is still faster than XRuby, but we have lots of room to improve too.
>java -Xmx512m -jar xruby-0.1.3.jar benchmark\run.rb
|Test||Ruby 1.8.5||XRuby 0.1.3|
* The test environment is Intel Pentium M 1G CPU, 1G Memory, Windows XP SP2, Java 1.5.0_09.