Samson's Weblog » Performance

List performance: ruby 1.8 and 1.9 (Part 2)

Samson Wu — Mon, 22 Feb 2010 05:54:36 +0000

接上篇： List performance: ruby 1.8 and 1.9 (Part 1)

有了之前的准备和热身，真正跑起步来似乎轻松不少，先贴上我们用来benchmark的code：

require 'benchmark'

n = ARGV.empty? ? 10 ** 6 : 10 ** ARGV[0].to_i
puts "n=#{n}"
Benchmark.bmbm do |x|
x.report do
list = []
n.times do
list << 0
end
end
end

和之前用来benchmark loop performance的code结构上基本没有区别，这里不再赘述，具体的解释亦可参照前文。

下面分别是ruby 1.8和ruby 1.9的一次完整运行（ n=10~10^8 ）的数据结果（也是我唯一运行的一次…）

ruby 1.8:

n=10
Rehearsal ------------------------------------
0.000000 0.000000 0.000000 ( 0.000014)
--------------------------- total: 0.000000sec

user system total real
0.000000 0.000000 0.000000 ( 0.000012)
==============================================

n=100
Rehearsal ------------------------------------
0.000000 0.000000 0.000000 ( 0.000118)
--------------------------- total: 0.000000sec

user system total real
0.000000 0.000000 0.000000 ( 0.000061)
==============================================

n=1000
Rehearsal ------------------------------------
0.000000 0.000000 0.000000 ( 0.000375)
--------------------------- total: 0.000000sec

user system total real
0.000000 0.000000 0.000000 ( 0.000314)
==============================================

n=10000
Rehearsal ------------------------------------
0.000000 0.000000 0.000000 ( 0.002900)
--------------------------- total: 0.000000sec

user system total real
0.010000 0.000000 0.010000 ( 0.002790)
==============================================

n=100000
Rehearsal ------------------------------------
0.030000 0.000000 0.030000 ( 0.027712)
--------------------------- total: 0.030000sec

user system total real
0.020000 0.000000 0.020000 ( 0.027713)
==============================================

n=1000000
Rehearsal ------------------------------------
0.280000 0.010000 0.290000 ( 0.323298)
--------------------------- total: 0.290000sec

user system total real
0.280000 0.010000 0.290000 ( 0.283621)
==============================================

n=10000000
Rehearsal ------------------------------------
2.740000 0.050000 2.790000 ( 2.809917)
--------------------------- total: 2.790000sec

user system total real
2.760000 0.040000 2.800000 ( 2.810491)
==============================================

n=100000000
Rehearsal ------------------------------------
27.410000 0.490000 27.900000 ( 28.164103)
-------------------------- total: 27.900000sec

user system total real
27.620000 0.480000 28.100000 ( 28.230073)
==============================================

ruby 1.9:

n=10
Rehearsal ------------------------------------
0.000000 0.000000 0.000000 ( 0.000068)
--------------------------- total: 0.000000sec

user system total real
0.000000 0.000000 0.000000 ( 0.000012)
==============================================

n=100
Rehearsal ------------------------------------
0.000000 0.000000 0.000000 ( 0.000038)
--------------------------- total: 0.000000sec

user system total real
0.000000 0.000000 0.000000 ( 0.000032)
==============================================

n=1000
Rehearsal ------------------------------------
0.000000 0.000000 0.000000 ( 0.000297)
--------------------------- total: 0.000000sec

user system total real
0.000000 0.000000 0.000000 ( 0.000209)
==============================================

n=10000
Rehearsal ------------------------------------
0.000000 0.000000 0.000000 ( 0.002164)
--------------------------- total: 0.000000sec

user system total real
0.000000 0.000000 0.000000 ( 0.002025)
==============================================

n=100000
Rehearsal ------------------------------------
0.020000 0.000000 0.020000 ( 0.020366)
--------------------------- total: 0.020000sec

user system total real
0.020000 0.000000 0.020000 ( 0.020873)
==============================================

n=1000000
Rehearsal ------------------------------------
0.190000 0.010000 0.200000 ( 0.208302)
--------------------------- total: 0.200000sec

user system total real
0.190000 0.000000 0.190000 ( 0.208822)
==============================================

n=10000000
Rehearsal ------------------------------------
1.920000 0.050000 1.970000 ( 2.050276)
--------------------------- total: 1.970000sec

user system total real
1.920000 0.050000 1.970000 ( 2.054335)
==============================================

n=100000000
Rehearsal ------------------------------------
19.220000 0.460000 19.680000 ( 20.466960)
-------------------------- total: 19.680000sec

user system total real
19.220000 0.460000 19.680000 ( 20.546417)
==============================================

就这次运行的结果而言，ruby 1.9的performance平均都有25%～35%的提升，加之Part 1测试过的循环性能的倒退，则更可以确定list append操作的performance确实在1.9里面较1.8有了很大的提高。

List performance: ruby 1.8 and 1.9 (Part 1)

Samson Wu — Mon, 22 Feb 2010 02:47:02 +0000

（本文数据较多，在rss reader里面可能可读性不是很好，可考虑直接在线阅读）

昨天看到pipitu的这篇blog，觉得蛮有意思，文章比较了python和java的list performance，只可惜没有include ruby 1.8和ruby 1.9，不然结论会更有悬念一点，呵呵.

由于无法营造一致的软硬件环境，也不愿意重复pipitu关于python和java部分的工作，这里就只好先简单benchmark（如果可以算是的话）一下ruby 1.8和1.9里的list (array) performance了，也许pipitu以后有空更新include ruby也说不定：）

先看一下我们用的ruby的版本：

$ ruby -v
ruby 1.8.7 (2009-06-12 patchlevel 174) [i686-darwin9]

$ ruby1.9 -v
ruby 1.9.1p376 (2009-12-07 revision 26041) [i386-darwin9]

还有os：

$ uname -a
Darwin macbook.local 9.8.0 Darwin Kernel Version 9.8.0: Wed Jul 15 16:55:01 PDT 2009; root:xnu-1228.15.4~1/RELEASE_I386 i386

benchmark的方法和pipitu的python sample类似，就是往空的list里面循环n次append，由于循环也被引入，其performance也值得考虑，因为众所周知ruby里面做一件事犹如孔乙己的茴字写法一样有n种办法，其performance也各不一样，说不定pipitu以后会和python, java进行横向，咱还是挑相对快一点的好～不过话说回来，再怎么的精挑细选，java那边还是基本没有悬念，真正有悬念的，我们都心知肚明，呵呵。

所以真正运动之前先来点热身运动，看看哪个循环在我这相对要快：

require 'benchmark'

n = ARGV.empty? ? 10 ** 6 : 10 ** ARGV[0].to_i
puts "n=#{n}"
Benchmark.bmbm do |b|
b.report("for:") { for i in 1..n ; a = "0"; end }
b.report("times:") { n.times do ; a = "0"; end }
b.report("upto:") { 1.upto(n) do ; a = "0"; end }
b.report("downto:") { n.downto(1) do ; a = "0"; end }
end

以上是我所想到的比较常见的4种茴字写法，程序默认取n=10^6进行循环，也可以接受一个作为幂指数的命令行参数，Benchmark的bmbm方法会先dry run一遍所有test作为rehearsal，等memory和garbage collection的状态相对stable了以后才进行真正的benchmark，这样先运行和后运行的report会相对公平一点。

默认运行（ n=10^6 ）的结果如下：

n=1000000
Rehearsal -------------------------------------------
for: 0.310000 0.000000 0.310000 ( 0.313392)
times: 0.290000 0.000000 0.290000 ( 0.297268)
upto: 0.300000 0.000000 0.300000 ( 0.300311)
downto: 0.310000 0.000000 0.310000 ( 0.304146)
---------------------------------- total: 1.210000sec

user system total real
for: 0.310000 0.000000 0.310000 ( 0.313129)
times: 0.290000 0.000000 0.290000 ( 0.297470)
upto: 0.300000 0.000000 0.300000 ( 0.302154)
downto: 0.310000 0.000000 0.310000 ( 0.305562)

一次完整运行（ n=10~10^8 ）的数据结果如下：

n=10
Rehearsal -------------------------------------------
for: 0.000000 0.000000 0.000000 ( 0.000018)
times: 0.000000 0.000000 0.000000 ( 0.000011)
upto: 0.000000 0.000000 0.000000 ( 0.000047)
downto: 0.000000 0.000000 0.000000 ( 0.000053)
---------------------------------- total: 0.000000sec

user system total real
for: 0.000000 0.000000 0.000000 ( 0.000014)
times: 0.000000 0.000000 0.000000 ( 0.000011)
upto: 0.000000 0.000000 0.000000 ( 0.000011)
downto: 0.000000 0.000000 0.000000 ( 0.000011)
=====================================================

n=100
Rehearsal -------------------------------------------
for: 0.000000 0.000000 0.000000 ( 0.000041)
times: 0.000000 0.000000 0.000000 ( 0.000031)
upto: 0.000000 0.000000 0.000000 ( 0.000033)
downto: 0.000000 0.000000 0.000000 ( 0.000040)
---------------------------------- total: 0.000000sec

user system total real
for: 0.000000 0.000000 0.000000 ( 0.000035)
times: 0.000000 0.000000 0.000000 ( 0.000030)
upto: 0.000000 0.000000 0.000000 ( 0.000031)
downto: 0.000000 0.000000 0.000000 ( 0.000031)
=====================================================

n=1000
Rehearsal -------------------------------------------
for: 0.000000 0.000000 0.000000 ( 0.000367)
times: 0.000000 0.000000 0.000000 ( 0.000315)
upto: 0.000000 0.000000 0.000000 ( 0.000319)
downto: 0.000000 0.000000 0.000000 ( 0.000854)
---------------------------------- total: 0.000000sec

user system total real
for: 0.000000 0.000000 0.000000 ( 0.000273)
times: 0.000000 0.000000 0.000000 ( 0.000250)
upto: 0.000000 0.000000 0.000000 ( 0.000254)
downto: 0.000000 0.000000 0.000000 ( 0.000258)
=====================================================

n=10000
Rehearsal -------------------------------------------
for: 0.000000 0.000000 0.000000 ( 0.003388)
times: 0.010000 0.000000 0.010000 ( 0.003056)
upto: 0.000000 0.000000 0.000000 ( 0.003006)
downto: 0.000000 0.000000 0.000000 ( 0.003084)
---------------------------------- total: 0.010000sec

user system total real
for: 0.010000 0.000000 0.010000 ( 0.003114)
times: 0.000000 0.000000 0.000000 ( 0.002955)
upto: 0.000000 0.000000 0.000000 ( 0.003047)
downto: 0.010000 0.000000 0.010000 ( 0.004650)
=====================================================

n=100000
Rehearsal -------------------------------------------
for: 0.030000 0.000000 0.030000 ( 0.032892)
times: 0.030000 0.000000 0.030000 ( 0.029808)
upto: 0.030000 0.000000 0.030000 ( 0.031190)
downto: 0.030000 0.000000 0.030000 ( 0.030379)
---------------------------------- total: 0.120000sec

user system total real
for: 0.030000 0.000000 0.030000 ( 0.033382)
times: 0.030000 0.000000 0.030000 ( 0.029139)
upto: 0.030000 0.000000 0.030000 ( 0.031608)
downto: 0.030000 0.000000 0.030000 ( 0.030498)
=====================================================

n=1000000
Rehearsal -------------------------------------------
for: 0.310000 0.000000 0.310000 ( 0.313392)
times: 0.290000 0.000000 0.290000 ( 0.297268)
upto: 0.300000 0.000000 0.300000 ( 0.300311)
downto: 0.310000 0.000000 0.310000 ( 0.304146)
---------------------------------- total: 1.210000sec

user system total real
for: 0.310000 0.000000 0.310000 ( 0.313129)
times: 0.290000 0.000000 0.290000 ( 0.297470)
upto: 0.300000 0.000000 0.300000 ( 0.302154)
downto: 0.310000 0.000000 0.310000 ( 0.305562)
=====================================================

n=10000000
Rehearsal -------------------------------------------
for: 3.090000 0.000000 3.090000 ( 3.109693)
times: 2.940000 0.010000 2.950000 ( 2.946662)
upto: 2.990000 0.010000 3.000000 ( 3.003812)
downto: 3.010000 0.000000 3.010000 ( 3.021900)
--------------------------------- total: 12.050000sec

user system total real
for: 3.110000 0.010000 3.120000 ( 3.119928)
times: 2.950000 0.010000 2.960000 ( 2.992141)
upto: 3.000000 0.010000 3.010000 ( 3.044853)
downto: 3.020000 0.000000 3.020000 ( 3.036987)
=====================================================

n=100000000
Rehearsal -------------------------------------------
for: 30.950000 0.070000 31.020000 ( 31.139756)
times: 29.360000 0.060000 29.420000 ( 29.543839)
upto: 29.890000 0.070000 29.960000 ( 30.102346)
downto: 30.080000 0.070000 30.150000 ( 30.271642)
-------------------------------- total: 120.550000sec

user system total real
for: 31.130000 0.080000 31.210000 ( 31.359814)
times: 29.420000 0.050000 29.470000 ( 29.630556)
upto: 30.050000 0.060000 30.110000 ( 30.197589)
downto: 30.180000 0.050000 30.230000 ( 30.361343)
=====================================================

由于精力有限，就不做多次运行取平均值这种相对无聊的事情了，结果虽然不严谨也无法真正判断出究竟谁最快（times和upto，downto是在伯仲之间），但至少可以看出for in range是相对最慢的。

换ruby1.9后的运行结果与上基本一致，详细数据如下：

n=10
Rehearsal -------------------------------------------
for: 0.000000 0.000000 0.000000 ( 0.000029)
times: 0.000000 0.000000 0.000000 ( 0.000050)
upto: 0.000000 0.000000 0.000000 ( 0.000014)
downto: 0.000000 0.000000 0.000000 ( 0.000013)
---------------------------------- total: 0.000000sec

user system total real
for: 0.000000 0.000000 0.000000 ( 0.000014)
times: 0.000000 0.000000 0.000000 ( 0.000011)
upto: 0.000000 0.000000 0.000000 ( 0.000012)
downto: 0.000000 0.000000 0.000000 ( 0.000012)
=====================================================

n=100
Rehearsal -------------------------------------------
for: 0.000000 0.000000 0.000000 ( 0.000061)
times: 0.000000 0.000000 0.000000 ( 0.000103)
upto: 0.000000 0.000000 0.000000 ( 0.000078)
downto: 0.000000 0.000000 0.000000 ( 0.000060)
---------------------------------- total: 0.000000sec

user system total real
for: 0.000000 0.000000 0.000000 ( 0.000066)
times: 0.000000 0.000000 0.000000 ( 0.000061)
upto: 0.000000 0.000000 0.000000 ( 0.000061)
downto: 0.000000 0.000000 0.000000 ( 0.000061)
=====================================================

n=1000
Rehearsal -------------------------------------------
for: 0.000000 0.000000 0.000000 ( 0.000465)
times: 0.000000 0.000000 0.000000 ( 0.000461)
upto: 0.000000 0.000000 0.000000 ( 0.000477)
downto: 0.000000 0.000000 0.000000 ( 0.000428)
---------------------------------- total: 0.000000sec

user system total real
for: 0.000000 0.000000 0.000000 ( 0.000388)
times: 0.000000 0.000000 0.000000 ( 0.000339)
upto: 0.000000 0.000000 0.000000 ( 0.000341)
downto: 0.000000 0.000000 0.000000 ( 0.000339)
=====================================================

n=10000
Rehearsal -------------------------------------------
for: 0.000000 0.000000 0.000000 ( 0.004486)
times: 0.010000 0.000000 0.010000 ( 0.003675)
upto: 0.000000 0.000000 0.000000 ( 0.004407)
downto: 0.000000 0.000000 0.000000 ( 0.004121)
---------------------------------- total: 0.010000sec

user system total real
for: 0.010000 0.000000 0.010000 ( 0.003961)
times: 0.000000 0.000000 0.000000 ( 0.005113)
upto: 0.010000 0.000000 0.010000 ( 0.003592)
downto: 0.000000 0.000000 0.000000 ( 0.003592)
=====================================================

n=100000
Rehearsal -------------------------------------------
for: 0.040000 0.000000 0.040000 ( 0.042573)
times: 0.030000 0.000000 0.030000 ( 0.039094)
upto: 0.030000 0.000000 0.030000 ( 0.041495)
downto: 0.040000 0.000000 0.040000 ( 0.049316)
---------------------------------- total: 0.140000sec

user system total real
for: 0.040000 0.000000 0.040000 ( 0.040422)
times: 0.040000 0.000000 0.040000 ( 0.037030)
upto: 0.040000 0.000000 0.040000 ( 0.041921)
downto: 0.040000 0.000000 0.040000 ( 0.039868)
=====================================================

n=1000000
Rehearsal -------------------------------------------
for: 0.390000 0.000000 0.390000 ( 0.401887)
times: 0.370000 0.000000 0.370000 ( 0.376656)
upto: 0.370000 0.010000 0.380000 ( 0.384750)
downto: 0.370000 0.000000 0.370000 ( 0.380184)
---------------------------------- total: 1.510000sec

user system total real
for: 0.400000 0.000000 0.400000 ( 0.411929)
times: 0.370000 0.000000 0.370000 ( 0.386443)
upto: 0.370000 0.010000 0.380000 ( 0.379820)
downto: 0.370000 0.000000 0.370000 ( 0.379068)
=====================================================

n=10000000
Rehearsal -------------------------------------------
for: 3.920000 0.020000 3.940000 ( 4.021434)
times: 3.690000 0.030000 3.720000 ( 3.844118)
upto: 3.710000 0.030000 3.740000 ( 3.844035)
downto: 3.700000 0.020000 3.720000 ( 3.795330)
--------------------------------- total: 15.120000sec

user system total real
for: 3.910000 0.020000 3.930000 ( 4.003016)
times: 3.670000 0.020000 3.690000 ( 3.749495)
upto: 3.700000 0.020000 3.720000 ( 3.919785)
downto: 3.690000 0.030000 3.720000 ( 3.789975)
=====================================================

n=100000000
Rehearsal -------------------------------------------
for: 39.260000 0.240000 39.500000 ( 40.293460)
times: 36.700000 0.210000 36.910000 ( 37.450910)
upto: 37.020000 0.260000 37.280000 ( 38.963391)
downto: 36.940000 0.230000 37.170000 ( 37.806189)
-------------------------------- total: 150.860000sec

user system total real
for: 39.230000 0.240000 39.470000 ( 40.471694)
times: 36.850000 0.240000 37.090000 ( 38.302817)
upto: 36.870000 0.230000 37.100000 ( 37.802815)
downto: 36.970000 0.220000 37.190000 ( 38.052300)
=====================================================

有趣的是ruby1.9的循环操作性能竟然普遍都是负增长，这倒为我们之后的list performance比较埋下了伏笔。

热身运动到此结束，似乎有喧宾夺主之嫌，所以赶紧搞个part 2…，下篇文章我们会用上面循环性能测试的疑似最佳表现者times来进行list append性能测试，敬请期待。

Samson's Weblog » Performance

List performance: ruby 1.8 and 1.9 (Part 2)

Related posts

List performance: ruby 1.8 and 1.9 (Part 1)

Related posts