IO benchmark between openfiler and nexenta

test performed on 1Gbe network

Client Filesystem : OCFS2

1. openfiler

Version  1.96       ------Sequential Output------ --Sequential Input- --Random-

Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--

Machine   Size:chnk K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP

of-client1.c 20G:1k   183  99 69688  79 33207  48   859  99 79535  44 175.7   4

Latency             14477us    4732ms    1815ms   25254us     141ms     232ms

Version  1.96       ------Sequential Create------ --------Random Create--------

of-client1.cafe24te -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--

              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP

                100   283  80 178076  99  5481  29   295  79 175161  99   417  28

Latency              2833ms      74us    2667ms    1483ms      74us   37497ms

1.96,1.96,of-client1.cafe24test.com,1,1291369522,20G,1k,183,99,69688,79,33207,48,859,99,79535,44,175.7,4,100,,,,,283,80,178076,99,5481,29,295,79,175161,99,417,28,14477us,4732ms,1815ms,25254us,141ms,232ms,2833ms,74us,2667ms,1483ms,74us,37497ms

2. nexenta

Version  1.96       ------Sequential Output------ --Sequential Input- --Random-

Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--

Machine   Size:chnk K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP

of-client1.c 20G:1k   183  99 66867  75 22076  32   906  99 39660  23 327.3   8

Latency             83638us    5235ms    4946ms    2595us    3797ms     313ms

Version  1.96       ------Sequential Create------ --------Random Create--------

of-client1.cafe24te -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--

              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP

                100   281  79 178105  99  7329  38   298  80 174927 100  1307  88

Latency              1616ms     102us    1632ms    1751ms      37us    1378ms

1.96,1.96,of-client1.cafe24test.com,1,1291362617,20G,1k,183,99,66867,75,22076,32,906,99,39660,23,327.3,8,100,,,,,281,79,178105,99,7329,38,298,80,174927,100,1307,88,83638us,5235ms,4946ms,2595us,3797ms,313ms,1616ms,102us,1632ms,1751ms,37us,1378ms

음....nexenta가 우월하게 나올 것으로 기대했는데 생각보다 openfiler의 성능이 더 부각되어버렸다. 다른 부분은 거의 비슷한데 Sequential Input 부분에선 openfiler가 거의 2배 가까운 성능을 보인다. 파일시스템을 ext3로 변경해서 다시 한번 테스트...

 

Client Filesystem : EXT3

1. openfiler

Version  1.96       ------Sequential Output------ --Sequential Input- --Random-

Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--

Machine   Size:chnk K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP

of-client1.c 20G:1k   473  99 68257  52 34886  27  1808  97 84977  39 177.0   6

Latency              3973us    5833ms    2187ms   37819us     150ms     217ms

Version  1.96       ------Sequential Create------ --------Random Create--------

of-client1.cafe24te -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--

              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP

                100 25756  50 198166 100 35566  61 31207  59 +++++ +++ 35956  63

Latency               427ms     730us     776us   15652us      22us     926us

1.96,1.96,of-client1.cafe24test.com,1,1291379021,20G,1k,473,99,68257,52,34886,27,1808,97,84977,39,177.0,6,100,,,,,25756,50,198166,100,35566,61,31207,59,+++++,+++,35956,63,3973us,5833ms,2187ms,37819us,150ms,217ms,427ms,730us,776us,15652us,22us,926us

2. nexenta

Version  1.96       ------Sequential Output------ --Sequential Input- --Random-

Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--

Machine   Size:chnk K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP

of-client1.c 20G:1k   465  99 50176  38 17137  13  1764  96 36093  17 364.1  13

Latency              3571us   11012ms    5019ms   43779us    1481ms     301ms

Version  1.96       ------Sequential Create------ --------Random Create--------

of-client1.cafe24te -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--

              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP

                100 23828  46 199387  99 37299  64 30508  58 +++++ +++ 36326  64

Latency               609ms     749us     765us   20666us      25us   41373us

1.96,1.96,of-client1.cafe24test.com,1,1291357519,20G,1k,465,99,50176,38,17137,13,1764,96,36093,17,364.1,13,100,,,,,23828,46,199387,99,37299,64,30508,58,+++++,+++,36326,64,3571us,11012ms,5019ms,43779us,1481ms,301ms,609ms,749us,765us,20666us,25us,41373us

여기서도 마찬가지...latency도 openfiler가 훨씬 더 좋게 나왔다...zfs를 base filesystem으로 사용하는 nexenta의 승리가 될 것으로 생각했는데 의외의 결과. zfs의 튜닝 포인트를 찾아봤는데, 이미 최적화된 값이 적용되어있으므로 "zfs 튜닝은 악마의 사역" 이라는 포스트만 찾았다...;;;;

이거 고민 좀 더 해봐야겠....