I am aware of performance limitations on ruby's http library. I have therefore also tested and observed similar results with apache ab. Also, I observe the same results regardless of the number concurrent processes and size of the bulk update. Also, on the machine running the ruby script, there is virtually no cpu or net load, most of the time is spent in wait for the couchdb script. This is also why I am not running the ruby script multithreaded, but rather forking processes.
Below is one sample run from AB: ab -T 'application/json' -n 1000 -c 5 -p test.dat http://xxxxxxxx:5984/otherdb_0/ This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0 Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Copyright 2006 The Apache Software Foundation, http://www.apache.org/ Benchmarking 172.23.8.99 (be patient) Completed 100 requests Completed 200 requests Completed 300 requests Completed 400 requests Completed 500 requests Completed 600 requests Completed 700 requests Completed 800 requests Completed 900 requests Finished 1000 requests Server Software: MochiWeb/1.0 Server Hostname: 172.23.8.99 Server Port: 5984 Document Path: /otherdb_0/ Document Length: 70 bytes Concurrency Level: 5 Time taken for tests: 12.693512 seconds Complete requests: 1000 Failed requests: 218 (Connect: 0, Length: 218, Exceptions: 0) Write errors: 0 Total transferred: 245765 bytes Total POSTed: 3013000 HTML transferred: 69765 bytes Requests per second: 78.78 [#/sec] (mean) Time per request: 63.468 [ms] (mean) Time per request: 12.694 [ms] (mean, across all concurrent requests) Transfer rate: 18.91 [Kbytes/sec] received 231.80 kb/s sent 250.71 kb/s total Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.0 0 1 Processing: 24 62 43.2 47 340 Waiting: 24 62 43.2 47 339 Total: 24 62 43.2 47 340 Percentage of the requests served within a certain time (ms) 50% 47 66% 48 75% 50 80% 59 90% 120 95% 133 98% 199 99% 319 100% 340 (longest request) Thanks, Josh Chris Anderson wrote: > On Fri, Jan 9, 2009 at 4:40 PM, Josh Bryan <[email protected]> wrote: > >> #!/usr/bin/ruby >> require 'rubygems' >> require 'couchrest' >> > > There are some hard-to-fix limits on performance of Ruby's Net/HTTP library. > > It's hard to find a perfect reference online, but this post covers all > the bases: > http://apocryph.org/2008/11/09/more_indepth_analysis_ruby_http_client_performance/ > > I also had a hard time getting concurrency higher than 10 from Ruby > but that my be my MacBook. > > Perhaps you could try one of your common queries with Apache bench? > > It would be worth investigating JRuby HTTP performance, especially > multi-threaded. > >
