Chris Anderson wrote: > On Wed, Jan 7, 2009 at 4:37 PM, Josh Bryan <[email protected]> wrote: > >> Hi, >> >> I am looking into CouchDB as a solution to store a bunch (approx 70 >> million) archived documents. While planning for the import process, I >> did some benchmarking to figure out how long the import will take. I >> get about 50-70 inserts per second on average. However, when I looked >> for the bottleneck, I couldn't figure it out. I am connected to the >> database via a fast lan and can verify that the network is not >> saturated. I can also verify that disk IO is not saturated. The only >> clue is that of the 4 cpus on the server, it seems that only one is >> getting fully loaded. Also, of the 5 erlang processes I can see >> running, only one of them seems to be getting most of the cpu time. I >> know that erlang is built with smp enabled, so if it is cpu bound, why >> can't it make use of the other 3 processors? >> >> I thought that perhaps there was some internal write lock issue per >> database that allowed only one thread to write to a db at a time, so I >> tried running the benchmarks while hitting multiple databases, but still >> got the same write rate across the databases. Is there some globally >> shared resource in couchdb that limits all writes to a single thread? >> >> Thanks, >> Josh >> >> > > Before we can help you diagnose the performance you're seeing, could > you tell us the version of CouchDB and the version of Erlang that you > are using? It wouldn't hurt to describe the hardware in more detail > either. > > I am seeing similar results on two systems.
System 1: Quad core Intel(R) Xeon(R) CPU 5160 @ 3.00GHz 2 GB ram Linux 2.6.18-4 -- Debian Lenny Erlang (BEAM) emulator version 5.6.3 [source] [64-bit] [smp:4] [async-threads:0] [kernel-poll:false] couchdb - Apache CouchDB 0.8.0-incubating System 2: Intel(R) Pentium(R) D CPU 3.00GHz 3 GB ram Erlang (BEAM) emulator version 5.5.5 [source] [async-threads:0] [kernel-poll:false] couchdb - Apache CouchDB 0.9.0a724455-incubating Thanks
