Hi, I am looking into CouchDB as a solution to store a bunch (approx 70 million) archived documents. While planning for the import process, I did some benchmarking to figure out how long the import will take. I get about 50-70 inserts per second on average. However, when I looked for the bottleneck, I couldn't figure it out. I am connected to the database via a fast lan and can verify that the network is not saturated. I can also verify that disk IO is not saturated. The only clue is that of the 4 cpus on the server, it seems that only one is getting fully loaded. Also, of the 5 erlang processes I can see running, only one of them seems to be getting most of the cpu time. I know that erlang is built with smp enabled, so if it is cpu bound, why can't it make use of the other 3 processors?
I thought that perhaps there was some internal write lock issue per database that allowed only one thread to write to a db at a time, so I tried running the benchmarks while hitting multiple databases, but still got the same write rate across the databases. Is there some globally shared resource in couchdb that limits all writes to a single thread? Thanks, Josh
