I went ahead and ran benchmarks again where I created 100 documents first using the bulk api, and then added attachments separately. I did these tests with couch 0.9.0a733182 (the same 0.9 version I used previously). On the dual core pentium, I saw a write rate of 138 documents per second. This is slightly slower than the 150 / second I saw when the attachments were included in the bulk request. Also, on the quad xeon, I saw a write rate of about 200 / second. This is about the same as I saw when attachments were included in the bulk request. However, if I ran two instance of couchdb on the xeon machine and distributed writes across them, the write rate increased to about 360 / second and utilized all 4 cpus. It seems that the only way I can get couchdb to use more cpu's while writing is to actually use multiple instances of couch. Anyway, thanks for the ideas.
Josh Josh Bryan wrote: > This is an interesting idea, and from Damien's note about attachments > executing in a separate thread, is probably worth looking into. I'll > go ahead and run benchmarks on this on Monday. > --thanks, > Josh > > Sven Helmberger wrote: >> Josh Bryan schrieb: >> >>> >>> On a dual core pentium 3.0ghz with erlang 5.6 and couch 0.8.0 using >>> bulk >>> writes, I get throughput of 95 writes / second . I didn't get the >>> 2000 >>> per second that Michael did, but that is likely due to the fact that >>> his >>> documents are considerably smaller than mine (each of my docs has a >>> 4K-10K attachment). By upgrading to the latest couchdb from svn, this >>> improved from 95 / second to about 150 / second. >>> >> >> My totally unsubstantiated guess would be that if you have lots of >> attachments you might actually be faster bulk creating the docs >> without attachments and then adding the attachments later as binary. >> >> Regards, >> Sven Helmberger > >
