Thanks for the response Adam :) Some updates below: On 1 Dec 2010, at 13:30, Adam Kocoloski wrote: <snip> > > So the Erlang VM starts 16 schedulers by default, right? Some people have > reported improvements in Erlang application performance with HyperThreading > disabled, but I've not heard of any CouchDB-specific tests of that option yet.
Yeah, that's right - 16:16 by default. <snip> > >> The database is pretty small (just under 100K docs) and I am querying a view >> that includes some other docs (the request contains include_docs=true) and >> using jmeter on another identical box to generate the traffic. > > include_docs=true is definitely more work at read time than embedding the > docs in the view index. I'm not sure about your application design > constraints, but given that your database and index seem to fit entirely in > RAM at the moment you could experiment with emitting the doc in your map > function instead ... > >> The total amount of data returned from the request is 1467 bytes. > > ... especially when the documents are this small. Sure, but I would have expected that to only really help if the system was contending for resources? I am using linked docs so not sure about emitting the entire doc in the view. <snip> >> >> >> Before coming here to question my findings I took a 3rd box (same spec) and >> built couch from the tip of the 1.1.x branch (rev 1040477). After compiling >> couch and installing it I found that it didn't start up (or log anything >> useful). After a bit of digging I figured it's probably due to the age of >> the erlang version being used - I upgraded to OTP R14B and rebuilt couch >> against it. This gave me a working install again. > > Hmm, I've heard that we did something to break compatibility with 12B-5 > recently. We should either fix it or bump the required version. Thanks for > the note. COUCHDB-856? > >> I got an immediate throughput increase to ~500 requests/s which was nice but >> the data being collected via sadc still showed that the cpu was at most 20% >> utilised and the disk controller was doing next to nothing (I assume the OS >> cache already has the data requested so no trip to disk required?) >> >> At this point I started to wonder if jmeter is unable to send in enough >> requests to stress couch so I started up another jmeter instance on another >> box and had it also send in requests to couch. What i noticed was that the >> total throughput didn't increase - it was just split over both jmeter >> instances. > > How many concurrent requests are submitted by each jmeter instance? 25. <snip> > > Do you know if the CPU load was spread across cores or concentrated on a > single one? One thing Kenneth did not mention in that thread is that you can > now bind Erlang schedulers to specific cores. By default the schedulers are > unbound; maybe RHEL is doing a poor job of distributing them. You can bind > them using the default strategy for your CPUs by starting the VM with the > "+sbt db" option. It was using most of 2 cores. I had a go with "+sbt db" and it didn't perform as well as "-S 16:2". WRT disabling HT - I need to take a trip to the datacentre to disable HT in the bios but I tried disabling some cores with: echo 0 > /sys/devices/system/node/nodeX/cpuX/online Which should stop the kernel seeing the core - not as clean as disabling it in the bios but should suffice. /proc/cpuinfo stopped showing the cores I removed so it looks like it worked. Again I didn't see any improvement. Cheers Huw
