When you say 3% io usage, what are you measuring? 3% of theoretical throughput on drive read?
What I'd be most interested in is something like the %wait metric from the iostat command to know how much couchdb is waiting for your disk. Without knowing what metric the 3% comes from it's hard to know if there is a lot of seek wait time happening or not. On Tue, Mar 30, 2010 at 18:29, Chris Stockton <[email protected]> wrote: > Hello, > > Recently we were having a issue with our index being many gigabytes > larger then our database. We were indexing the entire document, this > has a great performance increase, on a 8 core xeon machine with 50gigz > of ram calling a 450K doc view that is 80MB worth of raw data over the > wire we get roughly 5MB worth of throughput when calling our view. > > However because of the unrealistic index sizes (after compaction) we > were unable to continue storing the document within the index. Now we > use the include_docs flag, but it seems this comes with a great > performance penalty. On the same data & view (demote of storing the > doc within the view) we are capped at about 400KB per second. For each > additional call on the same view we end up sharing that 400KB of > throughput, I.E. 4 connections get 100KB each, 8, 50KB each. Something > i found interesting was we are NOT iobound at all, with 4 connections > running we run at about 290% cpu usage, but at only 3% io. That is > some pretty heavy cpu usage. > > Has anyone came across any tricks or optimizations for this? I would > greatly appreciate any feedback. > > -Chris >
