On Mar 20, 2009, at 7:35 AM, Anand Chitipothu wrote:
I don't see anything out of the ordinary there. If you're not on
trunk
then you'll definitely want to upgrade.
my working copy is just 3 days old.
Revision: 755487
Last Changed Date: 2009-03-17 11:52:57 +0000 (Tue, 17 Mar 2009)
I have about 30M documents and size of database is 26G.
One thing that is worrying me is couchjs is not taking much cpu, most
of the time cpu is taken by beam.smp. And it looks like both of them
are sharing single core. At least, these 2 processes can take one core
each. Any idea why it is not so?
Hi Anand, this matches my experience. I believe the reason they
appear to share a core is that the Erlang process waits on couchjs to
send back results before continuing on with the indexing. The
workflow looks like
1) Pull docs into memory until we reach some threshold
2) map over docs:
- convert doc to JSON
- send JSON to view server
- receive result from view server
3) Update the btrees with the results from the view server
4) Repeat
As others have said, serious attempts to optimize this workflow
haven't really begun yet. Best, Adam
