On 15/05/2009, at 2:44 PM, Antony Blakey wrote:
I have a 3.5G Couchdb database, consisting of 1000 small documents,
each with many attachments (0-30 per document), each attachment
varying wildly in size (1K..10M).
To test replication I am running a server on my MBPro and another
under Ubuntu in VMWare on the same machine. I'm testing using a pure
trunk.
Doing a pull-replicate from OSX to Linux fails to complete. The
point at which it fails is constant. I've added some debug logs into
couch_rep/attachment_loop like this: http://gist.github.com/112070
and made the suggested "couch_util:should_flush(1000)" mod to try
and guarantee progress (but to no avail). The debug output shows
this: http://gist.github.com/112069 and the document it seems to
fail on is this: http://gist.github.com/112074 . I'm only just
starting to look at this - any pointers would be appreciated.
It's not immediately obvious to me at the moment, but I wonder if with
many small documents with lots of attachments considerable larger than
the document size, it's possible to spawn too many async attachment
downloads? In which case, maybe throttling the number of concurrent
downloads would be a good idea.
Antony Blakey
--------------------------
CTO, Linkuistics Pty Ltd
Ph: 0438 840 787
The greatest challenge to any thinker is stating the problem in a way
that will allow a solution
-- Bertrand Russell
