On 16/05/2009, at 8:45 PM, Adam Kocoloski wrote:
I encountered this "one more byte" problem once before, but it
wasn't 100% reproducible, so I wasn't really comfortable checking in
a workaround. I've basically been waiting to see if it would ever
show up for anyone else :-/
I'm lucky to have a repeatable failure :/
I think we should commit this change, but I'd still like to confirm
that the attachment on the target is not corrupted by the chunk
processing issue (i.e. the last chunk starts with a \r or something
like that).
I can confirm that the target and source of replicated resources
affected by this issue are identical with this fix, and both are
correct i.e. uncorrupted, although this is only according to the
failures I've seen.
Now, on to the checkpointing conditions. I think there's some
confusion about the attachment workflow. Attachments are downloaded
_immediately_ and in their entirety by ibrowse, which then sends the
data as 1MB binary chunks to the attachment receiver processes.
Are they downloaded to disk by ibrowse?
In another thread Matt Goodall suggested checkpointing after a
certain amount of time has passed. So we'd have a checkpointing
algo that considers
* memory utilization
* number of pending writes
* time elapsed
That seems to cover both resource usage and incremental progress. As
far as the couch_util:should_flush mechanism is concerned, I think a
good idea would be to commit 1 document, then 2, then 4 i.e. a binary
increasing window which adapts well to both unreliable and reliable
connections without requiring configuration, which is tricky because
you may want to run the system in a variety of scenarios, and you
might not know what the failure characteristics are (and they may
change over time).
While we on this - any idea about why couchdb is quiting during
replication? It's not giving me any errors.
Antony Blakey
--------------------------
CTO, Linkuistics Pty Ltd
Ph: 0438 840 787
Hi, I'd like to do $THING. I know that $SOLUTION_A and $SOLUTION_B
will do it very easily and for a very reasonable price, but I don't
want to use $SOLUTION_A or $SOLUTION_B because $VAGUE_REASON and
$CONTRADICTORY_REASON. Instead, I'd like your under-informed ideas on
how to achieve my $POORLY_CONCEIVED_AMBITIONS using Linux, duct tape,
an iPod, and hours and hours of my precious time.
-- Slashdot response to an enquiry
