Hi Matt,
On May 14, 2009, at 10:36 AM, Matt Goodall wrote:
2009/5/14 Adam Kocoloski <[email protected]>:
Hi Ben, welcome! At the moment, CouchDB does not have any capacity
for
intra-document replication checkpointing. And you're right, in the
specific
situation you describe Couch would have a difficult time making any
replication progress.
Given that replication over slow, unreliable links is absolutely a
CouchDB
design goal, I think we might eventually conjure up some more magic
to make
some sort of intra-document (or at least intra-attachment)
checkpointing
possible. I think it will be post-1.0, though. Best,
Adam
On May 14, 2009, at 7:12 AM, Ben Cohen wrote:
Hi all --
This is my first message to the list. I've been watching it for a
little
while now and so far everything I read about the design of couchdb
I like a
lot! Thanks so much for all the cool work!
One of the uses I'm planning for couchdb involves replicating a
database
across a slow, unreliable link which will never become anything
other than
slow and unreliable. I understand the replication is incremental
and
designed to 'pick up where it left off' in the case of replication
interruption. From the technical overview on the website:
The replication process is incremental. At the database level,
replication only examines documents updated since the last
replication. Then
for each updated document, only fields and blobs that have
changed are
replicated across the network. If replication fails at any step,
due to
network problems or crash for example, the next replication
restarts at the
same document where it left off.
Is this actually accurate? It suggests that documents are replicated
one-by-one and that replication can be interrupted at any point and
will continue from wherever it got to before the interruption.
Yes, there are some inaccuracies in that paragraph. We do save
checkpoints, but not per-document. We also transfer the whole
document, not just changed fields. In some respects the Overview is
really part Roadmap. We've taken some flak for this before, perhaps
it's time to revisit that page.
Firstly, I believe the whole replication has to complete before any
updates are visible in the target database. If I restart the server in
charge of replication and then restart the replication it always seems
to start from the beginning. i.e. the Futon's "Processed source update
#xxx" status starts from 0 (when replicating an empty database).
The exact behavior has changed as CouchDB has evolved. Are you
running 0.9 or higher? In that case Couch should be saving
checkpoints for every ~10MB of document data that comes across the
wire. If it fails after a checkpoint, the next replication should not
be starting from 0. If it is restarting, I'd consider that a bug.
Others have commented that the 10MB threshold really needs to be
configurable. E.g., set it to zero and you get per-document
checkpoints, but your throughput will drop and the final DB size on
the target will grow. Super easy to do, but no one's gotten around to
it.
Secondly, if the network connection fails in the middle of replication
(closing an ssh tunnel is a good way to test this ;-)) then it seems
to retry a few (10) times before the replicator process terminates. If
the network connection becomes available again (restart the ssh
tunnel) the replicator doesn't seem to notice. Also, I just noticed
that Futon still lists the replication on its status page.
That's correct, the replicator does try to ignore transient failures.
If I'm correct, and I really hope I'm missing something, then
couchdb's replication is probably not currently suitable for
replicating anything but very small database differences over an
unstable connection. Does anyone have any real experience in this sort
of scenario?
Personally, I do not. I think the conclusion is a bit pessimistic,
though. Adding a configurable checkpoint threshold should make it
possible to (slowly) replicate very large DB differences. Ben's
original point about the inability to replicate very large documents
still stands, though. I've opened a ticket to remind us about adding
that feature in the future. Cheers,
Adam
