Hi Ben, welcome! At the moment, CouchDB does not have any capacity
for intra-document replication checkpointing. And you're right, in
the specific situation you describe Couch would have a difficult time
making any replication progress.
Given that replication over slow, unreliable links is absolutely a
CouchDB design goal, I think we might eventually conjure up some more
magic to make some sort of intra-document (or at least intra-
attachment) checkpointing possible. I think it will be post-1.0,
though. Best,
Adam
On May 14, 2009, at 7:12 AM, Ben Cohen wrote:
Hi all --
This is my first message to the list. I've been watching it for a
little while now and so far everything I read about the design of
couchdb I like a lot! Thanks so much for all the cool work!
One of the uses I'm planning for couchdb involves replicating a
database across a slow, unreliable link which will never become
anything other than slow and unreliable. I understand the
replication is incremental and designed to 'pick up where it left
off' in the case of replication interruption. From the technical
overview on the website:
The replication process is incremental. At the database level,
replication only examines documents updated since the last
replication. Then for each updated document, only fields and blobs
that have changed are replicated across the network. If replication
fails at any step, due to network problems or crash for example,
the next replication restarts at the same document where it left off.
I've got a question about this process. Say you have a document to
be replicated with a 1 megabyte attachment. A replication process
starts, half the doc is transferred successfully and then the
connection dies. Assuming no changes to the source doc, when the
replication restarts will the transfer start from the beginning of
the document or will it pick up somewhere within the doc?
For my use case I have a slow link that will periodically come
online for a certain fixed amount of time and initiate a
replication. If the replication isn't incremental 'within' a single
document, then a document in the database above a certain size will
for me, never make it across and I would imagine cause the
replication to never make forward progress ...
Does couchdb's replication magic avoid the issue for me and
eventually transfer the document across my link?
Thanks much,
Ben Cohen
