On Aug 12, 2009, at 7:52 AM, Tim Uckun wrote:
Has anybody done any benchmarks on the replication speed of couchdb?
If I have four servers each with couchdb running on them along with a
client application that talks to it and one client makes an entry how
long will it take for the other three to get the entry?
In CouchDB 0.9.x (0.9.1 is the current release) you have to trigger
the replication manually. You can write a script that listens for
update_notifications and POSTs to _replicate. So that's once source
of lag.
Replication basically loops through the following 4 steps:
1. ask the source for new updates since the last replication
2. ask the target which of those updates it does not have
3. get the missing updates on the source
4. write them to the target
In most cases, 2 of these will be local requests and two will be
remote HTTP requests, so the latency between the source and target
enters into the equation twice.
Will all that said, its certainly possible to configure things so that
latencies are O(100ms) for single document updates. I'm afraid there
isn't a great way to measure this in 0.9.x.
In the upcoming release CouchDB will have some facility for continuous
replication, which takes the burden off of you to trigger on update
and reduces the lag for updates to show up on the target. Also, each
DB will have a _changes feed that you can listen to. A script that
listens to _changes on both source and target could do a reasonable
calculation of the replication lag.
If each client makes a thousand entries in a 15 minute period will
there be a huge backlog built up waiting to replicate?
It depends on how you're triggering the replication. If you're
triggering it on update (or using continuous replication in the
upcoming release) there shouldn't be any backlog.
In CouchDB 0.9.x push replication (local source, remote target)
generally seems to run at about 80-100 docs/second for me using O(1KB)
documents and no attachments. Pull replication (remote source, local
target) is significantly slower because it replicates documents one at
a time. The trunk code is faster than either of these.
Also is it possible to put files in the couchdb? Specifically media
files like wav.
Yes, absolutely. We do this using document attachments -- PUT /dbname/
docname/media.wav
Will they replicate as well?
Yes, although replicating attachments uses a lot more memory than it
should. We're working on that. I'd stuck to a couple of MBs or less
per document for now.
Best, Adam
