replicate is not intended to be faster or in any respect more performant than the builtin replicator. it's not slow, but it's simply not a goal of the project to be faster than the builtin replicator.
my main issue with the builtin replicator has been opaqueness. when replication failures occur it's incredibly hard to diagnose *why*, with fragments of information buried in couch.log. it also keeps a cached state, which is held in _local_docs in a way that is neither documented nor easy to figure unless you can do a packet capture. replicate is written in node.js, it proxies the data through the machine your run it on, when there are failures it throws, in javascript, and exits with as much information as it can provide. it also does *not* keep any cached state around, ever. if you kill the process and start it up again it replicates from seq=0. this is intentional, better to take the performance hit so that we can avoid any issues with cache and _local_docs because if you had wanted replication to be fast, you would have used the builtin replicator :) -Mikeal On Oct 26, 2011, at October 26, 20119:26 AM, kowsik wrote: > Just a quick follow up. Maybe you guys know this already, but thought > I'll share. I was exchanging tweets with @fdmanana about this issue > and he said this profound thing that made me smack my head. > > "@pcapr @couchdb Likely. Push replications read from a local db, so no > connections used for that. And only 1 writer to write to remote db" > > This was news to me. So we've switched over blitz.io from using push > to pull replication and it's frickin fast! So all of the max_http_* > configuration for the replicator only applies to pull apparently. Now > the docs are getting replicated across east/west bi-directionally with > very low latency and I haven't seen the replicator give up after this > switch. > > Like I said, learned something new. Wasn't obvious from the documentation. > > Thanks, > > K. > > ps: If your app is distributed like ours, you might find this > interesting: http://bit.ly/sPEhFH > --- > http://blitz.io > @pcapr > > On Tue, Oct 25, 2011 at 8:46 AM, Max Ogden <[email protected]> wrote: >> MIkeal's is easy to test: npm install replicate && replicate sourceurl >> destinationurl >> >> it's probably slower than the current stable release, but more reliable. the >> new replicator in trunk is apparently the most optimal solution, but it's >> not released in a stable version yet >> >> On Tue, Oct 25, 2011 at 10:57 AM, kowsik <[email protected]> wrote: >> >>> Anyone done any benchmarking on @mikeal's replicate vs. the built-in >>> replicator? For blitz.io, we have east/west continuous replication (on >>> AWS) and I'm frequently seeing slow-downs and replication stalls (even >>> during periods of inactivity). Restarting CouchDB will make it quickly >>> catch up and then things will get backed up again. >>> >>> I like the convenience of having the built-in _replicator DB, but >>> sometimes it takes minutes for docs to get pushed across. We are >>> running 1.1.0 BTW. >>> >>> Thanks, >>> >>> K. >>> --- >>> http://blitz.io >>> @pcapr >>> >>
