Did you specify "continuous": true in the replication spec? If you did, do you see the same slowdown if you omit that flag?
This sounds rather similar to a bug we encountered in early builds of BigCouch, but I don't remember the history of the codebase well enough to say whether the bug might be latent in Apache CouchDB. Adam On Aug 20, 2013, at 8:49 AM, Jean-Yves Moulin <[email protected]> wrote: > Hi CouchDB users, > > I'm trying to replicate a big database: 35 millions documents. Average doc > size is 500 bytes. > > When replication starts, from server A to server B (from server B in pull > mode), we got replicate at 1100 documents per second. Then, after some > thousands of documents replicated, replication slowly decelerates until 100 > doc/sec. > > If I restart the couchdb daemon on server B, replication restarts at 1100 > doc/sec. And then slow down again. At 100 doc/sec I need multiple weeks to > replicate the full database.. :-( > > Obviously, the two servers are far from their performances limits: one half > CPU used (on 16), 10GB of RAM used (on 64GB), and very small I/O (on 24 disks > + 2 ssd for cache). > > You can see here: http://jym.eileo.net/couchdb_req.png , A small graph > showing the http requests on server A (almost all GET requests come from > server B). The restart of server B is clearly visible at 10:30. > > I already try to change worker_batch_size, worker_processes and > http_connections but this doesn't change anything. > > Do you know if this is a know bug or how can I improve the situation ? > > > Thank you very much. > > best, > jym
