Good to know. I implemented now a replication queue that manages pending replications and active replication processes.
Alexander > On 28. Mar. 2016, at 15:43, Robert Samuel Newson <[email protected]> wrote: > > > This is something we're working on. At present, the replication manager will > try to run all replications at once. We plan to introduce a smarter job > scheduler so that you can configure 'max_replications_running' independently > of other concerns. > > For now, you could use non-continuous replications and listen to /_db_updates > on your sources to know when to trigger those replications. This is, roughly > speaking, what the replication manager will do in a future release. > > B. > > >> On 28 Mar 2016, at 09:01, Alexander Harm <[email protected]> wrote: >> >> Hello all, >> >> I’ve been playing with CouchDB over the last weeks and am running into >> serious problems. My idea was, to have a central database and roughly 1.000 >> project dbs that stem from a filtered local one-way replication from the >> central db using _replicator db. However, triggering 1.000 replications >> grinds CouchDB to a standstill. I read that for each replication Couch >> spawns a process so that approach might not be feasible after all. >> I changed my setup now into a node process listening to the changes feed of >> the central database and then manually triggering a one-shot replication to >> the affected project dbs. That seems to work for now. >> >> However, now the clients should listen to the changes feed of their relevant >> project dbs and again CouchDB is thrashing and crashing all over the place. >> I increased "max_dbs_open" to 1.500 and added "+P 10.000” to the Erlang >> startup parameters. ULIMIT on OS X is set to unlimited. >> >> My central database only has 1.000 small documents so I’m a bit scared of >> what will happen if it grows to millions of documents and hundreds of >> clients listening to the changes feeds. >> >> Are there any recommendations on (nr_of_dbs, nr_of_clients, >> nr_of_replications) => {couch_settings, erlang_settings, system_settings}? >> Or should I not even attempt to host Couch myself (I would really like to) >> and directly use Cloudant? >> >> I’m using CouchDB 1.6.1 (installed via homebrew) on OS X 10.11.4 >> >> Any help appreciated. >> >> Regards, Alexander >
