On Feb 5, 2013, at 11:02 AM, Robert Newson <[email protected]> wrote:
> If you change settings via http requests to _config, no, but if you > just changed the .ini file on disk, yes. It's best to use PUT/GET to > _config/section/key imo. thanks. I've been making these config changes via request to _config. > > B. > > On 5 February 2013 18:56, Stephen Bartell <[email protected]> wrote: >> Nathan, >> >> I dropped the pool size down to 500 and still the same story. I also tried >> lower the number of replicator processes down to 1 per replicator. Again >> same thing. >> >> All the while, I keep an eye on how much memory beam.smp consumes during one >> replication "wave" and it never exceeds 2%. So Im reluctant to think that >> the os is running out of memory. It does seem like theres some sort of >> process contention however. The error code that the replicators are >> reporting while trying to POST is 503. I assume that this is for the web >> server being unavailable. >> >> Yes Im going to add filtering on top of this, and I think Im going to need >> to do those in eel, although Id like to try first to avoid it. >> >> This is probably a dumb question, do I need to restart couch after changes >> with these settings? >> >> >> On Feb 5, 2013, at 10:22 AM, Nathan Vander Wilt <[email protected]> >> wrote: >> >>> Hi Stephen, >>> >>> I've been doing some tests related to replication lately too >>> (continuous+filtered in my case). I suspect the reason Futon hangs is >>> because your whole VM is running out of RAM due to your very high >>> os_process_limit. I went in to a bit more detail in >>> http://mail-archives.apache.org/mod_mbox/couchdb-dev/201302.mbox/%[email protected]%3e >>> but this setting basically determines the size of the couchjs worker pool >>> — you'd probably rather have a bit of contention for the pool at a >>> reasonable size (maybe ~100 per GB free, tops?) than start paging. >>> >>> hth, >>> -natevw >>> >>> >>> >>> On Feb 4, 2013, at 5:15 PM, Stephen Bartell wrote: >>> >>>> Hi all, >>>> >>>> I'm hitting some limits while replicating , I'm hoping someone could >>>> advise. >>>> Im running this in a VM on my macbook with the following allocated >>>> resources: >>>> ubuntu 11.04 >>>> 4 cores @ 2.3ghz >>>> 8 gb mem >>>> >>>> I'm doing a one-to-many replication. >>>> 1) I create one db named test. >>>> 2) Then create [test_0 .. test_99] databases. >>>> 3) I then set up replications from test -> [test_0 .. test_99]. 100 >>>> replications total. >>>> 4) I finally go to test and create a doc, hit save. >>>> >>>> When I hit save, futon becomes completely unresponsive for around 10sec. >>>> It eventually returns to normal behavior. >>>> >>>> Tailing the couchdb log I find waves of the following errors: >>>> [Tue, 05 Feb 2013 00:46:26 GMT] [info] [<0.6936.1>] Retrying POST request >>>> to http://admin:*****@localhost:5984/test_25/_revs_diff in 1.0 seconds due >>>> to error {code,503} >>>> >>>> I see that the replicator is finding the server to be unresponsive. The >>>> waves of these messages show that replicator retries in 0.25 sec, then 0.5 >>>> sec, then 1sec, then 2sec. This is expected. Everything settles done >>>> after about 4 retries. >>>> >>>> So my first thought is resource limits. I threw the book at it and set : >>>> 1) max_dbs_open: 500 >>>> 2) os_process_limit: 5000 >>>> 3) http_connections: 20000 >>>> 4) ulimit -Sn 4096 (the hard limit is 4096) >>>> >>>> I really don't know whats reasonable for these values relative to how many >>>> replications I am setting up. So these values, save max_dbs_open, are >>>> all stabs in the dark. >>>> >>>> No change in performance. >>>> >>>> So, I'm at a loss now. what can I do to get all this to work? Or what am >>>> I doing wrong? And note that this is only a test. I aim to quadruple >>>> the amount of replications and have lots and lots of insertions on the so >>>> called "test" database. Actually, there will be several of these >>>> one-to-many databases. >>>> >>>> I've heard people get systems up to thousands of dbs and replicators >>>> running just fine. So I hope Im just not offering to right sacrifices up >>>> to couchdb yet. >>>> >>>> Thanks for any insight, >>>> >>>> sb >>>> >>> >>
