Hi, Instead of reducing worker batch size to such low value (which means that replication will have to make at least 3200 requests to transfer your documents), you can also try to increase Erlang process heap size by using +hms parameter via ERL_FLAGS environment variable. You can read more about it here: http://www.erlang.org/doc/man/erl.html http://www.erlang.org/doc/efficiency_guide/processes.html
-- ,,,^..^,,, On Thu, Oct 30, 2014 at 1:37 AM, Mat Degerholm <[email protected]> wrote: > Hi again, > > Apologies for the re-send but as I didn't receive any replies to my earlier > message I wanted to check it hadn't slipped through the cracks. Can anyone > advise of a way to increase the Couch/Erlang heap? > > Regards, > Mat > > > From: Mat Degerholm > Sent: Wednesday, 22 October 2014 5:25 p.m. > To: '[email protected]' > Cc: Mat Degerholm > Subject: Assistance in tuning CouchDB heap and/or replicator config > > Hi guys, > > Just looking for a little assistance in tuning the CouchDB heap and/or > replicator config. > > When performing an initial/full replication to an empty 2nd node using the > _replicator database, the 2nd node crashes with the following error: > > hend=0x00007f280d67f628 > stop=0x00007f280d67f418 > htop=0x00007f280d6a27b0 > heap=0x00007f280d592028 > beam/erl_gc.c, line 427: <0.194.0>: Overrun stack and heap > heart: Tue Oct 21 13:29:55 2014: Erlang has closed. > heart: Tue Oct 21 13:29:56 2014: Executed "/opt/couchdb/bin/couchdb -k" -> 0. > Terminating. > > As my replication source node does have a sizeable data set (16,000 docs, > 180,000 updates, total size around 1GB), I found I could prevent the crash by > reducing the replication worker batch size. But I found I had to drop down 2 > orders of magnitude (from the default of 500 down to 5) before the problem > disappeared! > > [replicator] > ; With lower batch sizes checkpoints are done more frequently. Lower batch > sizes > ; also reduce the total amount of used RAM memory. > worker_batch_size = 5 > > I'm nervous having to tweak the replicator setup so far from the defaults, so > is there any way I can tune the stack/heap? I have included my ulimit output > in case you could recommend any changes (note that my stack size and max > memory are unlimited). > > Thanks in advance, > Mat > > > Technical Details > > CouchDB version: 1.6.0 > Erlang: OTP 17.0 > OS: RHEL 6.5 > ERL_MAX_PORTS: 4096 > ERL_FLAGS: "+A 4" > > > ulimit output: > core file size (blocks, -c) 0 > data seg size (kbytes, -d) unlimited > scheduling priority (-e) 0 > file size (blocks, -f) unlimited > pending signals (-i) 30502 > max locked memory (kbytes, -l) 64 > max memory size (kbytes, -m) unlimited > open files (-n) 4096 > pipe size (512 bytes, -p) 8 > POSIX message queues (bytes, -q) 819200 > real-time priority (-r) 0 > stack size (kbytes, -s) unlimited > cpu time (seconds, -t) unlimited > max user processes (-u) 1024 > virtual memory (kbytes, -v) unlimited > file locks (-x) unlimited
