Re: CPU constantly above 15% even when no client is talking to CouchDB
Give `+sbwt none` a try and measure how it affects your cluster. It might reduce the idle CPU usage a bit. The only adverse effect it might have is it might increase the latency of requests, but it's something you can measure. The `+S ${SCHEDULERS}` is how you'd explicitly specify how many scheduler threads to start, as you've read in the guide, it's mostly when running in a containerized environment where the containers "sees" all the CPUs and would start a scheduler for each one of them even though its quota might effectively be only 1 CPUs. There is also some background stats gathering and expiring of open indices and db handles. Those won't show in active tasks. But without access to the system or more details it's hard to recommend changing any of them. > -ssl session_lifetime 300 Don't think that would have a large effect, especially if you terminate the SSL in haproxy. Cheers, -Nick On Fri, Sep 2, 2022 at 6:02 AM Al Z. wrote: > > Thank you very much Nick for the reply. > Most appreciated. > > 1) I checked _active_tasks and it return empty array [] > > > 2) About +sbwt, we are running on physical hardware ... no container. > > +sbwt none > +S ${SCHEDULERS} > > Would the above config changes you suggested earlier have any adverse > effect? > > > 3) In my config file, there is > > # Set maximum SSL session lifetime to reap terminated replication readers > -ssl session_lifetime 300 > > Note that I am using HAProxy for SSL. > Could this have any adverse effect? > > Thank you very much. > > Alfred. > > > On Thu, Sep 1, 2022 at 6:02 AM Nick Vatamaniuc wrote: > > > Hi Alfred, > > > > It could be you're seeing the effect of busy waiting from Erlang VM > > schedulers. Erlang VM would spawn a scheduler for each CPU core, and > > then after any of those schedulers run out of work, they would still > > spin in a busy loop for a bit waiting for more work. Sometimes that is > > not desirable, especially in containerized environments so then it's > > recommended to disable busy waiting by setting +sbwt to none in the > > vm.args file. Give that a try and see if it helps. More info about it > > can be found at https://adoptingerlang.org/docs/production/kubernetes/ > > > > Another reason could be that there are some background index building > > or compaction jobs running.. Check out the output of _active_tasks to > > see if that's the case. > > > > Cheers, > > -Nick > > > > On Sun, Aug 14, 2022 at 11:38 AM Al Z. wrote: > > > > > > Hello. > > > I have the latest CouchDB 3.2.2 on Ubuntu 22.04 > > > > > > Even when there are no active clients talking to couchdb, the CPU is > > always > > > above 15%. > > > Not sure why. > > > couchdb's beam.smp is currently the busiest process on the server. > > > > > > [image: image.png] > > > > > > > > > # ps aux | grep couchdb > > > couchdb 979 13.4 0.1 4647432 100284 ? Ssl 15:03 2:10 > > > /opt/couchdb/bin/../erts-12.2.1/bin/beam.smp -K true -A 16 -Bd -- -root > > > /opt/couchdb/bin/.. -progname couchdb -- -home /opt/couchdb -- -boot > > > /opt/couchdb/bin/../releases/3.2.2/couchdb -name couchdb@localhost > > > -setcookie SOME_SECRET_COOKIE -kernel inet_dist_use_interface {127,0,0,1} > > > -kernel error_logger silent -sasl sasl_error_logger false -noshell > > -noinput > > > -smp enable -ssl session_lifetime 300 -- -config > > > /opt/couchdb/bin/../releases/3.2.2/sys.config -- > > > couchdb 0.0 0.0 7136 220 ?S15:03 0:00 > > > /opt/couchdb/bin/../erts-12.2.1/bin/epmd -daemon > > > couchdb 1163 0.0 0.0 2780 972 ?Ss 15:03 0:00 > > > erl_child_setup 65536 > > > couchdb 2727 0.0 0.0 3736 1192 ?Ss 15:04 0:00 > > > inet_gethost 4 > > > couchdb 2728 0.0 0.0 3736 108 ?S15:04 0:00 > > > inet_gethost 4 > > > root2982 0.0 0.0 6612 2220 pts/1S+ 15:30 0:00 grep > > > --color=auto couchdb > > > > > > > > > Any hint will be appreciated. > > > > > > Thank you very much. > > > > > > Alfred. > >
Re: CPU constantly above 15% even when no client is talking to CouchDB
Thank you very much Nick for the reply. Most appreciated. 1) I checked _active_tasks and it return empty array [] 2) About +sbwt, we are running on physical hardware ... no container. +sbwt none +S ${SCHEDULERS} Would the above config changes you suggested earlier have any adverse effect? 3) In my config file, there is # Set maximum SSL session lifetime to reap terminated replication readers -ssl session_lifetime 300 Note that I am using HAProxy for SSL. Could this have any adverse effect? Thank you very much. Alfred. On Thu, Sep 1, 2022 at 6:02 AM Nick Vatamaniuc wrote: > Hi Alfred, > > It could be you're seeing the effect of busy waiting from Erlang VM > schedulers. Erlang VM would spawn a scheduler for each CPU core, and > then after any of those schedulers run out of work, they would still > spin in a busy loop for a bit waiting for more work. Sometimes that is > not desirable, especially in containerized environments so then it's > recommended to disable busy waiting by setting +sbwt to none in the > vm.args file. Give that a try and see if it helps. More info about it > can be found at https://adoptingerlang.org/docs/production/kubernetes/ > > Another reason could be that there are some background index building > or compaction jobs running.. Check out the output of _active_tasks to > see if that's the case. > > Cheers, > -Nick > > On Sun, Aug 14, 2022 at 11:38 AM Al Z. wrote: > > > > Hello. > > I have the latest CouchDB 3.2.2 on Ubuntu 22.04 > > > > Even when there are no active clients talking to couchdb, the CPU is > always > > above 15%. > > Not sure why. > > couchdb's beam.smp is currently the busiest process on the server. > > > > [image: image.png] > > > > > > # ps aux | grep couchdb > > couchdb 979 13.4 0.1 4647432 100284 ? Ssl 15:03 2:10 > > /opt/couchdb/bin/../erts-12.2.1/bin/beam.smp -K true -A 16 -Bd -- -root > > /opt/couchdb/bin/.. -progname couchdb -- -home /opt/couchdb -- -boot > > /opt/couchdb/bin/../releases/3.2.2/couchdb -name couchdb@localhost > > -setcookie SOME_SECRET_COOKIE -kernel inet_dist_use_interface {127,0,0,1} > > -kernel error_logger silent -sasl sasl_error_logger false -noshell > -noinput > > -smp enable -ssl session_lifetime 300 -- -config > > /opt/couchdb/bin/../releases/3.2.2/sys.config -- > > couchdb 0.0 0.0 7136 220 ?S15:03 0:00 > > /opt/couchdb/bin/../erts-12.2.1/bin/epmd -daemon > > couchdb 1163 0.0 0.0 2780 972 ?Ss 15:03 0:00 > > erl_child_setup 65536 > > couchdb 2727 0.0 0.0 3736 1192 ?Ss 15:04 0:00 > > inet_gethost 4 > > couchdb 2728 0.0 0.0 3736 108 ?S15:04 0:00 > > inet_gethost 4 > > root2982 0.0 0.0 6612 2220 pts/1S+ 15:30 0:00 grep > > --color=auto couchdb > > > > > > Any hint will be appreciated. > > > > Thank you very much. > > > > Alfred. >
Re: CPU constantly above 15% even when no client is talking to CouchDB
Hi Alfred, It could be you're seeing the effect of busy waiting from Erlang VM schedulers. Erlang VM would spawn a scheduler for each CPU core, and then after any of those schedulers run out of work, they would still spin in a busy loop for a bit waiting for more work. Sometimes that is not desirable, especially in containerized environments so then it's recommended to disable busy waiting by setting +sbwt to none in the vm.args file. Give that a try and see if it helps. More info about it can be found at https://adoptingerlang.org/docs/production/kubernetes/ Another reason could be that there are some background index building or compaction jobs running.. Check out the output of _active_tasks to see if that's the case. Cheers, -Nick On Sun, Aug 14, 2022 at 11:38 AM Al Z. wrote: > > Hello. > I have the latest CouchDB 3.2.2 on Ubuntu 22.04 > > Even when there are no active clients talking to couchdb, the CPU is always > above 15%. > Not sure why. > couchdb's beam.smp is currently the busiest process on the server. > > [image: image.png] > > > # ps aux | grep couchdb > couchdb 979 13.4 0.1 4647432 100284 ? Ssl 15:03 2:10 > /opt/couchdb/bin/../erts-12.2.1/bin/beam.smp -K true -A 16 -Bd -- -root > /opt/couchdb/bin/.. -progname couchdb -- -home /opt/couchdb -- -boot > /opt/couchdb/bin/../releases/3.2.2/couchdb -name couchdb@localhost > -setcookie SOME_SECRET_COOKIE -kernel inet_dist_use_interface {127,0,0,1} > -kernel error_logger silent -sasl sasl_error_logger false -noshell -noinput > -smp enable -ssl session_lifetime 300 -- -config > /opt/couchdb/bin/../releases/3.2.2/sys.config -- > couchdb 0.0 0.0 7136 220 ?S15:03 0:00 > /opt/couchdb/bin/../erts-12.2.1/bin/epmd -daemon > couchdb 1163 0.0 0.0 2780 972 ?Ss 15:03 0:00 > erl_child_setup 65536 > couchdb 2727 0.0 0.0 3736 1192 ?Ss 15:04 0:00 > inet_gethost 4 > couchdb 2728 0.0 0.0 3736 108 ?S15:04 0:00 > inet_gethost 4 > root2982 0.0 0.0 6612 2220 pts/1S+ 15:30 0:00 grep > --color=auto couchdb > > > Any hint will be appreciated. > > Thank you very much. > > Alfred.