I'm very sorry, it is probably "real" traffic that haproxy needs to handle. Some local DDoS occurred just in time when I upgraded to 1.9 (what terribly bad luck).
Please ignore previous email! Sorry for wasting your time. czw., 4 kwi 2019 o 14:36 Maciej Zdeb <[email protected]> napisał(a): > Hi, > > After haproxy starts everything is fine, but after some time one of > haproxy processes uses 100% of CPU. I've managed to dump some "show" > commands (I have also "show fd", "show sess all" and "strace" but if needed > I would prefer to send it privately). If you could give any advice howto > debug it further it would be great. > > ps aux | grep haprox > haproxy 99023 97.5 0.6 3139384 623476 ? Rs 14:07 16:38 > /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p /var/run/haproxy.pid -D > -sf 15434 15443 15444 15445 15454 15455 15456 15457 > haproxy 99024 13.0 0.6 3138964 623096 ? Ss 14:07 2:13 > /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p /var/run/haproxy.pid -D > -sf 15434 15443 15444 15445 15454 15455 15456 15457 > haproxy 99025 14.1 0.6 3139044 623160 ? Rs 14:07 2:25 > /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p /var/run/haproxy.pid -D > -sf 15434 15443 15444 15445 15454 15455 15456 15457 > haproxy 99034 11.9 0.6 3139024 623496 ? Ss 14:07 2:02 > /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p /var/run/haproxy.pid -D > -sf 15434 15443 15444 15445 15454 15455 15456 15457 > haproxy 99035 16.3 0.6 3139184 623116 ? Ss 14:07 2:47 > /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p /var/run/haproxy.pid -D > -sf 15434 15443 15444 15445 15454 15455 15456 15457 > haproxy 99037 13.0 0.6 3138724 623076 ? Ss 14:07 2:13 > /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p /var/run/haproxy.pid -D > -sf 15434 15443 15444 15445 15454 15455 15456 15457 > haproxy 99045 16.3 0.6 3139332 623216 ? Ss 14:07 2:47 > /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p /var/run/haproxy.pid -D > -sf 15434 15443 15444 15445 15454 15455 15456 15457 > haproxy 99046 12.0 0.6 3137792 622208 ? Rs 14:07 2:03 > /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p /var/run/haproxy.pid -D > -sf 15434 15443 15444 15445 15454 15455 15456 15457 > > socat /var/run/haproxy/haproxy1.sock - <<< "show info" > Name: HAProxy > Version: 1.9.6 > Release_date: 2019/03/29 > Nbthread: 1 > Nbproc: 8 > Process_num: 1 > Pid: 99023 > Uptime: 0d 0h12m26s > Uptime_sec: 746 > Memmax_MB: 0 > PoolAlloc_MB: 8 > PoolUsed_MB: 8 > PoolFailed: 0 > Ulimit-n: 1000000 > Maxsock: 4012387 > Maxconn: 2000000 > Hard_maxconn: 2000000 > CurrConns: 850 > CumConns: 114038 > CumReq: 179023 > MaxSslConns: 0 > CurrSslConns: 4 > CumSslConns: 6225 > Maxpipes: 0 > PipesUsed: 0 > PipesFree: 0 > ConnRate: 161 > ConnRateLimit: 0 > MaxConnRate: 366 > SessRate: 161 > SessRateLimit: 0 > MaxSessRate: 366 > SslRate: 6 > SslRateLimit: 0 > MaxSslRate: 18 > SslFrontendKeyRate: 3 > SslFrontendMaxKeyRate: 7 > SslFrontendSessionReuse_pct: 50 > SslBackendKeyRate: 0 > SslBackendMaxKeyRate: 10 > SslCacheLookups: 1054 > SslCacheMisses: 103 > CompressBpsIn: 0 > CompressBpsOut: 0 > CompressBpsRateLim: 0 > ZlibMemUsage: 0 > MaxZlibMemUsage: 0 > Tasks: 26283 > Run_queue: 3 > Idle_pct: 39 > node: XXXX > Stopping: 0 > Jobs: 7181 > Unstoppable Jobs: 0 > Listeners: 6330 > ActivePeers: 0 > ConnectedPeers: 0 > DroppedLogs: 6 > BusyPolling: 0 > > socat /var/run/haproxy/haproxy1.sock - <<< "show activity" > thread_id: 0 > date_now: 1554380334.734878 > loops: 355064374 > wake_cache: 2498224 > wake_tasks: 352539647 > wake_signal: 0 > poll_exp: 355037871 > poll_drop: 194560 > poll_dead: 0 > poll_skip: 0 > fd_skip: 0 > fd_lock: 0 > fd_del: 0 > conn_dead: 0 > stream: 710586336 > empty_rq: 15313 > long_rq: 57 > cpust_ms_tot: 747 > cpust_ms_1s: 0 > cpust_ms_15s: 0 > avg_loop_us: 1 > > haproxy -vv > HA-Proxy version 1.9.6 2019/03/29 - https://haproxy.org/ > Build options : > TARGET = linux2628 > CPU = generic > CC = gcc > CFLAGS = -O2 -g -fno-strict-aliasing -Wdeclaration-after-statement > -fwrapv -Wno-unused-label -Wno-sign-compare -Wno-unused-parameter > -Wno-old-style-declaration -Wno-ignored-qualifiers -Wno-clobbered > -Wno-missing-field-initializers -Wtype-limits -DIP_BIND_ADDRESS_NO_PORT=24 > OPTIONS = USE_GETADDRINFO=1 USE_ZLIB=1 USE_DL=1 USE_OPENSSL=1 USE_LUA=1 > USE_PCRE=1 USE_PCRE_JIT=1 > > Default settings : > maxconn = 2000, bufsize = 16384, maxrewrite = 1024, maxpollevents = 200 > > Built with OpenSSL version : OpenSSL 1.1.1b 26 Feb 2019 > Running on OpenSSL version : OpenSSL 1.1.1b 26 Feb 2019 > OpenSSL library supports TLS extensions : yes > OpenSSL library supports SNI : yes > OpenSSL library supports : TLSv1.0 TLSv1.1 TLSv1.2 TLSv1.3 > Built with Lua version : Lua 5.3.5 > Built with transparent proxy support using: IP_TRANSPARENT > IPV6_TRANSPARENT IP_FREEBIND > Built with zlib version : 1.2.8 > Running on zlib version : 1.2.8 > Compression algorithms supported : identity("identity"), > deflate("deflate"), raw-deflate("deflate"), gzip("gzip") > Built with PCRE version : 8.42 2018-03-20 > Running on PCRE version : 8.42 2018-03-20 > PCRE library supports JIT : yes > Encrypted password support via crypt(3): yes > Built with multi-threading support. > > Available polling systems : > epoll : pref=300, test result OK > poll : pref=200, test result OK > select : pref=150, test result OK > Total: 3 (3 usable), will use epoll. > > Available multiplexer protocols : > (protocols marked as <default> cannot be specified using 'proto' keyword) > h2 : mode=HTX side=FE|BE > h2 : mode=HTTP side=FE > <default> : mode=HTX side=FE|BE > <default> : mode=TCP|HTTP side=FE|BE > > Available filters : > [SPOE] spoe > [COMP] compression > [CACHE] cache > [TRACE] trace >
