Re: [1.9.6] One of haproxy processes using 100% CPU

2019-04-05 Thread Olivier Houchard
Hi Maciej, On Fri, Apr 05, 2019 at 01:33:29PM +0200, Maciej Zdeb wrote: > I think I found something, please look at the session 0x2110edc0 and > "calls", it never ends: > > socat /run/haproxy/haproxy2.sock - <<< "show sess all" > 0x2110edc0: [05/Apr/2019:12:03:32.141927] id=14505 proto=tcpv4 > so

Re: [1.9.6] One of haproxy processes using 100% CPU

2019-04-05 Thread Maciej Zdeb
I think I found something, please look at the session 0x2110edc0 and "calls", it never ends: socat /run/haproxy/haproxy2.sock - <<< "show sess all" 0x2110edc0: [05/Apr/2019:12:03:32.141927] id=14505 proto=tcpv4 source=S.S.S.S:52414 flags=0x4504e, conn_retries=3, srv_conn=0xe2e7fb0, pend_pos=(nil

Re: [1.9.6] One of haproxy processes using 100% CPU

2019-04-05 Thread Maciej Zdeb
After process spin up to 100% cpu, I switched traffic to another (backup) server then I attached gdb and executed "generate-core-file". However I don't know how to proceed with it further (it's quite big 3,2GB). When I'm attached with gdb sometimes (not always) "bt full" shows this (no idea if it's

Re: [1.9.6] One of haproxy processes using 100% CPU

2019-04-05 Thread Maciej Zdeb
Hi again, I verified that local DoS did not cause that high cpu usage. I'm upgrading from 1.8.19 to 1.9.6. Please look at strace: of process with 100% cpu usage: strace -fp 9790 -c Process 9790 attached % time seconds usecs/call callserrors syscall -- --- --- ---

Re: [1.9.6] One of haproxy processes using 100% CPU

2019-04-04 Thread Maciej Zdeb
I'm very sorry, it is probably "real" traffic that haproxy needs to handle. Some local DDoS occurred just in time when I upgraded to 1.9 (what terribly bad luck). Please ignore previous email! Sorry for wasting your time. czw., 4 kwi 2019 o 14:36 Maciej Zdeb napisaƂ(a): > Hi, > > After haproxy

[1.9.6] One of haproxy processes using 100% CPU

2019-04-04 Thread Maciej Zdeb
Hi, After haproxy starts everything is fine, but after some time one of haproxy processes uses 100% of CPU. I've managed to dump some "show" commands (I have also "show fd", "show sess all" and "strace" but if needed I would prefer to send it privately). If you could give any advice howto debug it