Hello,

Using haproxy 1.8.16-5c3f237, for unknown reason, one of the process uses all 
the available cores.

   PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
 25120 haproxy   20   0  813808 212372   7268 R 799.3  0.3   1222:23 haproxy
 26653 haproxy   20   0  814916 214880   6044 S   1.7  0.3   2:12.09 haproxy

As of now, the process is still running, haven't killed it for the sake of 
debugging the issue.

With the debug info below, do you have some ideas on why/how this occurs?

Thx and regards,

-- Basha


Using gcore, I've managed to get a core dump

$ gdb /usr/sbin/haproxy core.25120
GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-100.el7_4.1
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /usr/sbin/haproxy...done.
[New LWP 25121]
[New LWP 25122]
[New LWP 25123]
[New LWP 25124]
[New LWP 25125]
[New LWP 25126]
[New LWP 25127]
[New LWP 25120]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `/usr/sbin/haproxy'.
#0  0x000055ae8aa6ee90 in _do_poll (p=<optimized out>, exp=<optimized out>) at 
src/ev_epoll.c:137
137     src/ev_epoll.c: No such file or directory.
Missing separate debuginfos, use: debuginfo-install 
haproxy-1.8.16-criteo_1.el7.2.x86_64
(gdb) info frame
Stack level 0, frame at 0x7fc5cbe91110:
 rip = 0x55ae8aa6ee90 in _do_poll (src/ev_epoll.c:137); saved rip 0x55ae8aaf2b9b
 called by frame at 0x7fc5cbe91160
 source language c.
 Arglist at 0x7fc5cbe91088, args: p=<optimized out>, exp=<optimized out>
 Locals at 0x7fc5cbe91088, Previous frame's sp is 0x7fc5cbe91110
 Saved registers:
  rbx at 0x7fc5cbe910d8, rbp at 0x7fc5cbe910e0, r12 at 0x7fc5cbe910e8, r13 at 
0x7fc5cbe910f0, r14 at 0x7fc5cbe910f8, r15 at 0x7fc5cbe91100, rip at 
0x7fc5cbe91108
(gdb) bt
#0  0x000055ae8aa6ee90 in _do_poll (p=<optimized out>, exp=<optimized out>) at 
src/ev_epoll.c:137
#1  0x000055ae8aaf2b9b in run_poll_loop () at src/haproxy.c:2444
#2  run_thread_poll_loop (data=<optimized out>) at src/haproxy.c:2481
#3  0x00007fc5d1d6be25 in start_thread () from /lib64/libpthread.so.0
#4  0x00007fc5d0ee634d in clone () from /lib64/libc.so.6

(gdb) x/100x $sp
0x7fc5cbe91090: 0x00000010      0x00000000      0xfffffee0      0xffffffff
0x7fc5cbe910a0: 0x00000004      0x00000001      0x00000004      0x00000001
0x7fc5cbe910b0: 0x32305ab9      0x00007fc5      0x8adb3da0      0x000055ae
0x7fc5cbe910c0: 0x8adb7dc0      0x000055ae      0x8ab40894      0x000055ae
0x7fc5cbe910d0: 0x8ada6650      0x000055ae      0xfffffee0      0xffffffff
0x7fc5cbe910e0: 0xffffe400      0xffffffff      0x8adb63c0      0x000055ae
0x7fc5cbe910f0: 0x8adb7e20      0x000055ae      0x8adb7578      0x000055ae
0x7fc5cbe91100: 0xffffffec      0xffffffff      0x8aaf2b9b      0x000055ae
0x7fc5cbe91110: 0x00000000      0x00000000      0x00000000      0x00000000
0x7fc5cbe91120: 0x00000000      0x00000000      0x00000000      0x00000000
0x7fc5cbe91130: 0x00000000      0x00000000      0x00000000      0x00000000
0x7fc5cbe91140: 0xcbe939c0      0x00007fc5      0xcbe93700      0x00007fc5
0x7fc5cbe91150: 0x00000000      0x00000000      0xd1d6be25      0x00007fc5
0x7fc5cbe91160: 0x00000000      0x00000000      0xcbe93700      0x00007fc5
0x7fc5cbe91170: 0xcbe93700      0x00007fc5      0x69969903      0x6a98e5ca
0x7fc5cbe91180: 0x00000000      0x00000000      0xcbe939c0      0x00007fc5
0x7fc5cbe91190: 0xcbe93700      0x00007fc5      0x00000000      0x00000000
0x7fc5cbe911a0: 0x4b569903      0x95137218      0x124c9903      0x95134667
0x7fc5cbe911b0: 0x00000000      0x00000000      0x00000000      0x00000000
0x7fc5cbe911c0: 0x00000000      0x00000000      0x00000000      0x00000000
0x7fc5cbe911d0: 0x00000000      0x00000000      0x00000000      0x00000000
0x7fc5cbe911e0: 0x00000000      0x00000000      0x00000000      0x00000000
0x7fc5cbe911f0: 0xcbe93700      0x00007fc5      0xd0ee634d      0x00007fc5
0x7fc5cbe91200: 0x00000000      0x00000000      0x00000000      0x00000000
0x7fc5cbe91210: 0x00000000      0x00000000      0x00000000      0x00000000



Version information:

$ haproxy -vv                                                                   
                                                  [0/36516]
HA-Proxy version 1.8.16-5c3f237 2018/12/21
Copyright 2000-2018 Willy Tarreau <wi...@haproxy.org>

Build options :
  TARGET  = linux2628
  CPU     = generic
  CC      = gcc
  CFLAGS  = -O2 -g -fno-strict-aliasing -Wdeclaration-after-statement -fwrapv 
-Wno-unused-label -DTCP_USER_TIMEOUT=18
  OPTIONS = USE_LINUX_TPROXY=1 USE_GETADDRINFO=1 USE_ZLIB=1 USE_REGPARM=1 
USE_OPENSSL=1 USE_SYSTEMD=1 USE_PCRE=1 USE_PCRE_JIT=1 USE_TFO=1 USE_NS=1

Default settings :
  maxconn = 2000, bufsize = 16384, maxrewrite = 1024, maxpollevents = 200

Built with OpenSSL version : OpenSSL 1.1.1a  20 Nov 2018
Running on OpenSSL version : OpenSSL 1.1.1  11 Sep 2018
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : TLSv1.0 TLSv1.1 TLSv1.2 TLSv1.3
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT 
IP_FREEBIND
Encrypted password support via crypt(3): yes
Built with multi-threading support.
Built with PCRE version : 8.32 2012-11-30
Running on PCRE version : 8.32 2012-11-30
PCRE library supports JIT : yes
Built with zlib version : 1.2.7
Running on zlib version : 1.2.7
Compression algorithms supported : identity("identity"), deflate("deflate"), 
raw-deflate("deflate"), gzip("gzip")
Built with network namespace support.

Available polling systems :
      epoll : pref=300,  test result OK
       poll : pref=200,  test result OK
     select : pref=150,  test result OK
Total: 3 (3 usable), will use epoll.

Available filters :
        [SPOE] spoe
        [COMP] compression
        [TRACE] trace

Reply via email to