Hi,
I've got another bug with 100% CPU on HAProxy process, it is built from
HEAD of 1.9 branch.
One of processes stuck in infinite loop, admin socket is not responsive so
I've got information only from gdb:
0x0000000000484ab8 in h2_process_mux (h2c=0x2e8ff30) at src/mux_h2.c:2589
2589 if (h2s->send_wait->events & SUB_CALL_UNSUBSCRIBE)
(gdb) n
2587 if (h2c->st0 >= H2_CS_ERROR || h2c->flags &
H2_CF_MUX_BLOCK_ANY)
(gdb)
2586 list_for_each_entry(h2s, &h2c->send_list, list) {
(gdb)
2587 if (h2c->st0 >= H2_CS_ERROR || h2c->flags &
H2_CF_MUX_BLOCK_ANY)
(gdb)
2589 if (h2s->send_wait->events & SUB_CALL_UNSUBSCRIBE)
(gdb)
2587 if (h2c->st0 >= H2_CS_ERROR || h2c->flags &
H2_CF_MUX_BLOCK_ANY)
(gdb)
2586 list_for_each_entry(h2s, &h2c->send_list, list) {
(gdb)
2587 if (h2c->st0 >= H2_CS_ERROR || h2c->flags &
H2_CF_MUX_BLOCK_ANY)
(gdb) p h2c
$1 = (struct h2c *) 0x2e8ff30
(gdb) p *h2c
$2 = {conn = 0x2b4c900, st0 = H2_CS_FRAME_H, errcode = H2_ERR_NO_ERROR,
flags = 0, streams_limit = 100, max_id = 149, rcvd_c = 0, rcvd_s = 0, ddht
= 0x34099c0, dbuf = {size = 0, area = 0x0, data = 0, head = 0}, dsi = 149,
dfl = 0,
dft = 1 '\001', dff = 37 '%', dpl = 0 '\000', last_sid = -1, mbuf = {size
= 16384, area = 0x2ec3d50 "", data = 0, head = 0}, msi = -1, mfl = 0, mft =
0 '\000', mff = 0 '\000', miw = 6291456, mws = 15443076, mfs = 16384,
timeout = 20000, shut_timeout = 20000, nb_streams = 53, nb_cs = 53,
nb_reserved = 0, stream_cnt = 75, proxy = 0x219ffe0, task = 0x34081d0,
streams_by_id = {b = {0x2adc2e1, 0x0}}, send_list = {n = 0x2ac5b38, p =
0x3093c18},
fctl_list = {n = 0x2e90008, p = 0x2e90008}, sending_list = {n =
0x2ac5b48, p = 0x2ec2798}, buf_wait = {target = 0x0, wakeup_cb = 0x0, list
= {n = 0x2e90038, p = 0x2e90038}}, wait_event = {task = 0x2b2ae90, handle =
0x0, events = 1}}
(gdb) n
2589 if (h2s->send_wait->events & SUB_CALL_UNSUBSCRIBE)
(gdb) p *h2s
$3 = {cs = 0x297bdb0, sess = 0x819580 <pool_cache+96>, h2c = 0x2e8ff30, h1m
= {state = H1_MSG_RPBEFORE, flags = 12, curr_len = 0, body_len = 0, next =
0, err_pos = -1, err_state = 0}, by_id = {node = {branches = {b =
{0x2a72250,
0x2961c30}}, node_p = 0x2a72251, leaf_p = 0x2961c31, bit = 1, pfx
= 49017}, key = 103}, id = 103, flags = 16385, mws = 6291456, errcode =
H2_ERR_NO_ERROR, st = H2_SS_HREM, status = 0, body_len = 0, rxbuf = {size =
0,
area = 0x0, data = 0, head = 0}, wait_event = {task = 0x2fb3ee0, handle
= 0x0, events = 0}, recv_wait = 0x2b8d700, send_wait = 0x2b8d700, list = {n
= 0x3130108, p = 0x2b02238}, sending_list = {n = 0x3130118, p = 0x2b02248}}
(gdb) n
2587 if (h2c->st0 >= H2_CS_ERROR || h2c->flags &
H2_CF_MUX_BLOCK_ANY)
(gdb) p *h2c
$4 = {conn = 0x2b4c900, st0 = H2_CS_FRAME_H, errcode = H2_ERR_NO_ERROR,
flags = 0, streams_limit = 100, max_id = 149, rcvd_c = 0, rcvd_s = 0, ddht
= 0x34099c0, dbuf = {size = 0, area = 0x0, data = 0, head = 0}, dsi = 149,
dfl = 0,
dft = 1 '\001', dff = 37 '%', dpl = 0 '\000', last_sid = -1, mbuf = {size
= 16384, area = 0x2ec3d50 "", data = 0, head = 0}, msi = -1, mfl = 0, mft =
0 '\000', mff = 0 '\000', miw = 6291456, mws = 15443076, mfs = 16384,
timeout = 20000, shut_timeout = 20000, nb_streams = 53, nb_cs = 53,
nb_reserved = 0, stream_cnt = 75, proxy = 0x219ffe0, task = 0x34081d0,
streams_by_id = {b = {0x2adc2e1, 0x0}}, send_list = {n = 0x2ac5b38, p =
0x3093c18},
fctl_list = {n = 0x2e90008, p = 0x2e90008}, sending_list = {n =
0x2ac5b48, p = 0x2ec2798}, buf_wait = {target = 0x0, wakeup_cb = 0x0, list
= {n = 0x2e90038, p = 0x2e90038}}, wait_event = {task = 0x2b2ae90, handle =
0x0, events = 1}}
(gdb) n
2586 list_for_each_entry(h2s, &h2c->send_list, list) {
(gdb) n
2587 if (h2c->st0 >= H2_CS_ERROR || h2c->flags &
H2_CF_MUX_BLOCK_ANY)
(gdb) n
2589 if (h2s->send_wait->events & SUB_CALL_UNSUBSCRIBE)
HAProxy info:
HA-Proxy version 1.9.7-207ba5a 2019/05/05 - https://haproxy.org/
Build options :
TARGET = linux2628
CPU = generic
CC = gcc
CFLAGS = -O2 -g -fno-strict-aliasing -Wdeclaration-after-statement
-fwrapv -Wno-unused-label -Wno-sign-compare -Wno-unused-parameter
-Wno-old-style-declaration -Wno-ignored-qualifiers -Wno-clobbered
-Wno-missing-field-initializers -Wtype-limits -DIP_BIND_ADDRESS_NO_PORT=24
OPTIONS = USE_GETADDRINFO=1 USE_ZLIB=1 USE_REGPARM=1 USE_DL=1
USE_OPENSSL=1 USE_LUA=1 USE_PCRE=1 USE_PCRE_JIT=1
Default settings :
maxconn = 2000, bufsize = 16384, maxrewrite = 1024, maxpollevents = 200
Built with OpenSSL version : OpenSSL 1.1.1b 26 Feb 2019
Running on OpenSSL version : OpenSSL 1.1.1b 26 Feb 2019
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : TLSv1.0 TLSv1.1 TLSv1.2 TLSv1.3
Built with Lua version : Lua 5.3.5
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT
IP_FREEBIND
Built with zlib version : 1.2.8
Running on zlib version : 1.2.8
Compression algorithms supported : identity("identity"),
deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
Built with PCRE version : 8.43 2019-02-23
Running on PCRE version : 8.43 2019-02-23
PCRE library supports JIT : yes
Encrypted password support via crypt(3): yes
Built with multi-threading support.
Available polling systems :
epoll : pref=300, test result OK
poll : pref=200, test result OK
select : pref=150, test result OK
Total: 3 (3 usable), will use epoll.
Available multiplexer protocols :
(protocols marked as <default> cannot be specified using 'proto' keyword)
h2 : mode=HTX side=FE|BE
h2 : mode=HTTP side=FE
<default> : mode=HTX side=FE|BE
<default> : mode=TCP|HTTP side=FE|BE
Available filters :
[SPOE] spoe
[COMP] compression
[CACHE] cache
[TRACE] trace
