Re: segfault error 6 in haproxy=1.8-3 (pendconn_grab_from_px)

[Максим Куприянов]

Hello again!

Today I have also found another segfaults which occurred under heavy load
with nbthreads>1 on two haproxy-instances:
segfault at 0040 ip 5566f9ceb3af sp 7f98d0d569f8 error
7 in haproxy[5566f9be+169000]
and
segfault at fffe0040 ip 55bbf359d3af sp 7ffe59ed65b8 error
7 in haproxy[55bbf3492000+169000]
Both lead us to init_task:
$ /usr/bin/addr2line -e /usr/sbin/haproxy -fCi 0x10b3af
init_task
??:?

2018-01-17 13:12 GMT+03:00 Максим Куприянов :

> Hi!
>
> I have multiple instances of haproxy=1.8-3 running with nbthreads=2 and
> more. Those instances sometimes fail with an error like that in dmesg:
> haproxy[22287]: segfault at 5574915aab8a ip 55828ce3adea sp
> 7ffc105c8fb0 error 6 in haproxy[55828cd2f000+169000]
> Instances with no nbthreads-parameter set are not affected.
>
> addr2line gave a tip to the falling function:
> /usr/bin/addr2line -e /usr/sbin/haproxy -fCi 0x10bdea
> pendconn_grab_from_px
> ??:?
>
> Version of haproxy:
> haproxy -vv
> HA-Proxy version 1.8.3-1 2018/01/09
> Copyright 2000-2017 Willy Tarreau 
>
> Build options :
>   TARGET  = linux2628
>   CPU = generic
>   CC  = gcc
>   CFLAGS  = -g -O2 -fPIE -fstack-protector --param=ssp-buffer-size=4
> -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2
>   OPTIONS = USE_GETADDRINFO=1 USE_ZLIB=1 USE_REGPARM=1 USE_THREAD=1
> USE_OPENSSL=1 USE_LUA=1 USE_PCRE=1 USE_PCRE_JIT=1 USE_TFO=1 USE_NS=1
>
> Default settings :
>   maxconn = 2000, bufsize = 16384, maxrewrite = 1024, maxpollevents = 200
>
> Built with OpenSSL version : OpenSSL 1.0.1f 6 Jan 2014
> Running on OpenSSL version : OpenSSL 1.0.1f 6 Jan 2014
> OpenSSL library supports TLS extensions : yes
> OpenSSL library supports SNI : yes
> OpenSSL library supports : SSLv3 TLSv1.0 TLSv1.1 TLSv1.2
> Built with Lua version : Lua 5.3.1
> Built with transparent proxy support using: IP_TRANSPARENT
> IPV6_TRANSPARENT IP_FREEBIND
> Encrypted password support via crypt(3): yes
> Built with multi-threading support.
> Built with PCRE version : 8.31 2012-07-06
> Running on PCRE version : 8.31 2012-07-06
> PCRE library supports JIT : no (libpcre build without JIT?)
> Built with zlib version : 1.2.8
> Running on zlib version : 1.2.8
> Compression algorithms supported : identity("identity"),
> deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
> Built with network namespace support.
>
> Available polling systems :
>   epoll : pref=300,  test result OK
>poll : pref=200,  test result OK
>  select : pref=150,  test result OK
> Total: 3 (3 usable), will use epoll.
>
> Available filters :
> [SPOE] spoe
> [COMP] compression
> [TRACE] trace
>
>
> --
> Best regards,
> Maksim Kupriianov
>

segfault error 6 in haproxy=1.8-3 (pendconn_grab_from_px)

[Максим Куприянов]

Hi!

I have multiple instances of haproxy=1.8-3 running with nbthreads=2 and
more. Those instances sometimes fail with an error like that in dmesg:
haproxy[22287]: segfault at 5574915aab8a ip 55828ce3adea sp
7ffc105c8fb0 error 6 in haproxy[55828cd2f000+169000]
Instances with no nbthreads-parameter set are not affected.

addr2line gave a tip to the falling function:
/usr/bin/addr2line -e /usr/sbin/haproxy -fCi 0x10bdea
pendconn_grab_from_px
??:?

Version of haproxy:
haproxy -vv
HA-Proxy version 1.8.3-1 2018/01/09
Copyright 2000-2017 Willy Tarreau 

Build options :
  TARGET  = linux2628
  CPU = generic
  CC  = gcc
  CFLAGS  = -g -O2 -fPIE -fstack-protector --param=ssp-buffer-size=4
-Wformat -Werror=format-security -D_FORTIFY_SOURCE=2
  OPTIONS = USE_GETADDRINFO=1 USE_ZLIB=1 USE_REGPARM=1 USE_THREAD=1
USE_OPENSSL=1 USE_LUA=1 USE_PCRE=1 USE_PCRE_JIT=1 USE_TFO=1 USE_NS=1

Default settings :
  maxconn = 2000, bufsize = 16384, maxrewrite = 1024, maxpollevents = 200

Built with OpenSSL version : OpenSSL 1.0.1f 6 Jan 2014
Running on OpenSSL version : OpenSSL 1.0.1f 6 Jan 2014
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : SSLv3 TLSv1.0 TLSv1.1 TLSv1.2
Built with Lua version : Lua 5.3.1
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT
IP_FREEBIND
Encrypted password support via crypt(3): yes
Built with multi-threading support.
Built with PCRE version : 8.31 2012-07-06
Running on PCRE version : 8.31 2012-07-06
PCRE library supports JIT : no (libpcre build without JIT?)
Built with zlib version : 1.2.8
Running on zlib version : 1.2.8
Compression algorithms supported : identity("identity"),
deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
Built with network namespace support.

Available polling systems :
  epoll : pref=300,  test result OK
   poll : pref=200,  test result OK
 select : pref=150,  test result OK
Total: 3 (3 usable), will use epoll.

Available filters :
[SPOE] spoe
[COMP] compression
[TRACE] trace


--
Best regards,
Maksim Kupriianov