On 05/13/2017 01:14 AM, Grant Zhang wrote: > >> On May 10, 2017, at 04:51, Emeric Brun <[email protected]> wrote: >> >>> It looks like the main process stalls at DH_free(local_dh_1024) (part of >>> __ssl_sock_deinit). Not sure why but I will debug and report back. >>> >>> Thanks, >> >> I experienced the same issue (stalled on a futex) if i run haproxy in >> foreground and trying to kill it with kill -USR1. >> >> With this conf (dh param and ssl-async are disabled) >> global >> # tune.ssl.default-dh-param 2048 >> ssl-engine qat >> # ssl-async >> nbproc 1 > > It looks like that the stall on futex issue is related to DH_free() calling > ENGINE_finish in openssl 1.1: > https://github.com/openssl/openssl/blob/master/crypto/dh/dh_lib.c#L109 > (gdb) bt > #0 __lll_lock_wait () at > ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:132 > #1 0x00007fa1582c5571 in pthread_rwlock_wrlock () > at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_rwlock_wrlock.S:118 > #2 0x00007fa158a58559 in CRYPTO_THREAD_write_lock () from > /tmp/openssl_1.1.0_install/lib/libcrypto.so.1.1 > #3 0x00007fa1589d8800 in ENGINE_finish () from > /tmp/openssl_1.1.0_install/lib/libcrypto.so.1.1 > #4 0x00007fa158975e76 in DH_free () from > /tmp/openssl_1.1.0_install/lib/libcrypto.so.1.1 > #5 0x0000000000417c78 in free_dh () at src/ssl_sock.c:7905 > #6 0x00007fa1591d58ce in _dl_fini () at dl-fini.c:254 > #7 0x00007fa158512511 in __run_exit_handlers (status=0, > listp=0x7fa15888e688, run_list_atexit=true) > at exit.c:78 > #8 0x00007fa158512595 in __GI_exit (status=<optimized out>) at exit.c:100 > #9 0x0000000000408814 in main (argc=4, argv=0x7ffe72188548) at > src/haproxy.c:2235 > > Openssl 1.1 has changed the way ENGINE_cleanup works: > https://www.openssl.org/docs/man1.1.0/crypto/ENGINE_cleanup.html > "From OpenSSL 1.1.0 it is no longer necessary to explicitly call > ENGINE_cleanup and this function is deprecated. Cleanup automatically takes > place at program exit." > > I suppose by the time the destructor __ssl_sock_deinit is called, > engine-related cleanup are already done by openssl and ENGINE_finish (from > DH_free) stalls on a non-existing write lock. > > I have a workaround which moves the DH_free logic out of the destructor > __ssl_sock_deinit, and right before process exit. With the workaround I no > longer see the stall issue. I am not sure whether it is optimal solution > though. Let me know.
What does it look like? The issue is very similar: https://mta.openssl.org/pipermail/openssl-dev/2016-March/005909.html > > Thanks, > > Grant R, Emeric
