Hello Simon, or anyone else affected, Accepted haproxy into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/haproxy/1.8.8-1ubuntu0.11 in a few hours, and then in the -proposed repository.
Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users. If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed- bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification- failed-bionic. In either case, without details of your testing we will not be able to proceed. Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping! N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days. -- You received this bug notification because you are a member of Ubuntu High Availability Team, which is subscribed to haproxy in Ubuntu. https://bugs.launchpad.net/bugs/1884149 Title: haproxy crashes on in __pool_get_first if unique-id-header is used Status in HAProxy: Fix Released Status in haproxy package in Ubuntu: Fix Released Status in haproxy source package in Bionic: Fix Committed Status in haproxy package in Debian: Fix Released Bug description: [Impact] * The handling of locks in haproxy led to a state that between idle http connections one could have indicated a connection was destroyed. In that case the code went on and accessed a just freed resource. As upstream puts it "It can have random implications between requests as it may lead a wrong connection's polling to be re-enabled or disabled for example, especially with threads." * Backport the fix from upstreams 1.8 stable branch [Test Case] * It is a race and might be hard to trigger. An haproxy config to be in front of three webservers can be seen below. Setting up three apaches locally didn't trigger the same bug, but we know it is timing sensitive. * Simon (anbox) has a setup which reliably triggers this and will run the tests there. * The bad case will trigger a crash as reported below. [Regression Potential] * This change is in >=Disco and has no further bugs reported against it (no follow on change) which should make it rather safe. Also no other change to that file context in 1.8 stable since then. The change is on the locking of connections. So if we want to expect regressions, then they would be at the handling of concurrent connections. [Other Info] * Strictly speaking it is a race, so triggering it depends on load and machine cpu/IO speed. --- Version 1.8.8-1ubuntu0.10 of haproxy in Ubuntu 18.04 (bionic) crashes with ------------------------------------ Thread 2.1 "haproxy" received signal SIGSEGV, Segmentation fault. [Switching to Thread 0xfffff77b1010 (LWP 17174)] __pool_get_first (pool=0xaaaaaac6ddd0, pool=0xaaaaaac6ddd0) at include/common/memory.h:124 124 include/common/memory.h: No such file or directory. (gdb) bt #0 __pool_get_first (pool=0xaaaaaac6ddd0, pool=0xaaaaaac6ddd0) at include/common/memory.h:124 #1 pool_alloc_dirty (pool=0xaaaaaac6ddd0) at include/common/memory.h:154 #2 pool_alloc (pool=0xaaaaaac6ddd0) at include/common/memory.h:229 #3 conn_new () at include/proto/connection.h:655 #4 cs_new (conn=0x0) at include/proto/connection.h:683 #5 connect_conn_chk (t=0xaaaaaacb8820) at src/checks.c:1553 #6 process_chk_conn (t=0xaaaaaacb8820) at src/checks.c:2135 #7 process_chk (t=0xaaaaaacb8820) at src/checks.c:2281 #8 0x0000aaaaaabca0b4 in process_runnable_tasks () at src/task.c:231 #9 0x0000aaaaaab76f44 in run_poll_loop () at src/haproxy.c:2399 #10 run_thread_poll_loop (data=<optimized out>) at src/haproxy.c:2461 #11 0x0000aaaaaaad79ec in main (argc=<optimized out>, argv=0xaaaaaac61b30) at src/haproxy.c:3050 ------------------------------------ when running on an ARM64 system. The haproxy.cfg looks like this: ------------------------------------ global log /dev/log local0 log /dev/log local1 notice maxconn 4096 user haproxy group haproxy spread-checks 0 tune.ssl.default-dh-param 1024 ssl-default-bind-ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:!DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:!DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:!CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA defaults log global mode tcp option httplog option dontlognull retries 3 timeout queue 20000 timeout client 50000 timeout connect 5000 timeout server 50000 frontend anbox-stream-gateway-lb-5-80 bind 0.0.0.0:80 default_backend api_http mode http http-request redirect scheme https backend api_http mode http frontend anbox-stream-gateway-lb-5-443 bind 0.0.0.0:443 ssl crt /var/lib/haproxy/default.pem no-sslv3 default_backend app-anbox-stream-gateway mode http backend app-anbox-stream-gateway mode http balance leastconn server anbox-stream-gateway-0-4000 10.212.218.61:4000 check ssl verify none inter 2000 rise 2 fall 5 maxconn 4096 server anbox-stream-gateway-1-4000 10.212.218.93:4000 check ssl verify none inter 2000 rise 2 fall 5 maxconn 4096 server anbox-stream-gateway-2-4000 10.212.218.144:4000 check ssl verify none inter 2000 rise 2 fall 5 maxconn 4096 ------------------------------------ The crash occurs after a first few HTTP requests going through and happens again when systemd restarts the service. The bug is already reported in Debian https://bugs.debian.org/cgi- bin/bugreport.cgi?bug=921981 and upstream at https://github.com/haproxy/haproxy/issues/40 Using the 1.8.19-1+deb10u2 package from Debian fixes the crash. To manage notifications about this bug go to: https://bugs.launchpad.net/haproxy/+bug/1884149/+subscriptions _______________________________________________ Mailing list: https://launchpad.net/~ubuntu-ha Post to : [email protected] Unsubscribe : https://launchpad.net/~ubuntu-ha More help : https://help.launchpad.net/ListHelp
