Hi! I have an issue which I've don't know how to go ahead with. So first I will describe my setup. We have a pair of haproxy v2.0.5 that only takes HTTP traffic and relays HTTPS traffic down to a pair of haproxys v1.7.11 doing SSL termination. Here is a more "graphical" layout: Internet =HTTP+HTTPS=> haproxy (v2.0.5, no-TLS) haproxy (v2.0.5, no-TLS) =HTTP=> backend applications haproxy (v2.0.5, no-TLS) =TCP/HTTPS=> haproxy (v1.7.11, TLS) haproxy (v1.7.11, TLS) =HTTP=> backend applications
So the problem here is that this setup breaks when we try to upgrade the haproxy (v1.7.11, TLS) to v2.0.5. What happens is that it simply masses up established connections(`ss -s`) until the internet facing haproxy runs out of resources and uses all cores is 100% utilizied, gets too many orphaned connections and OOM killer starts killing pids. So in terms of hardware usage this is going from ~60% CPU utilization to 100% on 18 cores(HT turned off), 2 GB RAM usage to 32GB+swap. So we do zero config changes, upgrade haproxy to 2.0.x + restart haproxy and like a minute or so then it runs out of resources. Each haproxy (v2.0.5, no-TLS) have an request rate of 55-90K/s. Each haproxy (v1.7.11, TLS) have an request rate of 15-20/s. Each haproxy (v2.0.5, no-TLS) have a connection rate of 7-12K/s. Each haproxy (v1.7.11, TLS) has a connection rate of 6-7K/s. I have no clue why a zero config change upgrade would break this setup. Anyone that can help me go forward with troubleshooting this or explain what might cause it to mass up established connections? Any help appreciated, Elias
