Hello Robin, If there is production traffic on the node, it is possible that there are multiple haproxy instances still handling requests and until they finish serving requests (and the haproxy process finishes execution) the memory cannot be freed.
To avoid this problem, I would highly recommend setting an appropriate hard-stop-after value in the global section of your configuration that should be tuned to the frequency of the reloads (higher frequency of reloads = lower hard-stop-after value). However, a too low value should be avoided as it will cause abnormal termination for long-running sessions. See https://www.haproxy.com/documentation/hapee/1-8r2/onepage/#hard-stop-after for more information. Can you check if there is more than two haproxy processes running on the instance you are having issues with? The expected memory usage will highly depend on how long the old instances are running - values north of 2 x the size of the cache are prudent, as they should avoid the behavior above. Also, other HAProxy internal structures also consume resources: stick tables, the ssl cache, the http buffers associated with each session all consume memory. I would highly recommend checking out the performance section of the configuration manual to help with capacity planning. https://www.haproxy.com/documentation/hapee/1-8r2/onepage/#3.2 Hope this helps. Regards, Bruno Henc ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Wednesday, March 27, 2019 8:39 PM, Robin Björklin <[email protected]> wrote: > Hi, > > I've tried using the new haproxy cache with "total-max-size 4095" and > "nbthread 4" on a machine with 12GB of RAM and I'm getting hit with "[ALERT] > 084/222221 (1) : Unable to allocate cache." after a couple of reloads. > > HAProxy is started by running: /usr/bin/podman run --rm --name haproxy > --network=host -v /opt/haproxy/conf:/usr/local/etc/haproxy:ro > haproxy:1.9-alpine -W -db -f /usr/local/etc/haproxy > > The configuration is being reloaded by running: /usr/bin/podman kill --signal > USR2 haproxy > > What's the expected memory usage when reloading? 2x the size of the cache or > even more? > > Best regards, > Robin Bjorklin
