Re: haproxy stopped balancing after about 2 weeks

2019-05-10 Thread Willy Tarreau
Hello,

On Thu, May 09, 2019 at 11:42:54AM -0600, ericr wrote:
> A couple of weeks ago I installed haproxy on our server running FreeBSD
> 11.0-RELEASE-p16. (yes, I know it's an old version of the OS, I'm going to
> upgrade it as soon as I solve my haproxy problem.)

Can you tell us what exact version you're running ? Please send the output
of "haproxy -vv".

> Haproxy is supposed to load balance between 2 web servers running apache.
> haproxy ran fine and balanced well for about 2 weeks, and then it stopped
> sending client connections to the second web server.

But it still works for the first one ?

> It still does health checks to both servers just fine, and reports L7OK/200
> at every check for both servers. I've tried using both roundrobin and
> leastconn, with no luck.  I've restarted haproxy several times, and
> rebooted the server it's running on, and it the behavior doesn't change.

Did you notice if it's always after the exact same amount of time ? Or
maybe after a certain number of requests ? We could have imagined a bug
with one LB algo but if it does it regardless of the algo this rules it
out.

Oh wait a minute :

> # info about backend servers
> backend web_servers
> balance leastconn
> cookie phpsessid insert indirect nocache
> option httpchk HEAD /
> 
> default-server check maxconn 2048
> 
> server vi-www3 10.3.3.10:8080 cookie phpsessid inter 120s
> server vi-www4 10.3.3.11:8080 cookie phpsessid inter 120s

So for both servers you're setting a response cookie "phpsessid=phpsessid"
which has the effect that all your visitors will come back with this cookie
and that the first server which matches this value will take it, hence the
first server. First, I recommend against naming your stickiness cookies
"phpsessid" as it makes one think about the application's cookie which it
is not. Second, you need to use different cookie values here, for example
"cookie w3" and "cookie w4" for your two respective servers.

Lat recommendation, I don't know if it's on purpose that you check your
servers only once every two minutes, because it's extremely slow and will
take a very long time to detect a failure. Unless you're facing a specific
limitation, you should significantly shorten this interval to just a few
seconds.

Regards,
Willy



haproxy stopped balancing after about 2 weeks

2019-05-09 Thread ericr
A couple of weeks ago I installed haproxy on our server running FreeBSD
11.0-RELEASE-p16. (yes, I know it's an old version of the OS, I'm going to
upgrade it as soon as I solve my haproxy problem.)

Haproxy is supposed to load balance between 2 web servers running apache.
haproxy ran fine and balanced well for about 2 weeks, and then it stopped
sending client connections to the second web server.

It still does health checks to both servers just fine, and reports L7OK/200
at every check for both servers. I've tried using both roundrobin and
leastconn, with no luck.  I've restarted haproxy several times, and
rebooted the server it's running on, and it the behavior doesn't change.
I'm out of ideas, does anyone have suggestions for fixing this (or
improving my config in general)?

Here's my config file:


# global holds defaults, global variables, etc.
global
daemon
user haproxy
group haproxy
log /dev/log local0
stats socket /var/run/haproxy/admin.sock user haproxy group haproxy
mode 660 level admin

# https://www.haproxy.com/blog/multithreading-in-haproxy/
maxconn 2048 # max connections we handle at once
nbproc 1 # number of haproxy processes to start
nbthread 4 # max threads, 1 per CPU core

# cpu map = number of cpu cores
cpu-map all 0-3

ssl-default-bind-ciphers "EECDH+ECDSA+AESGCM ECDH+aRSA+AESGCM
EECDH+ECDSA+SHA384 EECDH+ECDSA+SHA256 EECDH+aRSA+SHA384 EECDH+aRSA+SHA256
EECDH+aRSA+RC4 EECDH EDH+aRSA RC4"
ssl-default-bind-options ssl-min-ver TLSv1.2

defaults
timeout connect 30s
timeout client 600s
timeout server 30s
log global
mode http

stats enable
stats uri /haproxy?stats
stats realm Statistics
stats auth REMOVED
stats refresh 10s

# frontend holds info about the public face of the site
frontend vi-gate2.docbasedirect.com
bind XXX.XX.XX.XXX:80
bind XXX.XX.XX.XXX:443 ssl crt
"/usr/local/etc/2019-www-prod-SSL.crt"
http-request redirect scheme https if !{ ssl_fc }
default_backend web_servers
option httplog

# info about backend servers
backend web_servers
balance leastconn
cookie phpsessid insert indirect nocache
option httpchk HEAD /

default-server check maxconn 2048

server vi-www3 10.3.3.10:8080 cookie phpsessid inter 120s
server vi-www4 10.3.3.11:8080 cookie phpsessid inter 120s

email-alert mailers vi-mailer
email-alert from REMOVED
email-alert to REMOVED

mailers vi-mailer
mailer localhost 127.0.0.1:25
mailer vi-backup2 10.3.3.100:25


Thanks!

-- ericr