Using HA-Proxy version 1.5.6 2014/10/18 on CentOS 6, recently updated, etc.
I'm trying to use agent-check to add/remove http servers on the back end, and
it will *not* enable a web-server; instead it marks its status as DRAIN even
though it's working fine and responding to requests.
I've set up xinetd and confirmed via telnet that it's available. I've played
with "up" and "down" and noted that check_status changes. I've confirmed that
http checks are passing with 200 status. I've watched the haproxy log file and
it doesn't seem to have any output indicating a setting or changing of status.
Below are details of output; any idea what's going on?
Ben
1) I've set up the xinetd service on the web server to report load and it's
visible from the load balancer. Sample output:
[root@backup9 html]# telnet 10.1.1.14 9333
Trying 10.1.1.14...
Connected to 10.1.1.14.
Escape character is '^]'.
up ready 88%
Connection closed by foreign host.
2) I've set up the record for the web server in haproxy.cfg as
backend logic333
server server14 10.1.1.14:20333 maxconn 256 check agent-check agent-
port 9333 agent-inter 4000
3) But when I restart haproxy, I briefly see
[status] => UP 1/3
before I see
[status] => DRAIN
When I query the stats interface. It's marked as drain, even though I see the
agent checks in the apache log on the webserver:
[root@backup8 calcload2]# tail -n 2 /var/log/httpd/access_log
10.1.1.1 - - [04/Mar/2015:22:45:32 +0000] "OPTIONS /mod.php/index.php 1.1" 200
70050 "-" "-"
10.1.1.1 - - [04/Mar/2015:22:45:34 +0000] "OPTIONS /mod.php/index.php 1.1" 200
70047 "-" "-"
They come thru every 4 seconds, just as the config would imply.
3) If I disable agent-check altogether and configure haproxy.cfg like:
server server14 10.1.1.14:20333 maxconn 256 check
Then everything *ELSE* works as expected, so it's not a problem with the web
server. I've tried manually setting the server via the stats interface, EG:
telnet 10.1.1.1 9999
enable server logic333/server14
but this seems to have no effect.
Any idea what's going on?
Here's the info from the stats interface in the error condition:
[pxname] => logic333
[svname] => server14
[qcur] => 0
[qmax] => 0
[scur] => 0
[smax] => 0
[slim] => 256
[stot] => 0
[bin] => 0
[bout] => 0
[dreq] =>
[dresp] => 0
[ereq] =>
[econ] => 0
[eresp] => 0
[wretr] => 0
[wredis] => 0
[status] => DRAIN
[weight] => 0
[act] => 1
[bck] => 0
[chkfail] => 0
[chkdown] => 0
[lastchg] => 105
[downtime] => 0
[qlimit] =>
[pid] => 1
[iid] => 5
[sid] => 3
[throttle] =>
[lbtot] => 0
[tracked] =>
[type] => 2
[rate] => 0
[rate_lim] =>
[rate_max] => 0
[check_status] => L7OK
[check_code] => 200
[check_duration] => 41
[hrsp_1xx] => 0
[hrsp_2xx] => 0
[hrsp_3xx] => 0
[hrsp_4xx] => 0
[hrsp_5xx] => 0
[hrsp_other] => 0
[hanafail] => 0
[req_rate] =>
[req_rate_max] =>
[req_tot] =>
[cli_abrt] => 0
[srv_abrt] => 0
[comp_in] =>
[comp_out] =>
[comp_byp] =>
[comp_rsp] =>
[lastsess] => -1
[last_chk] => OK
[last_agt] =>
[qtime] => 0
[ctime] => 0
[rtime] => 0
[ttime] => 0
Here's the output of the stats interface in the (brief) up state right after
restart:
[pxname] => logic333
[svname] => server14
[qcur] => 0
[qmax] => 0
[scur] => 0
[smax] => 0
[slim] => 256
[stot] => 0
[bin] => 0
[bout] => 0
[dreq] =>
[dresp] => 0
[ereq] =>
[econ] => 0
[eresp] => 0
[wretr] => 0
[wredis] => 0
[status] => UP 1/3
[weight] => 1
[act] => 1
[bck] => 0
[chkfail] => 0
[chkdown] => 0
[lastchg] => 0
[downtime] => 0
[qlimit] =>
[pid] => 1
[iid] => 5
[sid] => 3
[throttle] =>
[lbtot] => 0
[tracked] =>
[type] => 2
[rate] => 0
[rate_lim] =>
[rate_max] => 0
[check_status] => INI
[check_code] =>
[check_duration] =>
[hrsp_1xx] => 0
[hrsp_2xx] => 0
[hrsp_3xx] => 0
[hrsp_4xx] => 0
[hrsp_5xx] => 0
[hrsp_other] => 0
[hanafail] => 0
[req_rate] =>
[req_rate_max] =>
[req_tot] =>
[cli_abrt] => 0
[srv_abrt] => 0
[comp_in] =>
[comp_out] =>
[comp_byp] =>
[comp_rsp] =>
[lastsess] => -1
[last_chk] =>
[last_agt] =>
[qtime] => 0
[ctime] => 0
[rtime] => 0
[ttime] => 0
Here's the output of /var/log/haproxy.log since the most current restart:
(server12 and server13 are, in fact, down)
Mar 4 22:50:18 localhost haproxy[4987]: Proxy logic333 started.
Mar 4 22:50:19 localhost haproxy[4988]: Server logic333/server12 is DOWN,
reason: Layer4 connection problem, info: "Connection refused", check duration:
0ms. 2 active and 0 backup servers left. 0 sessions active, 0 requeued, 0
remaining in queue.
Mar 4 22:50:19 localhost haproxy[4988]: Server logic333/server13 is DOWN,
reason: Layer4 connection problem, info: "Connection refused", check duration:
0ms. 1 active and 0 backup servers left. 0 sessions active, 0 requeued, 0
remaining in queue.
Here's info on haproxy:
[root@backup9 html]# haproxy -vv
HA-Proxy version 1.5.6 2014/10/18
Copyright 2000-2014 Willy Tarreau <[email protected]>
Build options :
TARGET = linux26
CPU = generic
CC = gcc
CFLAGS = -O2 -g -fno-strict-aliasing
OPTIONS = USE_ZLIB=1 USE_OPENSSL=1
Default settings :
maxconn = 2000, bufsize = 16384, maxrewrite = 8192, maxpollevents = 200
Encrypted password support via crypt(3): yes
Built with zlib version : 1.2.3
Compression algorithms supported : identity, deflate, gzip
Built with OpenSSL version : OpenSSL 1.0.1e-fips 11 Feb 2013
Running on OpenSSL version : OpenSSL 1.0.1e-fips 11 Feb 2013
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports prefer-server-ciphers : yes
Built without PCRE support (using libc's regex instead)
Built with transparent proxy support using: IP_TRANSPARENT IP_FREEBIND
Available polling systems :
epoll : pref=300, test result OK
poll : pref=200, test result OK
select : pref=150, test result OK
Total: 3 (3 usable), will use epoll.
