Hello!
We use HAProxy for several services and it's working great. However, for
one specific load balancing need, we need to be able to put a server in
backup mode without having to modify the configuration file followed by a
reload.
Consider a single master MySQL deployment with multiple slaves, and two
backends/frontends on HAProxy: master, and slave, where the master database
is the single UP and operational server in the master pool, and the slave
pool consists of the slave servers plus the master in backup mode:
global
stats socket /etc/haproxy/haproxy.sock level admin
server-state-file /etc/haproxy/haproxy.state
listen mysql-master
bind :10000
option tcp-check
server server1 10.0.0.1:3306 check weight 100
server server2 10.0.0.2:3306 check weight 100
server server3 10.0.0.3:3306 check weight 100
listen mysql-slave
bind :10001
option tcp-check
server server1 10.0.0.1:3306 check weight 100 backup
server server2 10.0.0.2:3306 check weight 100
server server3 10.0.0.3:3306 check weight 100
In the mysql-master pool we have server1 (master), and server2 and server3
which are slaves. The state file makes sure that server2 and server3 will
be loaded into HAProxy in MAINT mode at launch time so that only the master
(server1) is served at port 10000. The master in the slave pool is in
backup mode, and HAProxy will only add it to slave LB farm only when the
slaves are not UP.
We use MHA for automating MySQL master failover. MHA supports hook scripts
that are launched at certain steps of a master switch over. For instance
the script that acts as a master_ip_failover_script:
https://code.google.com/p/mysql-master-ha/wiki/Parameters#master_ip_failover_script
will be kicked off by the MHA before shutting down the master, AND after
the newly promoted master is ready. The script that we implemented for this
purpose communicates with the HAProxy socket via socat at both of these
steps.
At the first step, we want the HAProxy to put the master (server1) offline,
but continue to serve current connections (DRAIN). We do this by either
running "set server mysql-master/server1 weight 0" or "set server
mysql-master/server1 state drain" command. At this stage, we don't initiate
a command for master server in the slave pool as it isn't necessary.
At the second step when the new master is ready (say server2 is the new
master), the script will run "set server mysql-master/server2 state ready"
and "set server mysql-master/server2 weight 100" commands to make the new
master UP and available. It will also run "disable server
mysql-master/server1" to disable the old master in the master pool - assume
the time passes between the two steps is sufficient for the connections to
be drained. Lastly, we need to be able to run "set server
mysql-slave/server1 state active" to remove its backup flag and "set server
mysql-slave/server2 state backup" to put the new master in backup mode.
Unfortunately, "set server <server> state active/backup" commands do not
exist and the only way to make this change is by modifying the haproxy.cfg
file and reloading the daemon. For the sake of automation, we've modified
the HAProxy 1.6.6 code that support active/backup commands. I've attached
the patch against v1.6.6 for your verification.
I didn't dive much into the HAProxy source code and I don't know at this
point whether or not this patch would mess up the inner mechanics of
HAProxy but I want to be able to run these commands on the HAProxy socket
file AND I want to avoid any potential short circuits.
Thanks in advance for your feedback.
Murat
diff -urN a/src/dumpstats.c b/src/dumpstats.c
--- a/src/dumpstats.c 2016-06-26 17:41:01.000000000 +0000
+++ b/src/dumpstats.c 2016-06-28 18:45:05.664074832 +0000
@@ -121,6 +121,8 @@
ST_ADM_ACTION_READY,
ST_ADM_ACTION_DRAIN,
ST_ADM_ACTION_MAINT,
+ ST_ADM_ACTION_BACKUP,
+ ST_ADM_ACTION_ACTIVE,
ST_ADM_ACTION_SHUTDOWN,
/* these are the ancient actions, still available for compatibility */
ST_ADM_ACTION_DISABLE,
@@ -1529,8 +1531,20 @@
srv_adm_set_drain(sv);
else if (strcmp(args[4], "maint") == 0)
srv_adm_set_maint(sv);
+ else if (strcmp(args[4], "backup") == 0) {
+ struct proxy *p = sv->proxy;
+ sv->flags |= SRV_F_BACKUP;
+ p->srv_act--;
+ p->srv_bck++;
+ }
+ else if (strcmp(args[4], "active") == 0) {
+ struct proxy *p = sv->proxy;
+ sv->flags &= ~SRV_F_BACKUP;
+ p->srv_act++;
+ p->srv_bck--;
+ }
else {
- appctx->ctx.cli.msg = "'set server
<srv> state' expects 'ready', 'drain' and 'maint'.\n";
+ appctx->ctx.cli.msg = "'set server
<srv> state' expects 'ready', 'drain', 'maint', 'active', and 'backup'.\n";
appctx->st0 = STAT_CLI_PRINT;
}
}
@@ -4084,6 +4098,8 @@
"<option value=\"ready\">Set state to
READY</option>"
"<option value=\"drain\">Set state to
DRAIN</option>"
"<option value=\"maint\">Set state to
MAINT</option>"
+ "<option value=\"backup\">Set state to
BACKUP</option>"
+ "<option value=\"active\">Set state to
ACTIVE</option>"
"<option value=\"dhlth\">Health: disable
checks</option>"
"<option value=\"ehlth\">Health: enable
checks</option>"
"<option value=\"hrunn\">Health: force
UP</option>"
@@ -4891,6 +4907,12 @@
else if (strcmp(value, "maint") == 0) {
action = ST_ADM_ACTION_MAINT;
}
+ else if (strcmp(value, "backup") == 0) {
+ action = ST_ADM_ACTION_BACKUP;
+ }
+ else if (strcmp(value, "active") == 0) {
+ action = ST_ADM_ACTION_ACTIVE;
+ }
else if (strcmp(value, "shutdown") == 0) {
action = ST_ADM_ACTION_SHUTDOWN;
}
@@ -5064,6 +5086,20 @@
altered_servers++;
total_servers++;
break;
+ case ST_ADM_ACTION_BACKUP:
+ sv->flags |= SRV_F_BACKUP;
+ px->srv_act--;
+ px->srv_bck++;
+ altered_servers++;
+ total_servers++;
+ break;
+ case ST_ADM_ACTION_ACTIVE:
+ sv->flags &= ~SRV_F_BACKUP;
+ px->srv_act++;
+ px->srv_bck--;
+ altered_servers++;
+ total_servers++;
+ break;
case ST_ADM_ACTION_SHUTDOWN:
if (px->state != PR_STSTOPPED) {
struct stream *sess,
*sess_bck;
