Hi Cyril, Awesome, that's exactly what I need. Thanks for sharing it, I'll definitely implement it this way!
Regards, Mark Op 15 feb. 2014, om 22:58 heeft Cyril Bonté <[email protected]> het volgende geschreven: > Hi Mark, > > Le 15/02/2014 21:28, Mark Ruys a écrit : >> Hi, >> >> I've read the blog >> http://blog.exceliance.fr/2014/01/17/emulating-activepassing-application-clustering-with-haproxy/ >> which >> describes how to prevent an automatic fail back in case of a a fail over >> in a active/passive setting. This is important for us as we want to be >> sure that MySQL replication is fully recovered before the failed master >> gets active again. > > It's confusing because the examples you provided are for HTTP servers and > you're talking about MySQL replication. In many cases, you'll manage it > differently. Well, I'll use HTTP too for my example but will try to describe > a solution that works for mysql. > >> In a typically HAProxy setting (at least in our case), we have backend >> servers serving a few IP-adresses (each SSL certificate needs its own IP >> address). So the frontend binds to a few IP-addresses. But the stick >> table can only hold a single dst IP-address. Shouldn't the size of stick >> table be at least at big as the number of addresses the frontend binds to? >> >> Then I have another problem. In some unfortunate scenarios the network >> gets unreliable. So it could be the case that server s1 is unreachable >> and automatic fail over to s2 occurs. This is sticky, so when s2 is >> reachable again, no automatic fail back happens, just as we want. But >> then s2 gets unreachable. In this case an automatic fail back will >> start! This is against the prerequisite that a fail back should /only/ >> occur by manual intervention. So wouldn't it be safer to define the >> backend like this: >> >> backend bk_app >> stick-table type ip size 1 peers LB >> stick on dst >> server s1 10.0.0.1:80 check rise 9999999 >> server s2 10.0.0.2:80 check backup >> >> Then we've got 100 days time to review the situation and force a manual >> fail back when I'm ready (by reloading the haproxy service). > > I think that for such a critical scenario, it can be a good idea to split > servers into 2 backends. > > Example : > backend active > server s-active 10.0.0.1:80 check > > backend passive > server s-passive 10.0.0.2:80 > > Then a frontend can elect the backend depending on the "active" backend state > : > frontend cluster :80 > acl PASSIVE nbsrv(active) eq 0 > use_backend passive if PASSIVE > default_backend active > > This will (almost) simulate the "backup" behaviour, without some features > such as redispatch. But in a replication scenario, I prefer a connection > error than a redispatch that will write to the wrong server. > > Now, it's time to have a sticky cluster. For that : > 1. Add 1 stick table in the "passive" backend, > 2. Add a new condition in the frontend to check if there's an entry or not > (this is where I introduce "always_true"); > 3. Peers are used to remember the states on reload > 4. Also add a stats socket to allow a manual reset (switch back to the active > server). > > Example : > global > daemon > pidfile /var/run/tmp/haproxy.pid > stats socket /var/run/haproxy.sock > > peers LB > peer LB1 10.0.0.98:1234 > peer LB2 10.0.0.99:1234 > > defaults > timeout client 20s > timeout server 20s > timeout connect 4s > > frontend cluster :80 > acl PASSIVE nbsrv(active) eq 0 > acl PASSIVE table_cnt(passive) gt 0 > use_backend passive if PASSIVE > default_backend active > > backend active > server s-active 10.0.0.1:80 check > > backend passive > stick-table type integer size 1 peers LB > stick on always_true > server s-passive 10.0.0.2:80 # is it necessary to check the passive one ? > > From this example, the default backend will be the active one. > Once "s-active" is DOWN or an entry exists in the "passive" table, the > "passive" backend will be used. > > To switch back to the "active" backend : > # echo "clear table passive" | socat stdio /var/run/haproxy.sock > > > -- > Cyril Bonté > --- dr M.P.J. Ruys :: Lead Software Engineer Peercode Oudenhof 4c, 4191NW Geldermalsen, The Netherlands Web site and travel directions: www.peercode.nl Phone +31.88.0084124 :: Mobile +31.6.51298623
