On 28 Jun 2017 12:45 am, "Norman Branitsky" <[email protected]> wrote:
Using the NS1 managed DNS service, I monitor the health of 2 HAProxy 1.7.7 servers defined as peers. Not related to the op but 1.7.8 got just released with some important fixes. NS1 checks the health of the HAProxy servers every 30 seconds. This is too long for production dns load balancing should be 10 seconds at least if allowed by the provider. If haproxy1 fails to respond, NS1 changes the DNS response to point to haproxy2. When haproxy1 comes back online, NS1 reverts the DNS response to haproxy1. NS1 checks the health of my Java application server every 60 seconds. NS1 DNS records looks like this: haproxy1 A record haproxy2 A record tm1 CNAME record “Dynamic” – NS1 “filter” returns the first in the list of all health haproxy servers vr CNAME record pointing to tm1 – name of the Java application server Not clear what is the TTL of the records though? Is it equal to the health check interval for each? In that case shouldn't the times of haproxy and the app be same ie 30 seconds? You can potentially hit the scenario where the client caches the dns for 60 seconds and haproxy failed over (tm1 changed) 30 seconds earlier. If I connect to my Java application with Chrome or Firefox, I often don’t notice the haproxy DNS failover. If I do get a connection error, it almost always reconnects within seconds. I don’t lose my session. If I connect to my Java application with IE (only tested IE10 mode so far), the haproxy DNS failover cause a DNS error. This error won’t clear for at least 20 minutes. Is this the session life time? What does it have to do with dns i wonder? What is the dns cache ttl in ie10 set to (i never use ie so no idea)? And is it tunable? If I open a new tab I connect instantly. Since the JSESSIONID cookie is still available, I’m still logged in but obviously not on the same data entry page. What can I do to kick IE in the head and cause it to refresh its DNS cache? It doesn’t seem to respect the TTL value. Nothing. Common problem with dns load balancing if the client doesn't respect the ttl you can't do anything about it. Although one problem I see is the single record NS1 returns for your app dns. Usually you want it to return both haproxy A records in round-robin order so (some) clients can try the second one if the first one fails before they make a new dns query. Never used NS1 so not sure if this functionality exists at all. Also is NS1 always returning the same A record out of the two (looks like the case if it uses sorting)? In that case all clients will connect to the same haproxy (not sure if you want/need this active-passive setup though). Norman *Norman Branitsky *Cloud Architect MicroPact (o) 416.916.1752 (c) 416.843.0670 (t) 1-888-232-0224 x61752 www.micropact.com Think it > Track it > Done
