Hi Baptiste, late feedback but it's better than none ;-)
Le 12/07/2015 23:38, Baptiste a écrit :
First, we want to fix the error when HAProxy fails starting up because the resolvers pointed by the system can't resolve a server's IP address (but HAProxy resolvers could).
It is pretty awaited from some teams at work. I'll add that even if HAProxy resolvers can't resolve the entries or are not configured, we should allow to make it non fatal and leave such server entries definitely DOWN with healthchecks disabled (except if it is updated from the CLI).
I see that Marco Corte has provided the same feedback ;-)
The idea here would to create a new flag on the server to tell HAProxy which IP to use. The server would be enabled when the IP has been provided by the expected tool. IE, a new server directive could be "init-addr" (for inital IP address) and would take a list of directive from 'libc', 'dns', 'a.b.c.d' (arbitrary IP address), etc... (non exhaustive live, more to come...) Currently, HAProxy works like this: "init-addr libc,dns" A new value could be "init-addr dns" Or "init-addr 1.2.3.4,dns"
To be clear, does it mean that we could meet such configuration ? server server1 host1.somewhere:80 init-addr 127.0.0.1,libc,dns Which would mean : - Use the init IP first - At start up, try to resolve whith the libc resolver - At runtime, try to resolve with HAProxy resolvers. Then, maybe we'll have to prevent such (un)funny things : server server1 192.0.2.1:80 init-addr 127.0.0.1,libc,dns
Second, we want to log server IP changes. For now, there are 2 ways to change a server IP address: DNS resolution or using the stats socket command: "set server addr" 2 options: - we setup a parameter to enable logging server IP changes, whatever has updated the server IP - we allow HAProxy to log server IP changes from a specific source only. IE, log only when DNS change a server's IP
Why not something similar to init-addr ? like "log-resolve-update libc,dns" or "log-resolve-update dns" to prevent logging dns resolution during the init step (which will keep the current behaviour, to prevent flooding the logs when haproxy starts with a lot of servers declared).
About the stats socket, I think it's important to always log server changes even it nothing is set in the configuration.
Third, we have to handle DNS response errors. We thought about the 4 following cases: - NX domain : all DNS servers can't resolve this host name - response timeout : no response was received - query refused : the DNS servers refused our query - other : all other cases => For each error, we can maintain the latest good IP for a period decided by the user. IE, if you want to keep a server up for 5 minutes while your servers return NX, then setup "hold nx 5m" in your resolvers section Fourth, we need a new server state when a DNS resolution is in error. Currently, we have 2 types of state: operational or administrative - administrative states: ready, maint, drain - operational states: down, failed, stopped We have to create a new state (should be operational) which reports that HAProxy is not able to perform a proper DNS resolution for this server. Once in that state, the server won't be able to get new traffic, health checks will be stopped too. HAProxy will turn the server in this state after the hold period described in step #3.
When servers fall to this state, maybe we'll have to define how haproxy will react with some configurations. For example, what's the effect of "force-persist" on a server in this state ? I'd say it's not like a DOWN server and it shouldn't persist, but we'll have to be clear on such use cases.
-- Cyril Bonté
