Re: Is it possible to avoid 503 error when one backend server has down and health check hasn't been launched yet
On Sat, Dec 24, 2016 at 05:16:53PM +, Patrick Hemmer wrote: > > On 2016/12/24 10:42, Alex.Chen wrote: > > for my scenario, i need to using "balance source" to keep the > > persistence of haproxy's balancing, I find that when one of my backend > > server (s1) has been killed, and if the next round health check is > > still not launched, then s1 is still be marked as UP. after 3 retries, > > the redispatch option does not work, I still get a 503 error. after a > > while, health check launched and s1 has been marked as DOWN, then my > > req has been forward to another backend server and everything is ok now. > > > > my quesition is that, is there any config can help me to avoid 503 > > error when 3 retries have been failed but s1 is still marked as UP > > before the next round health check > > > > I debug haproxy(1.6.10) and find that when I using "balance source", > > the redispatch option does not work actually. after 3 retries, > > redispatch does not work, I guess that is because "balance source" is > > deterministic based on source IP and server state info(UP/DOWN and > > weight) (from > > : > > http://blog.haproxy.com/2013/04/22/client-ip-persistence-or-source-ip-hash-load-balancing/ > > ) so if the server looks like "UP" then the balance source will still > > assign redispatch new conn to this deaded server s1. > > > > > I would think the "observe" option should handle this issue. > https://cbonte.github.io/haproxy-dconv/1.7/configuration.html#5.2-observe Yep definitely, it's one of its use cases. As a complement it is also recommended to set "fastinter" to a value much lower than "inter" so that once a first check fails, next health checks are triggered very quickly and the server is evicted fast. Willy
Re: Is it possible to avoid 503 error when one backend server has down and health check hasn't been launched yet
On 2016/12/24 10:42, Alex.Chen wrote: > for my scenario, i need to using "balance source" to keep the > persistence of haproxy's balancing, I find that when one of my backend > server (s1) has been killed, and if the next round health check is > still not launched, then s1 is still be marked as UP. after 3 retries, > the redispatch option does not work, I still get a 503 error. after a > while, health check launched and s1 has been marked as DOWN, then my > req has been forward to another backend server and everything is ok now. > > my quesition is that, is there any config can help me to avoid 503 > error when 3 retries have been failed but s1 is still marked as UP > before the next round health check > > I debug haproxy(1.6.10) and find that when I using "balance source", > the redispatch option does not work actually. after 3 retries, > redispatch does not work, I guess that is because "balance source" is > deterministic based on source IP and server state info(UP/DOWN and > weight) (from > : > http://blog.haproxy.com/2013/04/22/client-ip-persistence-or-source-ip-hash-load-balancing/ > ) so if the server looks like "UP" then the balance source will still > assign redispatch new conn to this deaded server s1. > > I would think the "observe" option should handle this issue. https://cbonte.github.io/haproxy-dconv/1.7/configuration.html#5.2-observe -Patrick
Is it possible to avoid 503 error when one backend server has down and health check hasn't been launched yet
for my scenario, i need to using "balance source" to keep the persistence of haproxy's balancing, I find that when one of my backend server (s1) has been killed, and if the next round health check is still not launched, then s1 is still be marked as UP. after 3 retries, the redispatch option does not work, I still get a 503 error. after a while, health check launched and s1 has been marked as DOWN, then my req has been forward to another backend server and everything is ok now. my quesition is that, is there any config can help me to avoid 503 error when 3 retries have been failed but s1 is still marked as UP before the next round health check I debug haproxy(1.6.10) and find that when I using "balance source", the redispatch option does not work actually. after 3 retries, redispatch does not work, I guess that is because "balance source" is deterministic based on source IP and server state info(UP/DOWN and weight) (from : http://blog.haproxy.com/2013/04/22/client-ip-persistence-or-source-ip-hash-load-balancing/ ) so if the server looks like "UP" then the balance source will still assign redispatch new conn to this deaded server s1.
