Hey, I have ran into a odd scenario, where the backend is DOWN however the layer 7 checks are passing. I have included the check which we received. The haproxy setup is fairly simple using proxy protocol. I could only find one example of this issue here, however, no follow up was done on the reason: http://comments.gmane.org/gmane.comp.web.haproxy/10454
The current setup is haproxy box uses proxy protocol to nginx which sends it to the backends. The only thing I question that may have effected this was "limit_conn slowdown 50;" in nginx. The nodes did hit the slowdown due to volume, I however was unable to replicate this same DOWN Layer7 check passed in synthetic testing afterwords. localhost haproxy[27338]: Server backend_name/ip is DOWN, reason: Layer7 check passed, code: 200, check duration: 4ms. 0 active and 0 backup servers left. 344 sessions active, 0 requeued, 0 remaining in queue. HAProxy Config (IP's and Service names changed): https://gist.github.com/coosh/63939a070ac60509ef16 Nginx Config <all of them are very simple> (IP's and Service names changed): https://gist.github.com/coosh/0dfefd164d4ccbce9ffa Rob