Hi Andrew, On Fri, Aug 19, 2011 at 10:01:39AM -0400, Andrew Simon wrote: > Hi Willy, > > I have a situation in the Amazon cloud where when configuring Ubuntu > HA-Proxy running your latest code to an Ubuntu Apache (latest code) Web > server backend, I am getting a 502 error from certain (PHP coded) links. > However, I receive no errors at all from the exact same links when not using > the HA-Proxy. According to your online help, I ran tcpdump -eSvvnpi eth0 > "tcp[13]&4!=0" and confirmed that, only when using HA-Proxy, the web head is > spitting out one RST packet, and the proxy eagerly agrees to stop receiving > and errors out with 502. Another symptom of the problem is that the link > works on the first click or two, then fails continually after-wards. > However, after waiting a minute or so, the link will once again 'magically' > work, but only once. It then returns to the 502 condition.
>From what you describe, it sounds like your script responds and closes before consuming the request, which makes the system emit a reset along with the response. The issue with this behaviour is that it's totally random : if the reset is sent after the data, it works. If it's sent before, the connection is destroyed and the data are lost (and sometimes destroyed on the server side). > I was wondering if you have a patch or setting for HA proxy that allows for > ignoring RST packets (and/or not causing them). It's not haproxy which processes the RST packet, it's the TCP stack. The RST kills the local connection and removes any pending data from the buffers. > I ask this because, despite > this 1 RST packet, the tcpdump with less filtering options shows the web > server is indeed sending (more-or-less) the entire web page - not just one > packet - thus, it is the entire web page of packets, plus this one RST. There is something very ugly that you could do in emergency, it would be to set an iptables rule which would drop RST packets from your PHP server for established connections. > It seems an unfortunate 'feature' of the proxy that, despite the web > server's sending literally tens of thousands of packets in response to a > link request, the proxy should use this one RST packet to determine the fate > of the transmission, and use it as an excuse to abort and essentially > discard all of the other tcp packets. As I explained, it's not the proxy, it's how TCP works. You may want to hack your TCP stack to prevent it from doing so, but you'll get many more issues. > So, as you would assert, this is an 'application' bug with the web server's > (PHP) code. But as I assert, there are two HA-Proxy bugs: 1) the error does > not occur unless one uses HA-Proxy; It's just a matter of timing. We're used to see such things on the other side too. There are some people who've been hit by exactly the same thing on their clients from haproxy, when either of those close the connection after the response. But if the client sends more data between the end of request and the close, the kernel emits an RST which voids the transfer. In general, the further the client from the server, the less risk of occurring. The fact that haproxy is very close to the server increases the risk of seeing it. > and 2) the HA-Proxy arrives at the wrong > conclusion on what to do with the transmission when it receives the one RST > packet. Again this is not haproxy but the kernel. Haproxy performs recv() on a socket and the kernel returns -1 to indicate an error. There is no way to force the kernel to ignore the error, when this happens, the socket already does not exist anymore. > In lieu of a setting I can use to cause HA-Proxy to ignore - and/or not > cause - the RST messages, I will use NGINX for load balancing, which does > not cause this error, nor does it trip over the RST packet, should it find > one. If/when/until I hear back from you on a resolution to the problem, I > will stick with NGINX and recommend others with this problem do the same. I find it really too bad that you adopt such an arrogant attitude when asking for help. I took the time to respond to your message, and if I had read this last paragraph before starting to reply, I would have sent your message to /dev/null. So instead of deleting it now that I took the time to respond, I'm CCing the mailing list so that it can at least be helpful to other people. I'm sure you won't mind since you suggested telling everyone about your issue anyway. > Thanks in advance for your help, > Andrew Simon > [email protected] Willy
