Re: Client timeout on http put shows as a server timeout with error 504
*From: *Patrick Hemmer hapr...@stormcloud9.net *Sent: * 2013-09-18 10:26:36 E *To: *haproxy@formilux.org *Subject: *Re: Client timeout on http put shows as a server timeout with error 504 *From: *Willy Tarreau w...@1wt.eu *Sent: * 2013-09-18 01:46:50 E *To: *Patrick Hemmer hapr...@stormcloud9.net *CC: *haproxy@formilux.org haproxy@formilux.org *Subject: *Re: Client timeout on http put shows as a server timeout with error 504 Hi Patrick, On Tue, Sep 17, 2013 at 06:29:13PM -0400, Patrick Hemmer wrote: We have this case with haproxy 1.5-dev19 where when a client is uploading data via a HTTP PUT request, the client will fail to send all it's data and haproxy will timeout the connection. The problem is that haproxy is reporting this an error 504 and connection flags of sH--, meaning it timed out waiting for the server. Now I've analyzed the http headers, and the PUT request has a content-length header, so would it be possible to have haproxy report these as a client side timeout instead of a server side timeout (when the amount of data after headers is less than the amount indicated in the content-length header)? And with a 4XX status code as well. We have monitoring in place which looks for server errors, and I'd love for it not to pick up client problems. I remember having checked for this in the past. I agree that ideally we should have a cH--. It's a bit trickier, because in practice it is permitted for the server to respond before the end. In fact we'd need another state before the Headers state, which is the client's body, so that we can report exactly what we were waiting for. I could check if it's easier to implement now. A first step could be to disable the server-side timeout as long as we're waiting for the client. That might do the trick. Probably that you could already check for this using a slightly larger server timeout than the client's (eg: 21s for the server, 20s for the client). If that works, it would confirm that we could do this by just disabling the server timeout in this situation. Seems like it's not going to be that simple. We currently have the server timeout set at 170s, and the client timeout at 60s (and the connection closes with 504 sH-- after 170s). Though this does seem like it'd be the right approach; if the client hasn't finished sending all it's data, the client timeout should kick in. -Patrick I'm also seeing a lot of connections being closed by the client and showing up as 503 (connection flags are CC--), and in my opinion the client closing the connection shouldn't be a 500 level error. In this specific case, nginx uses code 499. Perhaps haproxy should adopt this code as well. For this and the timeout issue, if this isn't something that will be fixed any time soon, I'm willing to try and dig into it myself. However I don't know the haproxy source, so it will likely take me quite some time. -Patrick
Re: Client timeout on http put shows as a server timeout with error 504
*From: *Willy Tarreau w...@1wt.eu *Sent: * 2013-09-18 01:46:50 E *To: *Patrick Hemmer hapr...@stormcloud9.net *CC: *haproxy@formilux.org haproxy@formilux.org *Subject: *Re: Client timeout on http put shows as a server timeout with error 504 Hi Patrick, On Tue, Sep 17, 2013 at 06:29:13PM -0400, Patrick Hemmer wrote: We have this case with haproxy 1.5-dev19 where when a client is uploading data via a HTTP PUT request, the client will fail to send all it's data and haproxy will timeout the connection. The problem is that haproxy is reporting this an error 504 and connection flags of sH--, meaning it timed out waiting for the server. Now I've analyzed the http headers, and the PUT request has a content-length header, so would it be possible to have haproxy report these as a client side timeout instead of a server side timeout (when the amount of data after headers is less than the amount indicated in the content-length header)? And with a 4XX status code as well. We have monitoring in place which looks for server errors, and I'd love for it not to pick up client problems. I remember having checked for this in the past. I agree that ideally we should have a cH--. It's a bit trickier, because in practice it is permitted for the server to respond before the end. In fact we'd need another state before the Headers state, which is the client's body, so that we can report exactly what we were waiting for. I could check if it's easier to implement now. A first step could be to disable the server-side timeout as long as we're waiting for the client. That might do the trick. Probably that you could already check for this using a slightly larger server timeout than the client's (eg: 21s for the server, 20s for the client). If that works, it would confirm that we could do this by just disabling the server timeout in this situation. Seems like it's not going to be that simple. We currently have the server timeout set at 170s, and the client timeout at 60s (and the connection closes with 504 sH-- after 170s). Though this does seem like it'd be the right approach; if the client hasn't finished sending all it's data, the client timeout should kick in. -Patrick
Client timeout on http put shows as a server timeout with error 504
We have this case with haproxy 1.5-dev19 where when a client is uploading data via a HTTP PUT request, the client will fail to send all it's data and haproxy will timeout the connection. The problem is that haproxy is reporting this an error 504 and connection flags of sH--, meaning it timed out waiting for the server. Now I've analyzed the http headers, and the PUT request has a content-length header, so would it be possible to have haproxy report these as a client side timeout instead of a server side timeout (when the amount of data after headers is less than the amount indicated in the content-length header)? And with a 4XX status code as well. We have monitoring in place which looks for server errors, and I'd love for it not to pick up client problems. -Patrick
Re: Client timeout on http put shows as a server timeout with error 504
Hi Patrick, On Tue, Sep 17, 2013 at 06:29:13PM -0400, Patrick Hemmer wrote: We have this case with haproxy 1.5-dev19 where when a client is uploading data via a HTTP PUT request, the client will fail to send all it's data and haproxy will timeout the connection. The problem is that haproxy is reporting this an error 504 and connection flags of sH--, meaning it timed out waiting for the server. Now I've analyzed the http headers, and the PUT request has a content-length header, so would it be possible to have haproxy report these as a client side timeout instead of a server side timeout (when the amount of data after headers is less than the amount indicated in the content-length header)? And with a 4XX status code as well. We have monitoring in place which looks for server errors, and I'd love for it not to pick up client problems. I remember having checked for this in the past. I agree that ideally we should have a cH--. It's a bit trickier, because in practice it is permitted for the server to respond before the end. In fact we'd need another state before the Headers state, which is the client's body, so that we can report exactly what we were waiting for. I could check if it's easier to implement now. A first step could be to disable the server-side timeout as long as we're waiting for the client. That might do the trick. Probably that you could already check for this using a slightly larger server timeout than the client's (eg: 21s for the server, 20s for the client). If that works, it would confirm that we could do this by just disabling the server timeout in this situation. Best regards, Willy
