RE: HTX no connection close - 2.0.9

2019-12-09 Thread Valters Jansons
> Could you share your configuration please ? If it only happens on a specific 
> endpoint, you can remove configuration of the others. Then if it is easily 
> reproducible, you may try to find the minimal config to do so. Finally a 
> network capture on a server side may help too (share it privately).
>
> In the mean time, could you describe your request and your response when the 
> problem occurs (size, chunked-encoding Vs content-length, compression ...) ? 
> And from the browser point of view, is there any difference with and without 
> the HTX ?
>
> If possible, could you try disabling the h2 on the frontend side ? It could 
> help to identify where the problem is.
>
> Finally, have you already tested other 2.X versions without encountering the 
> problem?

This was resolved outside of the mailing list.

The root cause had to do with the message length and handling of the HTX EOM 
block in a full HTX buffer in mux-h1.

Commit 7aae858001f99dd4a80e3f533284cda5702d501a in the public development tree 
resolves the issue.
Backported as 6f6886e021828a24295271bd83b33a9917d70323 for haproxy-2.1 and 
a882a46eefc4b2beeae2fc552e375bd0ff63f072 for haproxy-2.0 trees, expectedly 
shipping as v2.1.1 and v2.0.11 respectively.

Big thank you to Christopher for the work done!

Best regards,
Valters Jansons


Re: HTX no connection close - 2.0.9

2019-11-22 Thread Christopher Faulet

Le 21/11/2019 à 23:54, Valters Jansons a écrit :

Hello everyone,

I am running HAProxy v2.0.9 on Ubuntu using the dedicated PPA 
(ppa:vbernat/haproxy-2.0). There seems to be a behavior change for a specific 
endpoint between HTX enabled and HTX disabled, but I have not been able to 
pin-point the exact root cause.

With HTX disabled (`no option http-use-htx`), a browser makes a POST request 
(ALPN H2) which is shown as HTTP/1.1. That then reaches the backend (IIS) as 
HTTP/1.1 and finishes successfully in around 10 seconds.

With the default behavior of HTX enabled, the POST request comes in and is 
shown as HTTP/2.0. It then connects to backend as HTTP/1.1 and the client 
receives a 200 OK and the response data around the same time as without HTX. 
However, the connection does not get properly closed until server timeout with 
a termination_state of sD-- (server-side timeout in the DATA phase). At that 
point, debug log shows `srvcls` and the client connection is 'successfully' 
closed. The backend itself seems to think it handled the request 'as usual'.

The non-HTX debug log does not show srvcls, clicls and closed events on the 
backend whatsoever, but seeing as that connection does terminate, I am guessing 
the relevant events just don't get logged with HTX disabled.

We are using http-keep-alive as the default connection mode, but changing it to 
http-server-close or httpclose does not seem to make a difference.

The strange part here is that we are seeing this particular behavior with HTX 
enabled only on browsers (tested Chrome and Firefox on multiple machines), as 
testing using cURL (H2) or simply via OpenSSL's s_client (HTTP/1.1) appears to 
work even when HTX is enabled, and additionally, we are seeing this on the 
particular endpoint only for a specific user's context. That could also imply 
that it has something to do with the response data, or maybe it could just be a 
red herring. Maybe HTX is waiting on some trailing headers or some other 
feature of HTTP..

Any ideas as to where I should start troubleshooting HTX behavior for one 
production endpoint for one specific user context?



Hi,

Could you share your configuration please ? If it only happens on a specific 
endpoint, you can remove configuration of the others. Then if it is easily 
reproducible, you may try to find the minimal config to do so. Finally a network 
capture on a server side may help too (share it privately).


In the mean time, could you describe your request and your response when the 
problem occurs (size, chunked-encoding Vs content-length, compression ...) ? And 
from the browser point of view, is there any difference with and without the HTX ?


If possible, could you try disabling the h2 on the frontend side ? It could help 
to identify where the problem is.


Finally, have you already tested other 2.X versions without encountering the 
problem?


Thanks,
--
Christopher Faulet