Re: AW: Disabling HTTP/1.1 pipelining

2021-09-24 Thread Willy Tarreau
On Fri, Sep 24, 2021 at 10:05:19AM +0200, Christopher Faulet wrote:
> > So I guess I did just misunderstand the documentation and it would be nice
> > to just clarify it in the docs that haproxy does not support HTTP/1.1
> > pipelining.
> 
> I agree. Pipelining should at least be removed from "option
> http-server-close" description. And section 1.1 should be reword to be clear
> on this point.

Both of you are both a bit right and a bit wrong :-) But it's a matter of
wording and you're not speaking exactly about the same thing.

HAProxy *does* support request pipelining, it will just serialize what
it receives and will not do pipelining on the other side. This is
particularly important, especially with in the context of the HTTP model,
because pipelining is only useful on high latency links over the net, and
used not to work with a number of servers (and even broke a few times in
haproxy itself). Before H2 was released, some browsers (mostly Opera) used
to rely on tricks to try to detect whether pipelining was broken or not on
the server and to enable it. When it worked (with a modern server or with
haproxy installed in front of a modern server), it used to provide a very
noticeable performance gain, especially on slow or congested links, where
a client could send 10 requests at once and receive 10 responses at once,
while on the other side you'd observe 10 fast request/response cycles.

I've reread what is said in the option http-server-close and timeout
http-keep-alive sections and what is mentioned there remains perfectly
valid as it explicitly designates the communication with the client, so
for me there is nothing to change there.

Hoping this helps,
Willy



Re: AW: Disabling HTTP/1.1 pipelining

2021-09-24 Thread Christopher Faulet

Le 9/21/21 à 6:00 PM, Stefan Behte a écrit :

Hi Christopher,

thank you for the hint, I'm aware of the different ways to mitigate DDoS with 
rate limits etc., I was just curious about the pipelining vector. :)

http://www.haproxy.org/download/2.4/doc/configuration.txt says:

" By default HAProxy operates in keep-alive mode with regards to persistent
   connections: for each connection it processes each request and response, and
   leaves the connection idle on both sides between the end of a response and
   the start of a new request. This mode may be changed by several options such
   as "option http-server-close" or "option httpclose". Setting "option
   http-server-close" enables HTTP connection-close mode on the server side
   while keeping the ability to support HTTP keep-alive and pipelining on the
   client side."

"1.1. The HTTP transaction model" and " timeout http-keep-alive" also mention 
pipelining.



Section 1.1 mainly describes generalities about the HTTP protocol. Only the end 
of the section is focused on HAProxy and it is specified it only supports 
keep-alive mode, not the pipelining.  However, I agree it is pretty confusing 
because pipelining is mentioned in "option http-server-close" and "timeout 
http-keep-alive" descriptions.


In fact, the ambiguities comes from the fact that HAProxy does not performed any 
HTTP pipelining. But the client is free to send several requests in same time. 
No error will be triggered. However, the requests will be processed the one 
after the other. Thus, HAProxy does not perform any HTTP pipelining but it does 
not forbid it.



So I guess I did just misunderstand the documentation and it would be nice to 
just clarify it in the docs that haproxy does not support HTTP/1.1 pipelining.


I agree. Pipelining should at least be removed from "option http-server-close" 
description. And section 1.1 should be reword to be clear on this point.



--
Christopher Faulet



AW: Disabling HTTP/1.1 pipelining

2021-09-21 Thread Stefan Behte
Hi Christopher,

thank you for the hint, I'm aware of the different ways to mitigate DDoS with 
rate limits etc., I was just curious about the pipelining vector. :)

http://www.haproxy.org/download/2.4/doc/configuration.txt says:

" By default HAProxy operates in keep-alive mode with regards to persistent
  connections: for each connection it processes each request and response, and
  leaves the connection idle on both sides between the end of a response and
  the start of a new request. This mode may be changed by several options such
  as "option http-server-close" or "option httpclose". Setting "option
  http-server-close" enables HTTP connection-close mode on the server side
  while keeping the ability to support HTTP keep-alive and pipelining on the
  client side."

"1.1. The HTTP transaction model" and " timeout http-keep-alive" also mention 
pipelining.

So I guess I did just misunderstand the documentation and it would be nice to 
just clarify it in the docs that haproxy does not support HTTP/1.1 pipelining.
 
Best regards,
Stefan Behte

-Ursprüngliche Nachricht-
Von: Christopher Faulet  
Gesendet: Montag, 20. September 2021 19:04
An: Stefan Behte ; haproxy@formilux.org
Betreff: Re: Disabling HTTP/1.1 pipelining

Le 9/17/21 à 1:20 PM, Stefan Behte a écrit :
> Hi everyone,
> 
> surely many on this list have heard about the meris botnet 
> (https://krebsonsecurity.com/2021/09/krebsonsecurity-hit-by-huge-new-iot-botnet-meris/)
>  
> which uses HTTP/1.1 pipelining for layer 7 attacks.
> 
> As far as I can see, it's not possible to disallow HTTP pipelining in 
> haproxy, 
> so the best possibility could be "option httpclose"?
> 
> Of course, this does not solve everything when a ~100k botnet is attacking, 
> but 
> it could ease the initial load / mitigate the pipelining vector a bit, as the 
> attack clients have longer RTT.
> 
> Or maybe I am missing something?
> 
Hi,

HAproxy does not support HTTP pipelining. But it may be configured to mitigate 
ddos attack. There are several mechanisms that you can use, depending on your 
applications. A quick search on the net about "haproxy ddos prevention" will 
give you several hints.

Regards,
-- 
Christopher Faulet




Re: Disabling HTTP/1.1 pipelining

2021-09-20 Thread Christopher Faulet

Le 9/17/21 à 1:20 PM, Stefan Behte a écrit :

Hi everyone,

surely many on this list have heard about the meris botnet 
(https://krebsonsecurity.com/2021/09/krebsonsecurity-hit-by-huge-new-iot-botnet-meris/) 
which uses HTTP/1.1 pipelining for layer 7 attacks.


As far as I can see, it's not possible to disallow HTTP pipelining in haproxy, 
so the best possibility could be "option httpclose"?


Of course, this does not solve everything when a ~100k botnet is attacking, but 
it could ease the initial load / mitigate the pipelining vector a bit, as the 
attack clients have longer RTT.


Or maybe I am missing something?


Hi,

HAproxy does not support HTTP pipelining. But it may be configured to mitigate 
ddos attack. There are several mechanisms that you can use, depending on your 
applications. A quick search on the net about "haproxy ddos prevention" will 
give you several hints.


Regards,
--
Christopher Faulet



Disabling HTTP/1.1 pipelining

2021-09-17 Thread Stefan Behte
Hi everyone,

surely many on this list have heard about the meris botnet 
(https://krebsonsecurity.com/2021/09/krebsonsecurity-hit-by-huge-new-iot-botnet-meris/)
 which uses HTTP/1.1 pipelining for layer 7 attacks.
As far as I can see, it's not possible to disallow HTTP pipelining in haproxy, 
so the best possibility could be "option httpclose"?
Of course, this does not solve everything when a ~100k botnet is attacking, but 
it could ease the initial load / mitigate the pipelining vector a bit, as the 
attack clients have longer RTT.

Or maybe I am missing something?

Best regards,
Stefan Behte