Re: HTTP/3 | daniel.haxx.se

[Aleksandar Lazic]

Am 12.11.2018 um 23:31 schrieb Willy Tarreau:
> On Mon, Nov 12, 2018 at 10:52:41PM +0100, Aleksandar Lazic wrote:
>> Oh wow this is really a good time to get the hands dirty as QUIC is a major
>> design change in HTTP, IMHO.
> 
> Some first approaches were already attempted about one year ago already,
> to avoid being later in the party. But just like when we had to work on
> H2, trying to stuff this into an existing stream-based component is not
> easy and we ended up identifying a lot of lower layers that had to be
> changed first, so this work was paused and the protocol changed a lot
> during that time.
> 
>> To adopt the matrix from one of the last message, haproxy will be able to do 
>> the
>> conversion in almost every direction?
>>
>> HTTP/1.x <> HTTP/2
>> HTTP/2 <> HTTP/3
>> HTTP/1.x <> HTTP/3
>> HTTP/3 <> HTTP/3
> 
> Yes, that's the idea. In fact it will be slightly different, we're 
> implementing
> protocol conversion at the lower layers, to an internal version-agnostic HTTP
> representation supposed to accurately represent a message while keeping its
> semantics, that we've called HTX. I know that you love ASCII art, so it will
> look more or less like this (we'll have to provide a lot of doc but first we
> really want to focus on getting the code merged) :
> 
> 
>+---+ stream
>|   all HTTP processing | layer
>+---+
>^ ^ ^  ^
>HTX | HTX | HTX |  HTX |  normalised
>v v v  v  interface
>+--+ ++ ++ ++
>|applet| | HTTP/1 | | HTTP/2 | | HTTP/3 | whatever layer (called mux for 
> now
>+--+ ++ ++ ++ but may change once we have 
> others,
> cache || ||| could be presentation in OSI)
> stats | +--+  |+--+
> Lua svc   | |TLS   |  || QUIC |  transport layer
>   | +--+  |+--+ 
>   |   |   ||
> +-+ +-+
> | TCP/Unix/socketpair | | UDP |  control layer
> +-+ +-+
>   ||
>+--+
>|file descriptor   |  socket layer
>+--+
>|
>  +---+
>  | operating |
>  |  system   |
>  +---+
> 
> 
> It's really over-simplified since several layers above in fact have multiple
> upper arrows as they are multiplexed, but it's to explain more or less how
> stuff gets stacked. And since there are always transition periods, you have
> multiple protocols possible between each layer, otherwise it wouldn't be
> fun.  You have coded in the very old version where the file descriptors
> were directly present in the stream layer. As you can see, over a decade
> a lot of new layers have been built between the operating system and the
> streams, without even losing features nor compatibility. That's where the
> real challenges are.

Thank you for the picture, you know me too good 8-).

I follow the development a "little" bit, and it's amazing how you and your team
was able to rewrite a lot of haproxy and keep a high level of compatibility and
stability.

BTW: I loved to see that std* logger is now there ;-)

>> Is this technically possible as the UDP/QUIC may have not some information 
>> that
>> the TCP/HTTP need?
> 
> People tend to confuse QUIC and UDP too much. QUIC more or less uses
> UDP as a replacement for IP, which doesn't require privileges, and it
> reimplements its own transport stack on top of it, with congestion
> control, multiplexing, encryption etc. So it effectively represents a
> reliable transport layer for multiple streams based on datagrams below.
> UDP is seen as the revolutionary thing when you tell someone that HTTP
> will work over UDP but that's absolutely not true, it's almost comparable
> to saying that HTTP works over UDP when you're doing it over TLS over
> TCP over OpenVPN over UDP.

Hm, sound complicated.

I think to debug this beast isn't that "easy" as for now with `curl -v ...`,
even QUIC is planned for curl ;-)

>> As I try to imagine the new design in HAProxy will it look like this?
>>
>> TCP(TCP|UDP) HTTP/1.x,HTTP/2,HTTP/3
>>   \  /
>>  HTTP/TCP/UDP Engine - (TCP|UDP) HTTP/1.x,HTTP/2,HTTP/3
>>  (Here happen some magic)
>>   /  \
>> UDP(TCP|UDP) HTTP/1.x,HTTP/2,HTTP/3
>>
>>
>> Really 8-O ?
> 
> Precisely not, see above ;-)
> 
>> What happens with other protocols like plain TCP/UDP or grpc and so on?
> 
> Nothing, no particular change, 

Re: HTTP/3 | daniel.haxx.se

[Willy Tarreau]

On Mon, Nov 12, 2018 at 10:52:41PM +0100, Aleksandar Lazic wrote:
> Oh wow this is really a good time to get the hands dirty as QUIC is a major
> design change in HTTP, IMHO.

Some first approaches were already attempted about one year ago already,
to avoid being later in the party. But just like when we had to work on
H2, trying to stuff this into an existing stream-based component is not
easy and we ended up identifying a lot of lower layers that had to be
changed first, so this work was paused and the protocol changed a lot
during that time.

> To adopt the matrix from one of the last message, haproxy will be able to do 
> the
> conversion in almost every direction?
> 
> HTTP/1.x <> HTTP/2
> HTTP/2 <> HTTP/3
> HTTP/1.x <> HTTP/3
> HTTP/3 <> HTTP/3

Yes, that's the idea. In fact it will be slightly different, we're implementing
protocol conversion at the lower layers, to an internal version-agnostic HTTP
representation supposed to accurately represent a message while keeping its
semantics, that we've called HTX. I know that you love ASCII art, so it will
look more or less like this (we'll have to provide a lot of doc but first we
really want to focus on getting the code merged) :


   +---+ stream
   |   all HTTP processing | layer
   +---+
   ^ ^ ^  ^
   HTX | HTX | HTX |  HTX |  normalised
   v v v  v  interface
   +--+ ++ ++ ++
   |applet| | HTTP/1 | | HTTP/2 | | HTTP/3 | whatever layer (called mux for now
   +--+ ++ ++ ++ but may change once we have others,
cache || ||| could be presentation in OSI)
stats | +--+  |+--+
Lua svc   | |TLS   |  || QUIC |  transport layer
  | +--+  |+--+ 
  |   |   ||
+-+ +-+
| TCP/Unix/socketpair | | UDP |  control layer
+-+ +-+
  ||
   +--+
   |file descriptor   |  socket layer
   +--+
   |
 +---+
 | operating |
 |  system   |
 +---+


It's really over-simplified since several layers above in fact have multiple
upper arrows as they are multiplexed, but it's to explain more or less how
stuff gets stacked. And since there are always transition periods, you have
multiple protocols possible between each layer, otherwise it wouldn't be
fun.  You have coded in the very old version where the file descriptors
were directly present in the stream layer. As you can see, over a decade
a lot of new layers have been built between the operating system and the
streams, without even losing features nor compatibility. That's where the
real challenges are.

> Is this technically possible as the UDP/QUIC may have not some information 
> that
> the TCP/HTTP need?

People tend to confuse QUIC and UDP too much. QUIC more or less uses
UDP as a replacement for IP, which doesn't require privileges, and it
reimplements its own transport stack on top of it, with congestion
control, multiplexing, encryption etc. So it effectively represents a
reliable transport layer for multiple streams based on datagrams below.
UDP is seen as the revolutionary thing when you tell someone that HTTP
will work over UDP but that's absolutely not true, it's almost comparable
to saying that HTTP works over UDP when you're doing it over TLS over
TCP over OpenVPN over UDP.


> As I try to imagine the new design in HAProxy will it look like this?
> 
> TCP(TCP|UDP) HTTP/1.x,HTTP/2,HTTP/3
>   \  /
>  HTTP/TCP/UDP Engine - (TCP|UDP) HTTP/1.x,HTTP/2,HTTP/3
>  (Here happen some magic)
>   /  \
> UDP(TCP|UDP) HTTP/1.x,HTTP/2,HTTP/3
> 
> 
> Really 8-O ?

Precisely not, see above ;-)

> What happens with other protocols like plain TCP/UDP or grpc and so on?

Nothing, no particular change, that's the purpose of the stack you see
above. In fact from the info gathered, gRPC will only require that our
H2 mux supports trailers, which were not needed till they unearthed them.
But I want to see end-to-end H2 before working on trailers, as trailers
alone are useless, but H2 end-to-end is useful!

> Sorry a lot of questions, but you know I'm curious and try to understand some
> parts of the future ;-).

Yes I know, and you're welcome! It's important to discuss design but I
don't want to bore people with it, so it's easier if I have to answer
questions ;-)

> > We still have a lot of work to be done before supporting QUIC 

Re: HTTP/3 | daniel.haxx.se

[Aleksandar Lazic]

Hi Willy.

Am 12.11.2018 um 21:01 schrieb Willy Tarreau:
> On Mon, Nov 12, 2018 at 07:42:21PM +0100, Aleksandar Lazic wrote:
>> Even I agree with you to the point that HAProxy should be able to handle this
>> next upcomming/available technology, I'm not sure if it's really a benefit
>> for us, the enduser.
> 
> Hey, why do you guys imagine we're suffering that much redesigning all the
> connection management in 1.9 and making HTTP processing version-agnostic ?
> :-)

Well because you love to challenge your self, I'm just kidding ;-)

Oh wow this is really a good time to get the hands dirty as QUIC is a major
design change in HTTP, IMHO.

To adopt the matrix from one of the last message, haproxy will be able to do the
conversion in almost every direction?

HTTP/1.x <> HTTP/2
HTTP/2 <> HTTP/3
HTTP/1.x <> HTTP/3
HTTP/3 <> HTTP/3

Is this technically possible as the UDP/QUIC may have not some information that
the TCP/HTTP need?

As I try to imagine the new design in HAProxy will it look like this?

TCP(TCP|UDP) HTTP/1.x,HTTP/2,HTTP/3
  \  /
 HTTP/TCP/UDP Engine - (TCP|UDP) HTTP/1.x,HTTP/2,HTTP/3
 (Here happen some magic)
  /  \
UDP(TCP|UDP) HTTP/1.x,HTTP/2,HTTP/3


Really 8-O ?

What happens with other protocols like plain TCP/UDP or grpc and so on?

Sorry a lot of questions, but you know I'm curious and try to understand some
parts of the future ;-).

> We still have a lot of work to be done before supporting QUIC but we've
> started to have a clear idea how that will fit. The only thing is that
> experience taught us the the devil is in the details, and haproxy has
> accumulated a lot of details over the years.

Oh yes as always, this devil lay there for years and suddenly he jumps out and
say "nana I'm here" 8-O.

> Willy

Cheers
Aleks

Re: HTTP/3 | daniel.haxx.se

[Willy Tarreau]

On Mon, Nov 12, 2018 at 07:42:21PM +0100, Aleksandar Lazic wrote:
> Even I agree with you to the point that HAProxy should be able to handle this
> next upcomming/available technology, I'm not sure if it's really a benefit
> for us, the enduser.

Hey, why do you guys imagine we're suffering that much redesigning all the
connection management in 1.9 and making HTTP processing version-agnostic ?
:-)

We still have a lot of work to be done before supporting QUIC but we've
started to have a clear idea how that will fit. The only thing is that
experience taught us the the devil is in the details, and haproxy has
accumulated a lot of details over the years.

Willy

Re: HTTP/3 | daniel.haxx.se

[Aleksandar Lazic]

Hi Manu.

Even I agree with you to the point that HAProxy should be able to handle this 
next upcomming/available technology, I'm not sure if it's really a benefit for 
us, the enduser.

The future will show it :-)

As I don't want to bother all members on the list with this topic, let's 
discuss it  further off the list, if you like.

Regads aleks


 Ursprüngliche Nachricht 
Von: Emmanuel Hocdet 
Gesendet: 12. November 2018 18:44:40 MEZ
An: Aleksandar Lazic 
CC: haproxy@formilux.org
Betreff: Re: HTTP/3 | daniel.haxx.se


Hi Aleks,

> Le 12 nov. 2018 à 18:02, Aleksandar Lazic  a écrit :
> 
> Hi Manu.
> 
> Am 12.11.2018 um 16:19 schrieb Emmanuel Hocdet:
>> 
>> Hi,
>> 
>> The primary (major) step should be to deal with QUIC transport (over UDP).
>> At the same level as TCP for haproxy?
>> Willy should already have a little idea on it ;-)
> 
> Is then the conclusion for that that haproxy will be able to proxy/load 
> balance UDP?

The only conclusion is that haproxy should be able to proxy QUIC.
From wiki: «  QUIC, Quick UDP Internet Connections, aims to be nearly 
equivalent to an independent TCP 
<https://en.wikipedia.org/wiki/Transmission_Control_Protocol> connection »
It could be see as TCP/2, the connection part is provided by the application. 
For example,  the congestion avoidance algorithms
must be provide into the application space at both endpoints.
A very cool feature is that QUIC can support IP migration.

++
Manu

Re: HTTP/3 | daniel.haxx.se

[Emmanuel Hocdet]

Hi Aleks,

> Le 12 nov. 2018 à 18:02, Aleksandar Lazic  a écrit :
> 
> Hi Manu.
> 
> Am 12.11.2018 um 16:19 schrieb Emmanuel Hocdet:
>> 
>> Hi,
>> 
>> The primary (major) step should be to deal with QUIC transport (over UDP).
>> At the same level as TCP for haproxy?
>> Willy should already have a little idea on it ;-)
> 
> Is then the conclusion for that that haproxy will be able to proxy/load 
> balance UDP?

The only conclusion is that haproxy should be able to proxy QUIC.
From wiki: «  QUIC, Quick UDP Internet Connections, aims to be nearly 
equivalent to an independent TCP 
 connection »
It could be see as TCP/2, the connection part is provided by the application. 
For example,  the congestion avoidance algorithms
must be provide into the application space at both endpoints.
A very cool feature is that QUIC can support IP migration.

++
Manu

Re: HTTP/3 | daniel.haxx.se

[Aleksandar Lazic]

Hi Manu.

Am 12.11.2018 um 16:19 schrieb Emmanuel Hocdet:
> 
> Hi,
> 
> The primary (major) step should be to deal with QUIC transport (over UDP).
> At the same level as TCP for haproxy?
> Willy should already have a little idea on it ;-)

Is then the conclusion for that that haproxy will be able to proxy/load balance 
UDP?

> ++
> Manu
> 
>> Le 11 nov. 2018 à 20:38, Aleksandar Lazic  a écrit :
>>
>> Hi.
>>
>> FYI.
>>
>> Oh no, that was quite fast after HTTP/2
>>
>> https://daniel.haxx.se/blog/2018/11/11/http-3/
>>
>> Regards
>> Aleks
>>
> 
> 

Re: HTTP/3 | daniel.haxx.se

[Emmanuel Hocdet]

Hi,

The primary (major) step should be to deal with QUIC transport (over UDP).
At the same level as TCP for haproxy?
Willy should already have a little idea on it ;-)

++
Manu

> Le 11 nov. 2018 à 20:38, Aleksandar Lazic  a écrit :
> 
> Hi.
> 
> FYI.
> 
> Oh no, that was quite fast after HTTP/2
> 
> https://daniel.haxx.se/blog/2018/11/11/http-3/
> 
> Regards
> Aleks
> 

HTTP/3 | daniel.haxx.se

[Aleksandar Lazic]

Hi.

FYI.

Oh no, that was quite fast after HTTP/2

https://daniel.haxx.se/blog/2018/11/11/http-3/

Regards
Aleks