Am 12.11.2018 um 23:31 schrieb Willy Tarreau:
> On Mon, Nov 12, 2018 at 10:52:41PM +0100, Aleksandar Lazic wrote:
>> Oh wow this is really a good time to get the hands dirty as QUIC is a major
>> design change in HTTP, IMHO.
> 
> Some first approaches were already attempted about one year ago already,
> to avoid being later in the party. But just like when we had to work on
> H2, trying to stuff this into an existing stream-based component is not
> easy and we ended up identifying a lot of lower layers that had to be
> changed first, so this work was paused and the protocol changed a lot
> during that time.
> 
>> To adopt the matrix from one of the last message, haproxy will be able to do 
>> the
>> conversion in almost every direction?
>>
>> HTTP/1.x <> HTTP/2
>> HTTP/2 <> HTTP/3
>> HTTP/1.x <> HTTP/3
>> HTTP/3 <> HTTP/3
> 
> Yes, that's the idea. In fact it will be slightly different, we're 
> implementing
> protocol conversion at the lower layers, to an internal version-agnostic HTTP
> representation supposed to accurately represent a message while keeping its
> semantics, that we've called HTX. I know that you love ASCII art, so it will
> look more or less like this (we'll have to provide a lot of doc but first we
> really want to focus on getting the code merged) :
> 
> 
>    +---------------------------------------+ stream
>    |               all HTTP processing     | layer
>    +---------------------------------------+
>        ^         ^         ^          ^
>    HTX |     HTX |     HTX |      HTX |      normalised
>        v         v         v          v      interface
>    +------+ +--------+ +--------+ +--------+
>    |applet| | HTTP/1 | | HTTP/2 | | HTTP/3 | whatever layer (called mux for 
> now
>    +------+ +--------+ +--------+ +--------+ but may change once we have 
> others,
>     cache     |    |     |    |        |     could be presentation in OSI)
>     stats     | +----------+  |    +------+
>     Lua svc   | |    TLS   |  |    | QUIC |  transport layer
>               | +----------+  |    +------+ 
>               |       |       |        |
>             +---------------------+ +-----+
>             | TCP/Unix/socketpair | | UDP |  control layer
>             +---------------------+ +-----+
>                       |                |
>                    +----------------------+
>                    |    file descriptor   |  socket layer
>                    +----------------------+
>                                |
>                          +-----------+
>                          | operating |
>                          |  system   |
>                          +-----------+
> 
> 
> It's really over-simplified since several layers above in fact have multiple
> upper arrows as they are multiplexed, but it's to explain more or less how
> stuff gets stacked. And since there are always transition periods, you have
> multiple protocols possible between each layer, otherwise it wouldn't be
> fun.  You have coded in the very old version where the file descriptors
> were directly present in the stream layer. As you can see, over a decade
> a lot of new layers have been built between the operating system and the
> streams, without even losing features nor compatibility. That's where the
> real challenges are.

Thank you for the picture, you know me too good 8-).

I follow the development a "little" bit, and it's amazing how you and your team
was able to rewrite a lot of haproxy and keep a high level of compatibility and
stability.

BTW: I loved to see that std* logger is now there ;-)

>> Is this technically possible as the UDP/QUIC may have not some information 
>> that
>> the TCP/HTTP need?
> 
> People tend to confuse QUIC and UDP too much. QUIC more or less uses
> UDP as a replacement for IP, which doesn't require privileges, and it
> reimplements its own transport stack on top of it, with congestion
> control, multiplexing, encryption etc. So it effectively represents a
> reliable transport layer for multiple streams based on datagrams below.
> UDP is seen as the revolutionary thing when you tell someone that HTTP
> will work over UDP but that's absolutely not true, it's almost comparable
> to saying that HTTP works over UDP when you're doing it over TLS over
> TCP over OpenVPN over UDP.

Hm, sound complicated.

I think to debug this beast isn't that "easy" as for now with `curl -v ...`,
even QUIC is planned for curl ;-)

>> As I try to imagine the new design in HAProxy will it look like this?
>>
>> TCP                                (TCP|UDP) HTTP/1.x,HTTP/2,HTTP/3
>>   \                              /
>>      HTTP/TCP/UDP Engine         - (TCP|UDP) HTTP/1.x,HTTP/2,HTTP/3
>>      (Here happen some magic)
>>   /                              \
>> UDP                                (TCP|UDP) HTTP/1.x,HTTP/2,HTTP/3
>>
>>
>> Really 8-O ?
> 
> Precisely not, see above ;-)
> 
>> What happens with other protocols like plain TCP/UDP or grpc and so on?
> 
> Nothing, no particular change, that's the purpose of the stack you see
> above. In fact from the info gathered, gRPC will only require that our
> H2 mux supports trailers, which were not needed till they unearthed them.

Full ACK.

> But I want to see end-to-end H2 before working on trailers, as trailers
> alone are useless, but H2 end-to-end is useful!

Full ACK.

>> Sorry a lot of questions, but you know I'm curious and try to understand some
>> parts of the future ;-).
> 
> Yes I know, and you're welcome! It's important to discuss design but I
> don't want to bore people with it, so it's easier if I have to answer
> questions ;-)

Thank you for your patience and answering questions ;-)

>>> We still have a lot of work to be done before supporting QUIC but we've
>>> started to have a clear idea how that will fit. The only thing is that
>>> experience taught us the the devil is in the details, and haproxy has
>>> accumulated a lot of details over the years.
>>
>> Oh yes as always, this devil lay there for years and suddenly he jumps out 
>> and
>> say "nana I'm here" 8-O.
> 
> It's even worse. Sometimes we contemplate the thing and start to ask "hey,
> doesn't this look like a devil? Maybe we should destroy it right now", and
> afterwards we think "sadly, this devil was still slightly useful, now we
> have to repair even more than expected". Stay tuned!

Always ;-)

> Cheers,
> Willy

Cheers
Aleks

Reply via email to