Am 12.11.2018 um 23:31 schrieb Willy Tarreau: > On Mon, Nov 12, 2018 at 10:52:41PM +0100, Aleksandar Lazic wrote: >> Oh wow this is really a good time to get the hands dirty as QUIC is a major >> design change in HTTP, IMHO. > > Some first approaches were already attempted about one year ago already, > to avoid being later in the party. But just like when we had to work on > H2, trying to stuff this into an existing stream-based component is not > easy and we ended up identifying a lot of lower layers that had to be > changed first, so this work was paused and the protocol changed a lot > during that time. > >> To adopt the matrix from one of the last message, haproxy will be able to do >> the >> conversion in almost every direction? >> >> HTTP/1.x <> HTTP/2 >> HTTP/2 <> HTTP/3 >> HTTP/1.x <> HTTP/3 >> HTTP/3 <> HTTP/3 > > Yes, that's the idea. In fact it will be slightly different, we're > implementing > protocol conversion at the lower layers, to an internal version-agnostic HTTP > representation supposed to accurately represent a message while keeping its > semantics, that we've called HTX. I know that you love ASCII art, so it will > look more or less like this (we'll have to provide a lot of doc but first we > really want to focus on getting the code merged) : > > > +---------------------------------------+ stream > | all HTTP processing | layer > +---------------------------------------+ > ^ ^ ^ ^ > HTX | HTX | HTX | HTX | normalised > v v v v interface > +------+ +--------+ +--------+ +--------+ > |applet| | HTTP/1 | | HTTP/2 | | HTTP/3 | whatever layer (called mux for > now > +------+ +--------+ +--------+ +--------+ but may change once we have > others, > cache | | | | | could be presentation in OSI) > stats | +----------+ | +------+ > Lua svc | | TLS | | | QUIC | transport layer > | +----------+ | +------+ > | | | | > +---------------------+ +-----+ > | TCP/Unix/socketpair | | UDP | control layer > +---------------------+ +-----+ > | | > +----------------------+ > | file descriptor | socket layer > +----------------------+ > | > +-----------+ > | operating | > | system | > +-----------+ > > > It's really over-simplified since several layers above in fact have multiple > upper arrows as they are multiplexed, but it's to explain more or less how > stuff gets stacked. And since there are always transition periods, you have > multiple protocols possible between each layer, otherwise it wouldn't be > fun. You have coded in the very old version where the file descriptors > were directly present in the stream layer. As you can see, over a decade > a lot of new layers have been built between the operating system and the > streams, without even losing features nor compatibility. That's where the > real challenges are.
Thank you for the picture, you know me too good 8-). I follow the development a "little" bit, and it's amazing how you and your team was able to rewrite a lot of haproxy and keep a high level of compatibility and stability. BTW: I loved to see that std* logger is now there ;-) >> Is this technically possible as the UDP/QUIC may have not some information >> that >> the TCP/HTTP need? > > People tend to confuse QUIC and UDP too much. QUIC more or less uses > UDP as a replacement for IP, which doesn't require privileges, and it > reimplements its own transport stack on top of it, with congestion > control, multiplexing, encryption etc. So it effectively represents a > reliable transport layer for multiple streams based on datagrams below. > UDP is seen as the revolutionary thing when you tell someone that HTTP > will work over UDP but that's absolutely not true, it's almost comparable > to saying that HTTP works over UDP when you're doing it over TLS over > TCP over OpenVPN over UDP. Hm, sound complicated. I think to debug this beast isn't that "easy" as for now with `curl -v ...`, even QUIC is planned for curl ;-) >> As I try to imagine the new design in HAProxy will it look like this? >> >> TCP (TCP|UDP) HTTP/1.x,HTTP/2,HTTP/3 >> \ / >> HTTP/TCP/UDP Engine - (TCP|UDP) HTTP/1.x,HTTP/2,HTTP/3 >> (Here happen some magic) >> / \ >> UDP (TCP|UDP) HTTP/1.x,HTTP/2,HTTP/3 >> >> >> Really 8-O ? > > Precisely not, see above ;-) > >> What happens with other protocols like plain TCP/UDP or grpc and so on? > > Nothing, no particular change, that's the purpose of the stack you see > above. In fact from the info gathered, gRPC will only require that our > H2 mux supports trailers, which were not needed till they unearthed them. Full ACK. > But I want to see end-to-end H2 before working on trailers, as trailers > alone are useless, but H2 end-to-end is useful! Full ACK. >> Sorry a lot of questions, but you know I'm curious and try to understand some >> parts of the future ;-). > > Yes I know, and you're welcome! It's important to discuss design but I > don't want to bore people with it, so it's easier if I have to answer > questions ;-) Thank you for your patience and answering questions ;-) >>> We still have a lot of work to be done before supporting QUIC but we've >>> started to have a clear idea how that will fit. The only thing is that >>> experience taught us the the devil is in the details, and haproxy has >>> accumulated a lot of details over the years. >> >> Oh yes as always, this devil lay there for years and suddenly he jumps out >> and >> say "nana I'm here" 8-O. > > It's even worse. Sometimes we contemplate the thing and start to ask "hey, > doesn't this look like a devil? Maybe we should destroy it right now", and > afterwards we think "sadly, this devil was still slightly useful, now we > have to repair even more than expected". Stay tuned! Always ;-) > Cheers, > Willy Cheers Aleks
