So I can’t speak of all use cases, but take on this is two fold: 1. migration doesn’t currently support symmertric p2p server connections when you want robust high performance datacenter connections. For example, it is common for packet loss during equipment upgrades, and that could be mitigated by multipath without much effort on the application.
2. Having two separate connections to deal with multi-path makes it prohibitively difficult to coordinate a single stream For example a messing service could increase performance on multi-path, but ensuring in-order deliver on two connections require a new kind of application protocol on the both ends. Message ordering in actor bases systems can be very important in simplifying algorithm complexity such as transactions: If you get message A and but have not seen message B, then you know that it is safe to perform task X. This is really hard to do over multiple connections. You have to repeat a lot of the logic that is iinside QUIC or TCP. Mikkel On 7 November 2020 at 01.35.42, Christian Huitema ([email protected]) wrote: On 11/6/2020 3:18 PM, Spencer Dawkins at IETF wrote: Hi, Mikkel, On Fri, Nov 6, 2020 at 3:34 PM Mikkel Fahnøe Jørgensen <[email protected]> wrote: > Below > > On 6 November 2020 at 20.02.31, Spencer Dawkins at IETF ( > [email protected]) wrote: > > I'm not sure what the protocol difference is between the two cases. I > understand the difference between sending on one path and sending on > another path, but once you've started sending on two (or more) paths, I'm > not sure what the protocol requirement is to stop sending on one of them. > > I'm even more confused if we're talking about sending datagrams, rather > than reliable streams. > > I understand why a scheduler might want to limit itself to sending on one > path in normal operation, but I'm trying to understand a different point. > > Largely, the machinery that has to do with loss detection, bandwidth > estimation, congestion control, packet numbering is all set up to deal with > a single path in QUIC v1. When you migrate, you accept do a transactional > handover so both parties agree on the change, and you are very careful not > to get out of sync by one party changing path too frequently, and then > there are some issues with dealing with adversaries. You still allow for a > few packets on the old path, but but you can strictly place them as > originating from a point in the past (sort of like ballots being stamped no > later than election day). > > With multiple paths, you have to decide have you number packets on > different paths and how you estimate loss, etc., and also more esoteric > details like spin bits and privacy in this context. > > Ordering packets on multiple paths in the same stream is of course more of > a challenge in this case, as you point out, since packet numbering alone > might no longer be enough. But as stated, it there is a lot more to it than > just sequencing frames. Sequencing is a relatively small problem. The > bigger issues also apply to unordered datagrams. > Thanks - that was helpful, at least to me. Note there is no guarantee in QUIC that packets will be delivered in the order in which they are sent. That would require head of line blocking, and QUIC is specifically design to avoid that. So you could have three packets: PN 101: data on Stream 8, offset 0, length 1000 PN 102: data on Stream 12, offset 0, length 1000 PN 103: data on Stream 8, offset 1000, length 500, and data on Stream 12, offset 1000, length 500. If packet 103 arrives after packet 101 but before packet 102 the data on stream 8 will be delivered because it is in sequence, but most stack will queue the data on stream 12 until the packet 102 arrives or its content is repeated. I say most, because some applications may support out of order delivery. Having packets arrive "out of order" on multiple paths is merely a generalization of that feature. Of course, these problems are far from unsolvable, it just takes som work. > The question is if it is worth the effort, given what you can already do > without true multipath, and if it is, exactly what are the important use > cases. And more remotely, if the additional API complexity with priotising > paths work out in praxis. > I agree. That's the question indeed. Do we shove the multipath support in the QUIC transport, or do we just tell applications to support multiple connections. The more you ask the application to do something smart about routing some traffic one way and some other another way, the more the "multiple connection" solution seems appealing. Transport level multipath makes the most sense when the paths are roughly equivalent, or at least when the application policy can be described very simply. -- Christian Huitema
