Hi Matt, I think this discussion rather would belong on the MASQUE list and I guess we already had some of this before chartering MASQUE but let me reply to a couple of comments below.
From: QUIC <[email protected]> on behalf of Matt Joras <[email protected]> Date: Friday, 23. October 2020 at 22:10 To: IETF QUIC WG <[email protected]> Subject: "Proxying" multipath usecases and application scheduling Following up from Martin Thomson's excellent summary of the BoF-esque multipath QUIC interim, I thought I'd start a discussion extracting a couple things I saw as important takeaways from the presentation and subsequent questions. As David noted, there were essentially two categories of usecases presented. The first, embodied by the Apple and Alibaba presentations, were essentially existing applications that can benefit from a transport with the capability to simultaneously utilize multiple paths. For these it seemed everyone agreed that in order for multiple paths to be useful the application has to have input into the scheduling decisions on these paths. For example, the application knows how it wants to trade off latency of delivery and overall throughput of delivery for application data. Application input is needed if and only if there is actually a tradeoff to make. However, there are many scenarios where one can actually chose between a link that is better than the other, regarding both delay and latency, and then the decision is easy and the same for all applications. The second set of usecases utilized multipath "in the network". ATSSS and the two variants of Hybrid Access Networks presented fall into this category. In this scheme application data is transparently proxied over multiple paths. This would include things like TCP flows, which could be intercepted in the traditional way TCP PEPs operate, as well as UDP (including QUIC) datagrams. Intercepting TCP and tunneling is far not the same thing. There are many functions today that tunnel your traffic in some way in some network. Traffic is rerouted or EMCP is used to choose a path. These functions are needed for network management and network resilience and it’s explicitly the role of the end-to-end transport to adapt to changing network conditions and handle these situations. Both types of traffic would be scheduled along the multiple paths towards the "other end" of the proxy, where it presumably re-enters the general Internet towards the destination endpoint. Prior to the meeting this latter set of usecases always felt concerning to me, and the meeting did an excellent job of crystallizing my concerns. As others have said, a great benefit of QUIC has been its ability to "cut through" the meddling of things like TCP PEPs, which are often making transport-level decisions that harm the goals of the endpoint application. While the in-network multipath schemes cannot meddle with QUIC flows as much as TCP PEPs, I am still very concerned about the consequences of utilizing them. Again for me this is by far not the same and threating everything that is done with your flows in the network the same seems just oversimplifying it. We seem to have established that application input is necessary to achieve the benefits from utilizing multiple paths, as there is not one "best" way to schedule data. There are cases where there is actually a best way, whenever one path is better than the other without having to make tradeoff, and as I said in my other mail its usually not only application input that is needed to make a decision. E.g. Christoph brought this example that mobile data should be avoided and of course it also depends on the network condition, and that’s where congestion control comes into play. These proxyng solutions have no way to act on application concerns, and rather have to devise scheduling policies solely from network-level information, ignorant of signals from the application. Does this not inevitably lead to pathological scheduling decisions and applications which are helpless to do anything about it? The obvious counterpoint is that these proxies, through advent of being embedded "in the network", have access to information which the endpoint does not, e.g. about the state of the paths. An argument can be made that because of this they can make proactive decisions, rather than relying on the endpoint to react to changes. This is of course true, but completely ignores the fact that without the application context, it is impossible for the proxy to make an optimal scheduling decision. Again there are scenarios where there is clearly a better path, or where there is a strong desire for the network to avoid usage of a certain. E.g. in the hybrid access case, some operators lease one of the links from another operator. I think it’s understandable that they only offer dual connectivity if they have a way to minimize usage of the more expensive line. There are many scenario where network management decision need to be made independent of application characteristics. And regarding ATSSS is already exists (based on MPTCP for TCP and ATSSS-LL for other traffic) and will be used, similar as other techniques that impact routing. I think it’s understandable that operator try to utilize all available resources and usually that actually means that more capacity and a better is available for the user/the operator’s costumer. We can argue about the best technical way to make use of these resources but a) the final decision about ATSSS will be made in 3GPP and not the IETF and b) having an explicit proxy setup where the client can decide to use as it or not, as it is done also in other masque use case, seems to me actually a good approach (or at minimum better than what we have right now). If the only benefit to running such proxies "in the network" is their extended knowledge of network conditions, wouldn't a better path forward to have endpoints engaging in "native" multipath? To solve the information gap, i.e. the fact that the "network knows more", could we not develop mechanisms for signalling this information proactively to endpoints? I guess you can make the same proposal in the other direction: why cannot the endpoints tell the network what the application needs? I think this is a discussion to have in the APN BoF, however, we had this discussion many times and the short answer is trust. Might probably is a longer answer as well and there is a lot of disagreement. Now these concerns may not sound QUIC-specific, but I think it is still relevant to the working group for the following reason. Usecases are very important towards driving design of something like a multipath protocol. It seems to me that one of the two major categories of usecases for a multipath QUIC is, by design, incompatible with some of the core tenets of QUIC as an application-enhancing transport protocol. I think that is a way to broad statement and far too simplified. There have been so many discussions in the IETF including in the QUIC working and we clearly don’t have agreement about what the right level of cooperation is because there is just not a simple solution. I fully understand the pain we had with TCP. I’ve been working so long on ECN and am still sad how this good idea git killed in its infancy because of a buggy home router implementation that badly interfered. However, I personally also still believe that there is a way to further optimize and better utilize the resource we have if there is some cooperation between the network and endpoints. Anyway that’s my view on a different topic and not the point of this discussion. As Martin said, we obviously can't stop anyone from utilizing multipath in this way, and it of course does not serve as a reason to not design a multipath QUIC, but it seems to me something we should all keep in mind as we consider such designs. I think we should not consider this to make a decision on multipath support in QUIC because as you said you can’t stop it. The only thing you can achieve for ATSSS is that 3GPP may design some proprietary way to realize this function which will further increase complexity in an already complex system. However, that also an different discussion to have. I’m been trying here to mainly explain what 3GPP is doing on ATSSS in order to get this group some understand of what the requirements are. However, I mainly just think we should simply have multipath support in QUIC because a modern protocol should be able to utilize the fact that more and more devices have connectivity to multiple network paths. I think using path migration for handovers is a bit a hack and not the real solution because it’s actually not easy for an endpoint without further network knowledge to detect when it is the right time to switch. Having a more complete multipath feature were you have two congestion controllers and can get measurements from both paths does help that problem and we do have good experience with that for MPTCP. Mirja I am very curious to hear others' opinions in this direction and thanks for sitting through this wall of text. Matt Joras
