Hi, Jana, On Mon, Oct 5, 2020, 21:37 Jana Iyengar <[email protected]> wrote:
> After going through this thread, I'm still finding the framing of the > conversation to be: How do we get "multipath" into QUIC? Perhaps I'm not > being very charitable, but the title of this email thread isn't helping. > Thanks for your helpful thoughts below. Since the title of this thread was mine, it's fair to say that what I was hoping for, was to identify the reasons people were thinking that dropping the multipath charter deliverable was the right thing to do. https://mailarchive.ietf.org/arch/msg/quic/cjjI9-uP6WXVI8p28pcTcq8vS-E/ Obviously, we moved far beyond what I was hoping for, without changing the subject line ... Best, Spencer This framing is problematic and leads to the false scarcity of options we > see in this discussion, but I am still stuck at the premise. Let me explain. > > Transport multipath is not a _problem_, it is a _class of solutions_ that > can be implemented in a transport. Multipath is simply the use of multiple > paths. The manner in which multiple paths are used depends on the problem > you are trying to solve. > > For example, if you want to solve the handoff problem for reliability, > connection migration is a transport-level multipath solution for it. Note > that there are other ways to solve this problem. For example, if the > application is RESTful and simply fetching data (e.g. YouTube), then you > can implement this in the application quite easily. However, for other > applications, such as live transcription, where context is super important > and transport state is tied to application state, maintaining connection > state is necessary. We agreed that the problem was important -- I believe > that was in Melbourne -- and we then went about solving it with connection > migration with which, after many iterations, we now have a fragile peace. > > So echoing what others have been saying on this thread: What is the > problem we are trying to solve? Do we agree on the importance of it? Is > this bandwidth aggregation across WiFi/Cellular (is that an important > enough use case?)? Is it latency reduction for HTTP? Is it increased > reliability? In what ways is it more than connection migration? > > I don't want a laundry list of things that "multipath" can do. But an > analysis of the most important problem and why it needs to be solved. This > is where I'd like to engage, not around adoption of a solution, let alone > adoption of a draft. > > I've still not heard folks rallying around one problem that we need to > solve right now, and I'm not hearing people saying that we need to do > this before we gain any experience with the existing multipath capabilities > of QUICv1. I definitely believe that we ought to wait for experience with > QUICv1. > > Note that the problems MPTCP has solved for TCP can be quite different > from the problems it will need to solve with QUIC. For one, I agree with > Lucas that not talking about streams is like not talking about ordered > delivery with QUIC. Streams are QUIC's primary API model, and we need to > understand how an endpoint does streams across paths. This isn't an MPTCP > problem, but it is one we will need to solve when talking about QUIC. And I > like to remind myself that we spend time in working groups on engineering > and bit-coloring, and I suspect that there's plenty of that in just this > one single problem. > > It's a fallacy to think that "this won't take too much time". While that > is never a good answer to why we should work on anything, it rings > especially hollow in the context of our experience building connection > migration in QUIC: it took us a couple of years and we kept stumbling and > fixing things, especially security vulnerabilities. > > Because I like to hammer the point in: let's motivate the problem first > and agree on the shape of the problem and its importance. And let's see if > we agree to prioritize it/them over the other things that we are already > planning to work on (datagrams, version negotiation, deploying QUICv1 and > digesting experience.) > > And I'll note this because the point has been raised. I agree with > Christian that _if we were to build "general-purpose multipath", I am > thinking of a far simpler (and potentially incremental) design -- maybe > using a single PN space... > > ... but trying to solve the protocol problem is putting the cart before > the horse. We are all good at protocol design, and we are all great at > coloring each bit just the right shade, even though we are terrible at > agreeing on what the right shade is. Before we dive into coloring the bits, > let's please adequately motivate spending engineering time and resources > doing it. > > - jana > > On Mon, Oct 5, 2020 at 6:24 AM Olivier Bonaventure < > [email protected]> wrote: > >> Matt, >> >> >> > >> > Let me try with a simple example on a moving smartphone. The >> > application >> > will send small amounts of data and receive variable amounts of data >> > (depending on the type of requests). >> > >> > I want to start by saying this is a real usecase, and a problem we see >> > very obviously through quality of experience outliers for people using >> > our products. >> > >> > >> > We create a sending and a receiving uniflow on both the Wi-Fi and >> the >> > cellular interface. The smartphone has two sending uniflows and the >> > server as well. >> > >> > To send a short request, the client duplicates it over its two >> sending >> > uniflows since it does not know which of the two uniflows will be >> > the best. >> > >> > To return the response, the server could use the same scheduler if >> the >> > response is short. However, if the response is long, this is not >> very >> > efficient since data is sent over two paths. It could then use both >> > paths to send the data and get the lowest delay to deliver the >> response. >> > This could be modulated by policies if the user pays on a per volume >> > basis over one path and not over the other. >> > >> > This somewhat hand-waves away the scheduling problem. Most Internet >> > traffic flows in the direction of server -> client, where typical >> mobile >> > clients are obviously the ones with potentially multiple interfaces to >> > the Internet. The client also has the most information about the likely >> > quality of the first radio hop into the access network (e.g. signal >> > quality). The client also _may_ know about things like data pricing. >> >> Agreed >> >> > However, the server is the one which is responsible for scheduling most >> > data across these paths. To make good, proactive (rather than reactive) >> > scheduling decisions the server needs to be fed this information as >> > input to its scheduler. This seems a difficult thing to achieve in >> > practice, and without it I wonder whether the complexity of "full" >> > multipath will be worth it until we solve the signalling problem. >> >> There are several techniques that allow the server to learn about the >> different performance of the paths with MPTCP. Those can be applied to >> MPQUIC as well. >> >> If the application uses short requests and short responses, then a >> simple heuristic for the server is to send the response on the path >> where the last packet (data or ack) has been received. The reception of >> a packet is a good indication that a link works. >> >> If the application uses longer responses, then the congestion control >> used by the server over the two paths will enable it to easily find the >> best way to spread the load. If one path has a longer rtt or losses, >> then its congestion window will be slower than the other one and packets >> will naturally flow on the best path, but not only on this path. This >> does not require a specific scheduler and would work better than a >> strict weighted-round-robin scheduler where the client would indicate >> that it wants to receive 2/5 of the bytes over WiFi and 3/5 over cellular. >> >> > What >> > Ian is suggesting above, I think, is essentially an Active/Passive >> > extension to the existing connection migration mechanisms we have >> today. >> > What are your thoughts on this as an initial direction? It would allow >> > operators a way to solve this particular problem with only mild >> > modifications to the core protocol. Said another way, I think in theory >> > an omniscient packet scheduler can make very intelligent decisions >> which >> > would definitely benefit application quality. In practice though I have >> > concerns about how effective these schedulers will be versus the naive >> > approach which could be thought of as "Active/Passive" or "Failover", >> > eschewing bandwidth aggregation entirely. I'd also like to echo what >> > Kazuho said, which is that "Multipath" can mean many things, and I'd >> > prefer we narrow down the problem we want to solve in the WG, which >> will >> > drive our design direction. >> >> When coupled with congestion control, a simple packet scheduler such as >> lowest-rtt first (if paths have the same cost) or priority (for the >> lowest cost path) works well with MPTCP. The same would apply for MPQUIC >> >> > >> > Another example is the hybrid access network scenario with a DSL >> and an >> > LTE path. There, the objective is to send data over the LTE path >> only >> > when the DSL is full. >> > >> > In this case, the solution would differ. The client would first >> create >> > sending and receiving uniflows over the DSL path. It then monitors >> the >> > usage of this path. As long as the DSL is not fully used for some >> > period >> > of time (e.g. one or a few seconds), all data flows over the DSL >> path. >> > Once the DSL path is saturated, the client creates a receiving >> uniflow >> > (and possibly a sending one if the DSL upstream is saturated) over >> the >> > LTE path. The second path can be used to offload traffic. In >> practice, >> > the client and the server use a priority scheduler to always prefer >> the >> > DSL over the LTE path, see >> > >> > >> https://inl.info.ucl.ac.be/publications/increasing-broadband-reach-hybrid-access-networks.html >> > >> > For these usecases, are you imagining that they would largely be used >> > internally for Internet operators? I always struggle with the hybrid >> > access examples, as they seem to assume a lot of knowledge about the >> > underlying networks that typical endpoints can't simply intuit from the >> > ether. The linked paper seems to suggest MPTCP as a proxy solution, >> >> Yes, that's deployed with MPTCP proxies running on home routers. >> >> > which I gather is largely the usecase things like ATSSS have for >> MPQUIC. >> > However, as a mobile application how do I know the DSL link is "full" >> > and thus that I should create a uniflow over the LTE path? The same >> > question applies to the server. It would be awesome if clients and >> > servers had more active information about the underlying network's >> state >> > rather than being reactive, but beyond things like ECN this seems >> > difficult to achieve in practice. If the Hybrid Access Network usecase >> > of multipath presupposes a deployment in an operator's network then I >> > would argue it is somewhat antithetical to the goals of QUIC, which >> > deliberately puts more control at the endpoints. >> >> On endpoints, you can get the same result by having a target bandwidth >> on the Wi-Fi interface. For example, a smartphone application could >> assume that if the current Wi-Fi bandwidth is below x Mbps then it >> should enable the LTE interface to boost a long download. Similarly, a >> videostreaming application could have a target that corresponds to the >> default quality chosen by the user and enable the LTE when it cannot >> receive video at the expected quality. The same applies for a radio >> streaming application. >> >> >> Olivier >> >>
