Re: [Bloat] [Ecn-sane] [iccrg] Fwd: [tcpPrague] Implementation and experimentation of TCP Prague/L4S hackaton at IETF104
On Fri, Mar 15, 2019 at 9:04 PM Jonathan Morton wrote: > > > On 15 Mar, 2019, at 7:01 pm, David P. Reed wrote: > > > > > Next up for sce was going to be to find if dctcp could be modified to > > > use it. Also, bittorrent. > > > > YES! I endorse this message. > > Actually, today I was still figuring out how to fit it to CUBIC. I *think* > I've succeeded… > > https://github.com/dtaht/bufferbloat-rfcs/blob/master/sce/ELR%20Proposal%203%20(TCP).txt > > I recall DCTCP as being singularly built around a distinct setpoint from what > either Classic ECN or SCE expects. So I think I might look at LEDBAT first. LEDBAT as defined by the ietf is a waste of time, as is it did not deploy. In terms of dealing with bittorrent, I recommend you look over the UDP implementation in libutp, and in particular the embedded implementation in "transmission", which is easier to play with first. Along the way, getting setsockopt to set the diffserv bits right finally for libutp would be nice. > - Jonathan Morton > > ___ > Ecn-sane mailing list > ecn-s...@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/ecn-sane -- Dave Täht CTO, TekLibre, LLC http://www.teklibre.com Tel: 1-831-205-9740 ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] [Ecn-sane] [iccrg] Fwd: [tcpPrague] Implementation and experimentation of TCP Prague/L4S hackaton at IETF104
> On 15 Mar, 2019, at 7:01 pm, David P. Reed wrote: > > > Next up for sce was going to be to find if dctcp could be modified to > > use it. Also, bittorrent. > > YES! I endorse this message. Actually, today I was still figuring out how to fit it to CUBIC. I *think* I've succeeded… https://github.com/dtaht/bufferbloat-rfcs/blob/master/sce/ELR%20Proposal%203%20(TCP).txt I recall DCTCP as being singularly built around a distinct setpoint from what either Classic ECN or SCE expects. So I think I might look at LEDBAT first. - Jonathan Morton ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] [Ecn-sane] [iccrg] Fwd: [tcpPrague] Implementation and experimentation of TCP Prague/L4S hackaton at IETF104
> On 16 Mar, 2019, at 1:43 am, David P. Reed wrote: > > Now the other thing that is crucial is that the optimal state almost all of > the time of every link in the network is that a utilization far from max > capacity. The reason for this is the fact that the Internet (like almost all > networks) is bursty and fractal. That can be said about some types of links but not others. Last-mile links in particular are often saturated by their users' individual traffic for minutes or even hours at a time, especially the slower link technologies such as ADSL and 4G. The user wants their hundred-gigabyte game update installed as soon as possible, even if they only have 10Mbps to suck it through, and they still want to use their connection for other things while they wait. And this is not unreasonable; I do it regularly. At peak times, ISP backhaul capacity can often be enough of a bottleneck for users to notice the congestion and induced latency; it is far from the case that all ISPs worldwide massively over-provision their networks to avoid routine congestion, even in modern technologically advanced nations. There are remote islands where hundreds or thousands of users must share a single satellite or microwave uplink. Cell towers are also a shared medium with decidedly finite capacity. Only core networks, and the backhaul networks of certain particularly conscientious ISPs, can typically be described as congestion-free. And that is why we discuss AQM and ECN in such detail in the first place; without congestion, they wouldn't be required. The extent to which traffic classification is needed on particular types of link can be debated. It could fairly be argued that we've done remarkably well without the benefit of a functioning Diffserv ecosystem, so there is no particular urgency to create one. At the same time, it's worth discussing *why* Diffserv fails to do its intended job, and how a better system *could* be designed, because that may reveal lessons for the future. I will say this: there are times, even with the benefit of everything Cake does for me, when I would prefer that BitTorrent traffic would automatically defer to other stuff (as it was supposedly designed to). Several game updaters, including Wargaming.net's, use BitTorrent under the skin - opening and using several hundred flows in parallel, and thereby swamping any other traffic going to the same host. It would be very straightforward for them to mark all that traffic as Minimum Cost, while their games themselves use Minimum Latency for battle traffic. Minimum Cost is a natural choice for any transport using LEDBAT, or with similarly altruistic philosophy. Minimum Latency is a natural choice for any application requiring realtime response - games, VoIP, remote shells. Minimum Loss is a natural choice for applications involved in network control, where dropped packets could have a much greater than normal impact on overall network performance. Maximum Throughput is a natural choice for general-purpose applications not fitting any of the above. Pricing is not required. Making the wrong choice will simply make your application perform badly on a loaded network, as the bottleneck link optimises for the thing you told it to optimise for, rather than for what you actually want. That's all that's really needed. - Jonathan Morton ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] [Ecn-sane] [iccrg] Fwd: [tcpPrague] Implementation and experimentation of TCP Prague/L4S hackaton at IETF104
How many applications used by normal users have "admin" privileges? The Browser? Email? FTP? -Original Message- From: "Dave Taht" Sent: Friday, March 15, 2019 4:31pm To: "Jonathan Foulkes" Cc: ecn-s...@lists.bufferbloat.net, "bloat" Subject: Re: [Ecn-sane] [Bloat] [iccrg] Fwd: [tcpPrague] Implementation and experimentation of TCP Prague/L4S hackaton at IETF104 On Fri, Mar 15, 2019 at 1:28 PM Jonathan Foulkes wrote: > > All this discussion of DSCP marking brings to mind what happened on the > Windows platform, where the OS had to suppress ALL DSCP marks, as app authors > were trying to game the system. > And even if not trying to ‘game’ it, they have non-obvious reasons why they > don’t mark traffic how one would expect. Example: > > I know an engineer who works at a cloud-storage solution company, and I asked > why a long-standing customer request for DSCP marking (as bulk) was not > implemented. His answer was they’d never do that, as that would impact > benchmarks against their competitors for which service syncs faster. > > Which brings me to a question: Is anyone aware of an easy to use Windows app > that will allow the user to select an application and tell the OS to mark the > traffic (all or by port) with a user selected DSCP level? > There are many guides on using regedit and other error-prone (and geek-only) > means of doing this, but is there a simple Windows 10 home app? When I last tried it (years ago), in order to set the tos bits, an application merely had to have admin privs. > Now that Cake is out there with simple DiffServ3 support, it would be nice to > lower the priority of cloud-storage services and other bulk traffic by > correctly marking it at the origin. > > Cheers, > > Jonathan Foulkes > > > > On Mar 15, 2019, at 3:32 PM, Jonathan Morton wrote: > > > >> On 15 Mar, 2019, at 8:36 pm, Mikael Abrahamsson wrote: > >> > >> Having a "lower-than-best-effort" diffserve codepoint might work, because > >> it means worse treatment, not preferential treatment. > >> > >> The problem with having DSCP CPs that indicate preferential treatment is > >> typically a ddos magnet. > > > > This is true, and also why I feel that just 2 bits should be sufficient for > > Diffserv (rather than 6). They are sufficient to express four different > > optimisation targets: > > > > 0: Maximum Throughput (aka Best Effort) > > 1: Minimum Cost (aka Least Effort) > > 2: Minimum Latency (aka Maximum Responsiveness) > > 3: Minimum Loss (aka Maximum Reliability) > > > > It is legitimate for traffic to request any of these four optimisations, > > with the explicit tradeoff of *not* necessarily getting optimisation in the > > other three dimensions. > > > > The old TOS spec erred in specifying 4 non-exclusive bits to express this, > > in addition to 3 bits for a telegram-office style "priority level" (which > > was very much ripe for abuse if not strictly admission-controlled). TOS was > > rightly considered a mess, but was replaced with Diffserv which was far too > > loose a spec to be useful in practice. > > > > But that's a separate topic from ECN per se. > > > > - Jonathan Morton > > > > ___ > > Bloat mailing list > > Bloat@lists.bufferbloat.net > > https://lists.bufferbloat.net/listinfo/bloat > > ___ > Bloat mailing list > Bloat@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/bloat -- Dave Täht CTO, TekLibre, LLC http://www.teklibre.com Tel: 1-831-205-9740 ___ Ecn-sane mailing list ecn-s...@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/ecn-sane___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] [Ecn-sane] [iccrg] Fwd: [tcpPrague] Implementation and experimentation of TCP Prague/L4S hackaton at IETF104
My point is that the argument for doing such balancing is that somehow ISPs at the entry points (representing somehow the goals of source and destination of each flow) will classify their flows correctly based on some criterion, and not select the option that allows them to "beat" all the others, which then causes them be "game theoreitically" incented to screw up the labeling. The business argument that the users at both ends will choose the rignt labels is that they are responsive to price signals in a very sensitive way that will get them to mark things correctly. (that includes, by the way, the Internet Access Providers, if they take over the labeling job and force their choice on their users, because they become the "endpoint") So if pricing mechanisms don't work to incent labeling correctly, it does not matter that there exists an optimum that can be achieved by an Oracle who fully decides the settings on all packets of all protocols ever to be invented. And that's why I brought up the issue of pricing and economics, which sadly really affect any kind of queue management. That's why pricing becomes a practical issue, and issues of "utility" to the users become important. Now the other thing that is crucial is that the optimal state almost all of the time of every link in the network is that a utilization far from max capacity. The reason for this is the fact that the Internet (like almost all networks) is bursty and fractal. The law of large numbers doesn't smooth traffic volume over any timescale (that's the sense of fractalness here). There is no statistical smoothing of load - there are rare high peaks on some links but most links are underutilized, *if you want all the protocols currently used in the Internet to make users happy with minimal time-to-task-completion* (response time at the scale that matters for the particular user's needs at that moment). So if most links are uncongested most of the time (and they should be if the folks maintaining the subnets are all doing their job by growing links that are congested to handle more traffic), then congestion management is only a peak load problem, and only affects things a small percentage of the time. This is very, very different from the typical "benchmark" case, which focuses only on dealing with peak loads, which are transient in the real world. Most "benchmarks" make the strange and unrealistic assumption that overload is steady state, and that users themselves don't give up and stop using an overloaded system. -Original Message- From: "Jonathan Morton" Sent: Friday, March 15, 2019 4:13pm To: "David P. Reed" Cc: "Mikael Abrahamsson" , ecn-s...@lists.bufferbloat.net, "bloat" Subject: Re: [Bloat] [Ecn-sane] [iccrg] Fwd: [tcpPrague] Implementation and experimentation of TCP Prague/L4S hackaton at IETF104 > On 15 Mar, 2019, at 9:44 pm, David P. Reed wrote: > > pricing (even dynamic pricing) of different qualities of service is unstable An interesting result, but I should note that the four-way optimisation system I described doesn't rely on pricing, only a sufficiently correct implementation of those optimisations at enough bottlenecks to make it worthwhile for applications to mark their traffic appropriately. The technology exists to do so, but is not standardised in a way that makes it usable. - Jonathan Morton ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] [iccrg] Fwd: [tcpPrague] Implementation and experimentation of TCP Prague/L4S hackaton at IETF104
Hi, Dave, strangely it looks like nobody has been copying TSVWG on this thread, even though that is where the L4S work is happening in the IETF! :) I just wanted to respond to one part of your message since I am currently acting as document shepherd for the L4S drafts in TSVWG, and it seems like maybe you don't know where to find all of the IETF materials in order to participate (based on the "stalled out indefinitely" comment below). So in case it's helpful here are some pointers to where things are kept. The 3 base L4S documents were adopted in the TSVWG WG, and have been regularly updated. You can find the charter and milestone dates (currently June 2019) quite easily on the WG datatracker page: https://datatracker.ietf.org/group/tsvwg/about/ and of course the "Documents" tab there takes you to the copies of all the documents. There have been updates on L4S and presentations/discussions at the IETF meetings (with minutes and charts posted to the proceedings), as well as L4S draft reviews and comment threads on the TSVWG list whose archives are under "List archive". You can find meeting minutes and slide decks also readily linked from that WG page: https://datatracker.ietf.org/group/tsvwg/meetings/ There are source code repositories, papers, videos, etc. that the proponents have also made very easy to find, e.g. https://riteproject.eu/dctth/#code (and as linked in meeting materials). Since we (TSVWG chairs) are looking for inputs towards WGLC readiness to proceed on these, hopefully this information is helpful for you. On 3/15/2019 6:46 AM, Dave Taht wrote: > ... > > Some background to this: after the L4S/TCP Prague/and dualpi experiments appeared stalled out indefinitely in the IETF, and with our own frustration with IETF processes, bufferbloat.net project members publicly formed our own working group to look into the problems with ecn, back in august of last year. ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] [Ecn-sane] [iccrg] Fwd: [tcpPrague] Implementation and experimentation of TCP Prague/L4S hackaton at IETF104
On Fri, Mar 15, 2019 at 1:28 PM Jonathan Foulkes wrote: > > All this discussion of DSCP marking brings to mind what happened on the > Windows platform, where the OS had to suppress ALL DSCP marks, as app authors > were trying to game the system. > And even if not trying to ‘game’ it, they have non-obvious reasons why they > don’t mark traffic how one would expect. Example: > > I know an engineer who works at a cloud-storage solution company, and I asked > why a long-standing customer request for DSCP marking (as bulk) was not > implemented. His answer was they’d never do that, as that would impact > benchmarks against their competitors for which service syncs faster. > > Which brings me to a question: Is anyone aware of an easy to use Windows app > that will allow the user to select an application and tell the OS to mark the > traffic (all or by port) with a user selected DSCP level? > There are many guides on using regedit and other error-prone (and geek-only) > means of doing this, but is there a simple Windows 10 home app? When I last tried it (years ago), in order to set the tos bits, an application merely had to have admin privs. > Now that Cake is out there with simple DiffServ3 support, it would be nice to > lower the priority of cloud-storage services and other bulk traffic by > correctly marking it at the origin. > > Cheers, > > Jonathan Foulkes > > > > On Mar 15, 2019, at 3:32 PM, Jonathan Morton wrote: > > > >> On 15 Mar, 2019, at 8:36 pm, Mikael Abrahamsson wrote: > >> > >> Having a "lower-than-best-effort" diffserve codepoint might work, because > >> it means worse treatment, not preferential treatment. > >> > >> The problem with having DSCP CPs that indicate preferential treatment is > >> typically a ddos magnet. > > > > This is true, and also why I feel that just 2 bits should be sufficient for > > Diffserv (rather than 6). They are sufficient to express four different > > optimisation targets: > > > > 0: Maximum Throughput (aka Best Effort) > > 1: Minimum Cost (aka Least Effort) > > 2: Minimum Latency (aka Maximum Responsiveness) > > 3: Minimum Loss (aka Maximum Reliability) > > > > It is legitimate for traffic to request any of these four optimisations, > > with the explicit tradeoff of *not* necessarily getting optimisation in the > > other three dimensions. > > > > The old TOS spec erred in specifying 4 non-exclusive bits to express this, > > in addition to 3 bits for a telegram-office style "priority level" (which > > was very much ripe for abuse if not strictly admission-controlled). TOS > > was rightly considered a mess, but was replaced with Diffserv which was far > > too loose a spec to be useful in practice. > > > > But that's a separate topic from ECN per se. > > > > - Jonathan Morton > > > > ___ > > Bloat mailing list > > Bloat@lists.bufferbloat.net > > https://lists.bufferbloat.net/listinfo/bloat > > ___ > Bloat mailing list > Bloat@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/bloat -- Dave Täht CTO, TekLibre, LLC http://www.teklibre.com Tel: 1-831-205-9740 ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] [Ecn-sane] [iccrg] Fwd: [tcpPrague] Implementation and experimentation of TCP Prague/L4S hackaton at IETF104
All this discussion of DSCP marking brings to mind what happened on the Windows platform, where the OS had to suppress ALL DSCP marks, as app authors were trying to game the system. And even if not trying to ‘game’ it, they have non-obvious reasons why they don’t mark traffic how one would expect. Example: I know an engineer who works at a cloud-storage solution company, and I asked why a long-standing customer request for DSCP marking (as bulk) was not implemented. His answer was they’d never do that, as that would impact benchmarks against their competitors for which service syncs faster. Which brings me to a question: Is anyone aware of an easy to use Windows app that will allow the user to select an application and tell the OS to mark the traffic (all or by port) with a user selected DSCP level? There are many guides on using regedit and other error-prone (and geek-only) means of doing this, but is there a simple Windows 10 home app? Now that Cake is out there with simple DiffServ3 support, it would be nice to lower the priority of cloud-storage services and other bulk traffic by correctly marking it at the origin. Cheers, Jonathan Foulkes > On Mar 15, 2019, at 3:32 PM, Jonathan Morton wrote: > >> On 15 Mar, 2019, at 8:36 pm, Mikael Abrahamsson wrote: >> >> Having a "lower-than-best-effort" diffserve codepoint might work, because it >> means worse treatment, not preferential treatment. >> >> The problem with having DSCP CPs that indicate preferential treatment is >> typically a ddos magnet. > > This is true, and also why I feel that just 2 bits should be sufficient for > Diffserv (rather than 6). They are sufficient to express four different > optimisation targets: > > 0: Maximum Throughput (aka Best Effort) > 1: Minimum Cost (aka Least Effort) > 2: Minimum Latency (aka Maximum Responsiveness) > 3: Minimum Loss (aka Maximum Reliability) > > It is legitimate for traffic to request any of these four optimisations, with > the explicit tradeoff of *not* necessarily getting optimisation in the other > three dimensions. > > The old TOS spec erred in specifying 4 non-exclusive bits to express this, in > addition to 3 bits for a telegram-office style "priority level" (which was > very much ripe for abuse if not strictly admission-controlled). TOS was > rightly considered a mess, but was replaced with Diffserv which was far too > loose a spec to be useful in practice. > > But that's a separate topic from ECN per se. > > - Jonathan Morton > > ___ > Bloat mailing list > Bloat@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/bloat ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] [Ecn-sane] [iccrg] Fwd: [tcpPrague] Implementation and experimentation of TCP Prague/L4S hackaton at IETF104
> On 15 Mar, 2019, at 9:44 pm, David P. Reed wrote: > > pricing (even dynamic pricing) of different qualities of service is unstable An interesting result, but I should note that the four-way optimisation system I described doesn't rely on pricing, only a sufficiently correct implementation of those optimisations at enough bottlenecks to make it worthwhile for applications to mark their traffic appropriately. The technology exists to do so, but is not standardised in a way that makes it usable. - Jonathan Morton ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] [Ecn-sane] [iccrg] Fwd: [tcpPrague] Implementation and experimentation of TCP Prague/L4S hackaton at IETF104
Just to throw in one more thing not well understood by engineers. Economists I have discussed this with (real ones, not fringe right-wing true believers that the market "just works"), have observed that pricing (even dynamic pricing) of different qualities of service is unstable and extremely unlikely to reflect the correct price for the particular utility of the achieved service quality. The point of that observation is that even a simple 2 classes of service system (so-called Paris Metro Pricing) is unstable, such that users of such a system will not be encouraged to set the priorities/service types to make system optimal or stable. I can explain more, but the end user doesn't benefit from multiple choices of class of service at the packet level. -Original Message- From: "Jonathan Morton" Sent: Friday, March 15, 2019 3:32pm To: "Mikael Abrahamsson" Cc: "David P. Reed" , ecn-s...@lists.bufferbloat.net, "bloat" Subject: Re: [Bloat] [Ecn-sane] [iccrg] Fwd: [tcpPrague] Implementation and experimentation of TCP Prague/L4S hackaton at IETF104 > On 15 Mar, 2019, at 8:36 pm, Mikael Abrahamsson wrote: > > Having a "lower-than-best-effort" diffserve codepoint might work, because it > means worse treatment, not preferential treatment. > > The problem with having DSCP CPs that indicate preferential treatment is > typically a ddos magnet. This is true, and also why I feel that just 2 bits should be sufficient for Diffserv (rather than 6). They are sufficient to express four different optimisation targets: 0: Maximum Throughput (aka Best Effort) 1: Minimum Cost (aka Least Effort) 2: Minimum Latency (aka Maximum Responsiveness) 3: Minimum Loss (aka Maximum Reliability) It is legitimate for traffic to request any of these four optimisations, with the explicit tradeoff of *not* necessarily getting optimisation in the other three dimensions. The old TOS spec erred in specifying 4 non-exclusive bits to express this, in addition to 3 bits for a telegram-office style "priority level" (which was very much ripe for abuse if not strictly admission-controlled). TOS was rightly considered a mess, but was replaced with Diffserv which was far too loose a spec to be useful in practice. But that's a separate topic from ECN per se. - Jonathan Morton ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] [Ecn-sane] [iccrg] Fwd: [tcpPrague] Implementation and experimentation of TCP Prague/L4S hackaton at IETF104
> On 15 Mar, 2019, at 8:36 pm, Mikael Abrahamsson wrote: > > Having a "lower-than-best-effort" diffserve codepoint might work, because it > means worse treatment, not preferential treatment. > > The problem with having DSCP CPs that indicate preferential treatment is > typically a ddos magnet. This is true, and also why I feel that just 2 bits should be sufficient for Diffserv (rather than 6). They are sufficient to express four different optimisation targets: 0: Maximum Throughput (aka Best Effort) 1: Minimum Cost (aka Least Effort) 2: Minimum Latency (aka Maximum Responsiveness) 3: Minimum Loss (aka Maximum Reliability) It is legitimate for traffic to request any of these four optimisations, with the explicit tradeoff of *not* necessarily getting optimisation in the other three dimensions. The old TOS spec erred in specifying 4 non-exclusive bits to express this, in addition to 3 bits for a telegram-office style "priority level" (which was very much ripe for abuse if not strictly admission-controlled). TOS was rightly considered a mess, but was replaced with Diffserv which was far too loose a spec to be useful in practice. But that's a separate topic from ECN per se. - Jonathan Morton ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] [Ecn-sane] [iccrg] Fwd: [tcpPrague] Implementation and experimentation of TCP Prague/L4S hackaton at IETF104
Hi Mikael, > On Mar 15, 2019, at 19:36, Mikael Abrahamsson wrote: > > On Fri, 15 Mar 2019, David P. Reed wrote: > >> So if the responsible network engineers in the carriers cannot agree on >> anything, IETF is wasting its time. > > The IETF has already said that anything diffserv is domain-internal only. I > have joined the effort of the LE PHB and see if we can get some kind of > agreement and transparancy for a PHB that is aimed at customer access only > and "drop most of me and my pals at any sign of customer access line > congestion", and see if that can be agreed on. +1 > > Having a "lower-than-best-effort" diffserve codepoint might work, because it > means worse treatment, not preferential treatment. > > The problem with having DSCP CPs that indicate preferential treatment is > typically a ddos magnet. Hence splitting it up, three for the current transport domain to do with as it sees fit and 3 for signaling intent; this very much does not give a guarantee that any intermediate hop will follow the intent, but only make it possible for the endpoints to transmit intent. This IMHO is completely compatible with a LE PHB and transports honoring that request. > See my emails on this topic on (this? other?) mailing lists where I try to > create a three class buffering system saying "LE gets 5%, BE and > 'everything-else' gets to split the difference". We can haggle over the numbers but that seems a) sane and b) underspecified... > > I even got pushback on this here, and then we're not even close to people > running large ISP networks who see ddos attacks happen hourly. > > Saying L4S should "just use diffserv" is as constructive to say "go away and > pound a rock" or "we want that bit pattern so.. screw you". But just nodding expertly when they go and claim an unrelated bit in the IP header for their separation l4s vs legacy (as if l4s would be the end all of network design), and then having resorting to modifying so-far not-deployed-at the edge DCTCP (instead of modifying well-deployed TCP) because they already spent the one bit usable to extend ECN for less binary congestion signaling in a backward-compatible fashion... I might be wording things to strongly here, but that is the general gist. > > L4S has a much better possibility of actually getting deployment into the > wider Internet packet-moving equipment than anything being talked about here. That is not a high bar to clear though... > Same with PIE as opposed to FQ_CODEL. I know it's might not be as good, Debatable, and from my perspective this is the reason to talk about it at all. > but it fits better into actual silicon Does it? > and it's being proposed by people who actually have better channels into the > people setting hard requirements. That would be great if the proposal would throw end-user like me a bone instead of treating me as the product. It would also help if the architectural RFC would not be so breathlessly over-hyping/over-promising... But they really need end-points to switch over to a neutered DCTCP before things start to make sense, so they actually need to convince end-users and so far they are doing a terrible job IMHO. But what do I know... > > I suggest you consider joining them instead of opposing them. Join where? it pretty much looks like a "fait accompli" as they do seem way past the design stages and seem pretty much crystallized in what they see the path forward. Best Regards Sebastian > > -- > Mikael Abrahamssonemail: swm...@swm.pp.se ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] [Ecn-sane] [iccrg] Fwd: [tcpPrague] Implementation and experimentation of TCP Prague/L4S hackaton at IETF104
On Fri, 15 Mar 2019, David P. Reed wrote: So if the responsible network engineers in the carriers cannot agree on anything, IETF is wasting its time. The IETF has already said that anything diffserv is domain-internal only. I have joined the effort of the LE PHB and see if we can get some kind of agreement and transparancy for a PHB that is aimed at customer access only and "drop most of me and my pals at any sign of customer access line congestion", and see if that can be agreed on. Having a "lower-than-best-effort" diffserve codepoint might work, because it means worse treatment, not preferential treatment. The problem with having DSCP CPs that indicate preferential treatment is typically a ddos magnet. See my emails on this topic on (this? other?) mailing lists where I try to create a three class buffering system saying "LE gets 5%, BE and 'everything-else' gets to split the difference". I even got pushback on this here, and then we're not even close to people running large ISP networks who see ddos attacks happen hourly. Saying L4S should "just use diffserv" is as constructive to say "go away and pound a rock" or "we want that bit pattern so.. screw you". L4S has a much better possibility of actually getting deployment into the wider Internet packet-moving equipment than anything being talked about here. Same with PIE as opposed to FQ_CODEL. I know it's might not be as good, but it fits better into actual silicon and it's being proposed by people who actually have better channels into the people setting hard requirements. I suggest you consider joining them instead of opposing them. -- Mikael Abrahamssonemail: swm...@swm.pp.se ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] [Ecn-sane] [iccrg] Fwd: [tcpPrague] Implementation and experimentation of TCP Prague/L4S hackaton at IETF104
On Fri, 15 Mar 2019, Dave Taht wrote: It appears, also, ironically, (I have confirmation from several sources now) that cake, fq_codel and dualpi are now illegal for an ISP to use in their provided equipment under california law. The idea of one day having to appear in court to defend our key algorithms reminds me of the famous john fogerty case where he demonstrated how blues music was made. I would love to know more about this. Running an AQM on the customer access that doesn't prioritize some specific service should be fine, ie it doesn't explicitly do something the *customer* doesn't benefit from. Net neutrality should be about what the ISP does to benefit itself, not what is done on the customer access line (the scheduler per customer) that just looks at packet flow characteristics and isn't configured to prioritize any specific content (src/destination/port etc). -- Mikael Abrahamssonemail: swm...@swm.pp.se ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] [Ecn-sane] [iccrg] Fwd: [tcpPrague] Implementation and experimentation of TCP Prague/L4S hackaton at IETF104
On March 15, 2019 6:01:23 PM GMT+01:00, "David P. Reed" wrote: > >The absolute fundamental issue with diffserv, IMO, is that the carriers >cannot agree on an actual specification of what routers and gateways >are supposed to do, while being compatible with the core principle of >the IP layer: do not hold packets if they cause increasing queueing >delay. (this is in the Best Practices RFC) > IMHO it is the Charme of the 2 times 3 bits approach, that carriers get 3 bits they can do with whatever they want. As VLAN tags and MPLS? only support 3 bit priorities this looks to me like a match made in heaven, and we get 3 bits with end to end guarantees. Not that rolling that out would ever work in reality Best Regards Sebastian And yes, this is not an idea I came up with, I just forgot who proposed that initially. >And it's not for lack of trying. Dave Clark led a working group at the >MIT communications futures program (where I was a principle) that >included most major backbone providers' key folks that was specifically >focused on getting a widespread agreement on what any of the code >points might mean, not as bullshit English descriptions of what kind of >traffic each one represented, but as an operational description of what >should be done by a router to manage its queues. > >They couldn't even agree (after about 18 months of meetings) about what >the bullshit English intent was, much less what operational semantics >in the packet forwarding network had to be. > >So if the responsible network engineers in the carriers cannot agree on >anything, IETF is wasting its time. > > > > >-Original Message- >From: "Sebastian Moeller" >Sent: Friday, March 15, 2019 11:52am >To: "Dave Täht" >Cc: ecn-s...@lists.bufferbloat.net, "bloat" > >Subject: Re: [Ecn-sane] [Bloat] [iccrg] Fwd: [tcpPrague] Implementation >and experimentation of TCP Prague/L4S hackaton at IETF104 > > > >Hi Dave, > > >> On Mar 15, 2019, at 15:06, Dave Taht wrote: >> >> I would really prefer to move this discussion to the ecn-sane mailing >> list, as IMHO, ecn is generally such a tiny thing needed for good >> congestion control compared to better transports like pacing + bbr, >> and things like bql, fq, and aqm using drop. >> >> I'm going to keep cc-ing that list in the hope that we can keep >better >> track of the discussion here. > >Ah, sure, I basically wanted to avoid annoying the ietf lists as I feel >out of place there, having only had a cursory read of the relevant >RFCs. > > >> >> On Fri, Mar 15, 2019 at 6:01 AM Sebastian Moeller >wrote: >>> >>> Hi Dave, >>> >>> I pruned the CC list as I am out of my league here and want to >restrict the radius of my embarrassment to those that already know my >level of incompetence before hand. >> >> IMHO, your work on educating the OpenWrt community over the years on >> how to use sqm, makes you much more than "only a grasshopper". You >> have a firm grip on what can be achieved in the real world. >> >>> >>> That said, having read through the L4S architecture description and >the related appendices of draft-ietf-tsvwg-ecn-l4s-id-05 I came to the >conclusion, that this is a mess. >> >> I am so glad someone other than I has now read it. >> >>> The L4S project proposes a really wide-ranging change of basically >the internet (but allow a concurrent operation with legacy probably to >cater to the fact that an internet-wide flag-day seems daunting to >organize). But then they chicken out when figuring out how to >differentiate between their new and the old by proposing to use ECT(1) >for a purpose outside of its nominal purpose namely explicit congestion >notification, why not think bolder? If the plan is to change everything >the feasibility can not hinge upon the ability to re-using one old >legacy bit, can it... >>> Conceptually it seems much cleaner to use ECT(1) for congestion >notification directly (there are already proposals out there and SCE >just added another fully back-ward compatible one) and find another way >to mark l4s traffic, sure that is going to be hard and inconvenient, >but if you set out to change the internet that is par for the course. >>> IMHO they would do more good if they actually fought for a better >use of the 6 DSCP bits instead. (say by splitting into two groups of 3 >bits, one group for reduced diffserv and one group for new features, >that would even >> >> The existing diffserv deployment is a failure. > > Which is a shame, but one RFC's failure is another one's opportunity. > > >> I have another ID >> cooking that suggests a better way, going forward, to use them, but I >> do not feel at this time it would be useful to present. One big >battle >> at a time. > >:) > >> That ID is very simple, it basically proposes that all >> diffserv codepoints be passed through ISPs and transit providers >> unchanged, and if any given codepoint must be changed, the only >> permissible change is to 0 (BE), and MUST be not be remarked to >>
Re: [Bloat] [Ecn-sane] [iccrg] Fwd: [tcpPrague] Implementation and experimentation of TCP Prague/L4S hackaton at IETF104
The absolute fundamental issue with diffserv, IMO, is that the carriers cannot agree on an actual specification of what routers and gateways are supposed to do, while being compatible with the core principle of the IP layer: do not hold packets if they cause increasing queueing delay. (this is in the Best Practices RFC) And it's not for lack of trying. Dave Clark led a working group at the MIT communications futures program (where I was a principle) that included most major backbone providers' key folks that was specifically focused on getting a widespread agreement on what any of the code points might mean, not as bullshit English descriptions of what kind of traffic each one represented, but as an operational description of what should be done by a router to manage its queues. They couldn't even agree (after about 18 months of meetings) about what the bullshit English intent was, much less what operational semantics in the packet forwarding network had to be. So if the responsible network engineers in the carriers cannot agree on anything, IETF is wasting its time. -Original Message- From: "Sebastian Moeller" Sent: Friday, March 15, 2019 11:52am To: "Dave Täht" Cc: ecn-s...@lists.bufferbloat.net, "bloat" Subject: Re: [Ecn-sane] [Bloat] [iccrg] Fwd: [tcpPrague] Implementation and experimentation of TCP Prague/L4S hackaton at IETF104 Hi Dave, > On Mar 15, 2019, at 15:06, Dave Taht wrote: > > I would really prefer to move this discussion to the ecn-sane mailing > list, as IMHO, ecn is generally such a tiny thing needed for good > congestion control compared to better transports like pacing + bbr, > and things like bql, fq, and aqm using drop. > > I'm going to keep cc-ing that list in the hope that we can keep better > track of the discussion here. Ah, sure, I basically wanted to avoid annoying the ietf lists as I feel out of place there, having only had a cursory read of the relevant RFCs. > > On Fri, Mar 15, 2019 at 6:01 AM Sebastian Moeller wrote: >> >> Hi Dave, >> >> I pruned the CC list as I am out of my league here and want to restrict the >> radius of my embarrassment to those that already know my level of >> incompetence before hand. > > IMHO, your work on educating the OpenWrt community over the years on > how to use sqm, makes you much more than "only a grasshopper". You > have a firm grip on what can be achieved in the real world. > >> >> That said, having read through the L4S architecture description and the >> related appendices of draft-ietf-tsvwg-ecn-l4s-id-05 I came to the >> conclusion, that this is a mess. > > I am so glad someone other than I has now read it. > >> The L4S project proposes a really wide-ranging change of basically the >> internet (but allow a concurrent operation with legacy probably to cater to >> the fact that an internet-wide flag-day seems daunting to organize). But >> then they chicken out when figuring out how to differentiate between their >> new and the old by proposing to use ECT(1) for a purpose outside of its >> nominal purpose namely explicit congestion notification, why not think >> bolder? If the plan is to change everything the feasibility can not hinge >> upon the ability to re-using one old legacy bit, can it... >> Conceptually it seems much cleaner to use ECT(1) for congestion notification >> directly (there are already proposals out there and SCE just added another >> fully back-ward compatible one) and find another way to mark l4s traffic, >> sure that is going to be hard and inconvenient, but if you set out to change >> the internet that is par for the course. >> IMHO they would do more good if they actually fought for a better use of the >> 6 DSCP bits instead. (say by splitting into two groups of 3 bits, one group >> for reduced diffserv and one group for new features, that would even > > The existing diffserv deployment is a failure. Which is a shame, but one RFC's failure is another one's opportunity. > I have another ID > cooking that suggests a better way, going forward, to use them, but I > do not feel at this time it would be useful to present. One big battle > at a time. :) > That ID is very simple, it basically proposes that all > diffserv codepoints be passed through ISPs and transit providers > unchanged, and if any given codepoint must be changed, the only > permissible change is to 0 (BE), and MUST be not be remarked to > anything else, especially not CS1. I like the simplicity, but I also like the split into two groups of 3 bits, say "active bit pattern" (for transport purposes) and "intenden bit pattern" for the sender to transmit intent, which than can be conveyed lossless to the receiver; my gut feeling is that throwing the transport people a bone here might work better, as in the end they are the one that will carry the "burden" of the change. IMHO this has the additional advantage that it will make "tabula rasa" of the existing distict set
Re: [Bloat] [iccrg] Fwd: [tcpPrague] Implementation and experimentation of TCP Prague/L4S hackaton at IETF104
Hi Dave, > On Mar 15, 2019, at 15:06, Dave Taht wrote: > > I would really prefer to move this discussion to the ecn-sane mailing > list, as IMHO, ecn is generally such a tiny thing needed for good > congestion control compared to better transports like pacing + bbr, > and things like bql, fq, and aqm using drop. > > I'm going to keep cc-ing that list in the hope that we can keep better > track of the discussion here. Ah, sure, I basically wanted to avoid annoying the ietf lists as I feel out of place there, having only had a cursory read of the relevant RFCs. > > On Fri, Mar 15, 2019 at 6:01 AM Sebastian Moeller wrote: >> >> Hi Dave, >> >> I pruned the CC list as I am out of my league here and want to restrict the >> radius of my embarrassment to those that already know my level of >> incompetence before hand. > > IMHO, your work on educating the OpenWrt community over the years on > how to use sqm, makes you much more than "only a grasshopper". You > have a firm grip on what can be achieved in the real world. > >> >> That said, having read through the L4S architecture description and the >> related appendices of draft-ietf-tsvwg-ecn-l4s-id-05 I came to the >> conclusion, that this is a mess. > > I am so glad someone other than I has now read it. > >> The L4S project proposes a really wide-ranging change of basically the >> internet (but allow a concurrent operation with legacy probably to cater to >> the fact that an internet-wide flag-day seems daunting to organize). But >> then they chicken out when figuring out how to differentiate between their >> new and the old by proposing to use ECT(1) for a purpose outside of its >> nominal purpose namely explicit congestion notification, why not think >> bolder? If the plan is to change everything the feasibility can not hinge >> upon the ability to re-using one old legacy bit, can it... >> Conceptually it seems much cleaner to use ECT(1) for congestion notification >> directly (there are already proposals out there and SCE just added another >> fully back-ward compatible one) and find another way to mark l4s traffic, >> sure that is going to be hard and inconvenient, but if you set out to change >> the internet that is par for the course. >> IMHO they would do more good if they actually fought for a better use of the >> 6 DSCP bits instead. (say by splitting into two groups of 3 bits, one group >> for reduced diffserv and one group for new features, that would even > > The existing diffserv deployment is a failure. Which is a shame, but one RFC's failure is another one's opportunity. > I have another ID > cooking that suggests a better way, going forward, to use them, but I > do not feel at this time it would be useful to present. One big battle > at a time. :) > That ID is very simple, it basically proposes that all > diffserv codepoints be passed through ISPs and transit providers > unchanged, and if any given codepoint must be changed, the only > permissible change is to 0 (BE), and MUST be not be remarked to > anything else, especially not CS1. I like the simplicity, but I also like the split into two groups of 3 bits, say "active bit pattern" (for transport purposes) and "intenden bit pattern" for the sender to transmit intent, which than can be conveyed lossless to the receiver; my gut feeling is that throwing the transport people a bone here might work better, as in the end they are the one that will carry the "burden" of the change. IMHO this has the additional advantage that it will make "tabula rasa" of the existing distict set of bit patterns used in the real world. Which in turn probably is this ideas downfall... > >> allow for concurrent use of the inevitable L5S and L6S ;) ). Especially >> since as far as I can understand l4s actually would like to have a more >> gradual congestion information stream than classic ECN, and since they need >> to modify DCTCP anyway to make it save for the wider internet, replacing its >> ECN response should be well inside the scope of work they already have on >> their list. > > Next up for sce was going to be to find if dctcp could be modified to > use it. Also, bittorrent. YES! I endorse this message. > >> If I sound a bit miffed, it is because after reading >> https://tools.ietf.org/html/draft-ietf-tsvwg-l4s-arch-03 I do not have the >> feeling they are trying to build abetter internet, but rather that they are >> building an internet where I can be a better "product" and customer of >> newfangled services, and I do not look forward to such a facebooky future >> with much enthusiasm. > > I have rigorously tried to keep the network neutrality debate out of > this. And I really do not want to start this here, as I agree that this is about a technical question. That said, I had hoped more out of the l4s promises from an end-user perspective. Then again, it if this is going to happen it needs the ISPs
Re: [Bloat] [iccrg] Fwd: [tcpPrague] Implementation and experimentation of TCP Prague/L4S hackaton at IETF104
> On 15 Mar, 2019, at 4:44 pm, Sebastian Moeller wrote: > > In essence, they do not want to deal with the diffserv mess under the > end-to-end perspective and making it reliable. Yeah, that's pretty much what I thought. Diffserv really does need to be fixed sometime. >> But Codel-type AQMs don't provide the ideal ECN signal for L4S (and nor do >> RED-type AQMs without specific tuning for L4S), and any single-queue AQM is >> going to end up with L4S flows outcompeting standard ones quite seriously. > > Except L$S flows are required to "degrade" to classic congestion contro once > they have proof of not being handled by a l4s aware AQM, like real packet > drops or ECT(0) + CE... That isn't enough, because ECN AQMs as presently specified won't change ECT(1) to ECT(0) (nor will SCE), and it would require a lot of overload before dropping actually started. So those signals don't reliably identify a legacy AQM. > L4S would find uses for SCE, but that still faces the same roll-out issue... It's not the same roll-out issue, because in an SCE context L4S would have to be legacy-compatible by default, and only employ the "new" behaviour when SCE information appears - the reverse of its present behaviour - which then makes it backwards compatible and incrementally deployable. The real snag is that DCTCP's setpoint is 2 marks per RTT, which is different from SCE's specified setpoint of 50% packets marked (unless the cwnd is down to 4 packets). That'd make it difficult for a straight port of DCTCP to SCE to achieve full throughput. A sane compromise could be for SCE to be marked in the same way as L4S marks CE, and a valid interpretation of SCE to follow the 1/n response of DCTCP. That would leverage existing TCP-Prague R, but in a backwards-compatible, incrementally-deployable manner. Perhaps that's something worth discussing at IETF? - Jonathan Morton ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] [iccrg] Fwd: [tcpPrague] Implementation and experimentation of TCP Prague/L4S hackaton at IETF104
Hi Jonathan, > On Mar 15, 2019, at 15:27, Jonathan Morton wrote: > >> On 15 Mar, 2019, at 3:01 pm, Sebastian Moeller wrote: >> >> That said, having read through the L4S architecture description and the >> related appendices of draft-ietf-tsvwg-ecn-l4s-id-05 I came to the >> conclusion, that this is a mess. >> >> The L4S project proposes a really wide-ranging change of basically the >> internet (but allow a concurrent operation with legacy probably to cater to >> the fact that an internet-wide flag-day seems daunting to organize). But >> then they chicken out when figuring out how to differentiate between their >> new and the old by proposing to use ECT(1)… > > I think I would be less annoyed by L4S if it proposed to assign a DSCP or > three to identify its traffic. In fact, I'd be interested to hear a defence > of why DSCP identification of L4S flows was *not* adopted. I suspect it > would revolve around the widespread practice of re-marking DSCPs by ISPs, > which renders DSCP too unreliable for L4S' purposes. https://tools.ietf.org/html/draft-ietf-tsvwg-ecn-l4s-id-05 contains a discussion of the alternatives, specifically to your point it argues: "Appendix B. Alternative Identifiers [...] B.2. ECN Plus a Diffserv Codepoint (DSCP) Definition: For packets with a defined DSCP, all codepoints of the ECN field (except Not-ECT) would signify alternative L4S semantics to those for classic ECN [RFC3168], specifically: * The L4S DSCP would signifiy that the packet came from an L4S- capable sender; * ECT(0) and ECT(1) would both signify that the packet was travelling between transport endpoints that were both ECN- capable; * CE would signify that the packet had been marked by an AQM implementing the L4S service. Use of a DSCP is the only approach for alternative ECN semantics given as an example in [RFC4774]. However, it was perhaps considered more for controlled environments than new end-to-end services; Cons: Consumes DSCP pairs: A DSCP is obviously not orthogonal to Diffserv. Therefore, wherever the L4S service is applied to multiple Diffserv scheduling behaviours, it would be necessary to replace each DSCP with a pair of DSCPs. Uses critical lower-layer header space: The resulting increased number of DSCPs might be hard to support for some lower layer technologies, e.g. 802.1p and MPLS both offer only 3-bits for a maximum of 8 traffic class identifiers. Although L4S should reduce and possibly remove the need for some DSCPs intended for differentiated queuing delay, it will not remove the need for Diffserv entirely, because Diffserv is also used to allocate bandwidth, e.g. by prioritising some classes of traffic over others when traffic exceeds available capacity. Not end-to-end (host-network): Very few networks honour a DSCP set by a host. Typically a network will zero (bleach) the Diffserv field from all hosts. Sometimes networks will attempt to identify applications by some form of packet inspection and, based on network policy, they will set the DSCP considered appropriate for the identified application. Network-based application identification might use some combination of protocol ID, port numbers(s), application layer protocol headers, IP address(es), VLAN ID(s) and even packet timing. Not end-to-end (network-network): Very few networks honour a DSCP received from a neighbouring network. Typically a network will zero (bleach) the Diffserv field from all neighbouring networks at an interconnection point. Sometimes bilateral arrangements are made between networks, such that the receiving network remarks some DSCPs to those it uses for roughly equivalent services. The likelihood that a DSCP will be bleached or ignored depends on the type of DSCP: Local-use DSCP: These tend to be used to implement application- specific network policies, but a bilateral arrangement to remark certain DSCPs is often applied to DSCPs in the local-use range simply because it is easier not to change all of a network's internal configurations when a new arrangement is made with a neighbour; Global-use DSCP: These do not tend to be honoured across network interconnections more than local-use DSCPs. However, if two networks decide to honour certain of each other's DSCPs, the reconfiguration is a little easier if both of their globally recognised services are already represented by the relevant global-use DSCPs. Note that today a global-use DSCP gives little more assurance of end-to-end service than a local-use DSCP. In future the global-use range might give more assurance of end-to-end
Re: [Bloat] [iccrg] Fwd: [tcpPrague] Implementation and experimentation of TCP Prague/L4S hackaton at IETF104
> On 15 Mar, 2019, at 3:01 pm, Sebastian Moeller wrote: > > That said, having read through the L4S architecture description and the > related appendices of draft-ietf-tsvwg-ecn-l4s-id-05 I came to the > conclusion, that this is a mess. > > The L4S project proposes a really wide-ranging change of basically the > internet (but allow a concurrent operation with legacy probably to cater to > the fact that an internet-wide flag-day seems daunting to organize). But then > they chicken out when figuring out how to differentiate between their new and > the old by proposing to use ECT(1)… I think I would be less annoyed by L4S if it proposed to assign a DSCP or three to identify its traffic. In fact, I'd be interested to hear a defence of why DSCP identification of L4S flows was *not* adopted. I suspect it would revolve around the widespread practice of re-marking DSCPs by ISPs, which renders DSCP too unreliable for L4S' purposes. But using the ECN field for this purpose is *also* unreliable, because it is *intended* to be altered on route (in prescribed ways). In particular, both ECT codepoints can be replaced by CE, erasing the distinction between them when it comes to choosing which half of a "dual queue" AQM to pass it through. I'm not convinced they've spent very much of the past several years thinking about that. Fortunately, L4S flows should work with flow-isolating AQMs (including Cake) without modifying the latter, and without needing to specifically identify which flows are L4S or otherwise. That really says more about the robustness of proper flow isolation than anything else. But Codel-type AQMs don't provide the ideal ECN signal for L4S (and nor do RED-type AQMs without specific tuning for L4S), and any single-queue AQM is going to end up with L4S flows outcompeting standard ones quite seriously. Since very little "big iron" hardware supports flow-isolating AQM so far, that puts a damper on any "incremental deployment" argument to be made for L4S, even if they claim their dual-queue AQM is easier to implement at high link rates than full flow isolation. The fact is that either dual-queue or flow-isolation is required in middleboxes *before* endpoint deployment of L4S can begin. SCE explicitly avoids these problems, as both SCE-aware endpoints and SCE-aware middleboxes can safely be deployed without waiting for each other. Work remains on figuring out precisely how SCE information should be produced and consumed, and verifying that the resulting system behaves as intended when fully deployed - but its interaction with existing deployed hardware and software is explicitly defined to be benign. - Jonathan Morton ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] [iccrg] Fwd: [tcpPrague] Implementation and experimentation of TCP Prague/L4S hackaton at IETF104
I would really prefer to move this discussion to the ecn-sane mailing list, as IMHO, ecn is generally such a tiny thing needed for good congestion control compared to better transports like pacing + bbr, and things like bql, fq, and aqm using drop. I'm going to keep cc-ing that list in the hope that we can keep better track of the discussion here. On Fri, Mar 15, 2019 at 6:01 AM Sebastian Moeller wrote: > > Hi Dave, > > I pruned the CC list as I am out of my league here and want to restrict the > radius of my embarrassment to those that already know my level of > incompetence before hand. IMHO, your work on educating the OpenWrt community over the years on how to use sqm, makes you much more than "only a grasshopper". You have a firm grip on what can be achieved in the real world. > > That said, having read through the L4S architecture description and the > related appendices of draft-ietf-tsvwg-ecn-l4s-id-05 I came to the > conclusion, that this is a mess. I am so glad someone other than I has now read it. > The L4S project proposes a really wide-ranging change of basically the > internet (but allow a concurrent operation with legacy probably to cater to > the fact that an internet-wide flag-day seems daunting to organize). But then > they chicken out when figuring out how to differentiate between their new and > the old by proposing to use ECT(1) for a purpose outside of its nominal > purpose namely explicit congestion notification, why not think bolder? If the > plan is to change everything the feasibility can not hinge upon the ability > to re-using one old legacy bit, can it... > Conceptually it seems much cleaner to use ECT(1) for congestion notification > directly (there are already proposals out there and SCE just added another > fully back-ward compatible one) and find another way to mark l4s traffic, > sure that is going to be hard and inconvenient, but if you set out to change > the internet that is par for the course. > IMHO they would do more good if they actually fought for a better use of the > 6 DSCP bits instead. (say by splitting into two groups of 3 bits, one group > for reduced diffserv and one group for new features, that would even The existing diffserv deployment is a failure. I have another ID cooking that suggests a better way, going forward, to use them, but I do not feel at this time it would be useful to present. One big battle at a time. That ID is very simple, it basically proposes that all diffserv codepoints be passed through ISPs and transit providers unchanged, and if any given codepoint must be changed, the only permissible change is to 0 (BE), and MUST be not be remarked to anything else, especially not CS1. >allow for concurrent use of the inevitable L5S and L6S ;) ). Especially since >as far as I can understand l4s actually would like to have a more gradual >congestion information stream than classic ECN, and since they need to modify >DCTCP anyway to make it save for the wider internet, replacing its ECN >response should be well inside the scope of work they already have on their >list. Next up for sce was going to be to find if dctcp could be modified to use it. Also, bittorrent. > If I sound a bit miffed, it is because after reading > https://tools.ietf.org/html/draft-ietf-tsvwg-l4s-arch-03 I do not have the > feeling they are trying to build abetter internet, but rather that they are > building an internet where I can be a better "product" and customer of > newfangled services, and I do not look forward to such a facebooky future > with much enthusiasm. I have rigorously tried to keep the network neutrality debate out of this. dualpi is just another aqm that needs the same thorough technical and public evaluation done to it that pie, codel, fq_codel were subjected to.The use of ect_1 in dualpi for it is nuts IMHO, and I'd vastly prefer that another L4S codepoint be added to make it work, but any attempt to do so would also require industry consolidation on that ID and that would be distracting at this time. It appears, also, ironically, (I have confirmation from several sources now) that cake, fq_codel and dualpi are now illegal for an ISP to use in their provided equipment under california law. The idea of one day having to appear in court to defend our key algorithms reminds me of the famous john fogerty case where he demonstrated how blues music was made. I wish I knew a lawyer willing to take on "bufferbloat.net vs the state of california", though, as it may come down to that. I blew up on this part of the issue here: http://blog.cerowrt.org/post/net_neutrality_customers/ > > I hope that the discussion in Prague go well and a compromise/consense can be > hashed out as I see different implementations duking it out here, but the > overall goal of the competitors seems quite compatible, improving the > internet by focussing on latency. > > Best Regards > Sebastian > > > On Mar 15, 2019, at 11:46, Dave Taht
Re: [Bloat] [iccrg] Fwd: [tcpPrague] Implementation and experimentation of TCP Prague/L4S hackaton at IETF104
Hi Dave, I pruned the CC list as I am out of my league here and want to restrict the radius of my embarrassment to those that already know my level of incompetence before hand. That said, having read through the L4S architecture description and the related appendices of draft-ietf-tsvwg-ecn-l4s-id-05 I came to the conclusion, that this is a mess. The L4S project proposes a really wide-ranging change of basically the internet (but allow a concurrent operation with legacy probably to cater to the fact that an internet-wide flag-day seems daunting to organize). But then they chicken out when figuring out how to differentiate between their new and the old by proposing to use ECT(1) for a purpose outside of its nominal purpose namely explicit congestion notification, why not think bolder? If the plan is to change everything the feasibility can not hinge upon the ability to re-using one old legacy bit, can it... Conceptually it seems much cleaner to use ECT(1) for congestion notification directly (there are already proposals out there and SCE just added another fully back-ward compatible one) and find another way to mark l4s traffic, sure that is going to be hard and inconvenient, but if you set out to change the internet that is par for the course. IMHO they would do more good if they actually fought for a better use of the 6 DSCP bits instead. (say by splitting into two groups of 3 bits, one group for reduced diffserv and one group for new features, that would even allow for concurrent use of the inevitable L5S and L6S ;) ). Especially since as far as I can understand l4s actually would like to have a more gradual congestion information stream than classic ECN, and since they need to modify DCTCP anyway to make it save for the wider internet, replacing its ECN response should be well inside the scope of work they already have on their list. If I sound a bit miffed, it is because after reading https://tools.ietf.org/html/draft-ietf-tsvwg-l4s-arch-03 I do not have the feeling they are trying to build abetter internet, but rather that they are building an internet where I can be a better "product" and customer of newfangled services, and I do not look forward to such a facebooky future with much enthusiasm. I hope that the discussion in Prague go well and a compromise/consense can be hashed out as I see different implementations duking it out here, but the overall goal of the competitors seems quite compatible, improving the internet by focussing on latency. Best Regards Sebastian > On Mar 15, 2019, at 11:46, Dave Taht wrote: > > Bufferbloat.net's ecn-sane working group members have a co-ordinated response > to your efforts brewing but it's not ready yet. We have a worldwide team of > linux and freebsd developers co-ordinating on landing code for our competing > proposal "Some Congestion Experienced", which we submitted to tsvwg last > sunday. > > That draft is under continuous revision, here: > > https://github.com/dtaht/bufferbloat-rfcs/blob/master/sce/draft-morton-taht-tsvwg-sce.txt > > Our Linux and FreeBSD team is also flying into prague for SCE presentations > at netdevconf and ietf. > > Some background to this: after the L4S/TCP Prague/and dualpi experiments > appeared stalled out indefinitely in the IETF, and with our own frustration > with IETF processes, bufferbloat.net project members publicly formed our own > working group to look into the problems with ecn, back in august of last year. > > Its charter is here: https://www.bufferbloat.net/projects/ecn-sane/wiki/ > > We were unaware, until last month, that the cable industry had 16 months back > gone and formed its own private working group also, and was intending to turn > the tcp prague/l4s/dualpi IETF "experiments" into an actual DOCSIS standard. > > Our SCE proposal appears to be backward compatible with the existing 10s-100s > of millions of ecn-enabled fq_codel[1] and sch_cake[2] deployments, and > doesn't require any changes to any RFC3168 tcps (or any tcp-friendly > congestion control) at all in order to basically work. tcp-prague is subtly > incompatible with that, and dualpi, more so. Our proposal is different also, > it proposes some receiver side changes in order to get the full benefit of > SCE while remaining backward compatible with the existing meaning of the CE > codepoint. > > In either case, either approach essentially permanently redefines the ECT_1 > codepoint incompatibly, once and for all, and for all time. This is a final > battle over the meaning of a single bit in IP, and I expect the debates to be > as difficult as the ones described in https://www.ietf.org/rfc/ien/ien137.txt > - I would really, really, really prefer that they stay technical and not veer > into politics, but I have little hope for that. > > The members of the ecn-sane working group are delighted to finally hear that > running code for tcp-prague might land this ietf, and look
Re: [Bloat] [iccrg] Fwd: [tcpPrague] Implementation and experimentation of TCP Prague/L4S hackaton at IETF104
Bufferbloat.net's ecn-sane working group members have a co-ordinated response to your efforts brewing but it's not ready yet. We have a worldwide team of linux and freebsd developers co-ordinating on landing code for our competing proposal "Some Congestion Experienced", which we submitted to tsvwg last sunday. That draft is under continuous revision, here: https://github.com/dtaht/bufferbloat-rfcs/blob/master/sce/draft-morton-taht-tsvwg-sce.txt Our Linux and FreeBSD team is also flying into prague for SCE presentations at netdevconf and ietf. Some background to this: after the L4S/TCP Prague/and dualpi experiments appeared stalled out indefinitely in the IETF, and with our own frustration with IETF processes, bufferbloat.net project members publicly formed our own working group to look into the problems with ecn, back in august of last year. Its charter is here: https://www.bufferbloat.net/projects/ecn-sane/wiki/ We were unaware, until last month, that the cable industry had 16 months back gone and formed its own private working group also, and was intending to turn the tcp prague/l4s/dualpi IETF "experiments" into an actual DOCSIS standard. Our SCE proposal appears to be backward compatible with the existing 10s-100s of millions of ecn-enabled fq_codel[1] and sch_cake[2] deployments, and doesn't require any changes to any RFC3168 tcps (or any tcp-friendly congestion control) at all in order to basically work. tcp-prague is subtly incompatible with that, and dualpi, more so. Our proposal is different also, it proposes some receiver side changes in order to get the full benefit of SCE while remaining backward compatible with the existing meaning of the CE codepoint. In either case, either approach essentially permanently redefines the ECT_1 codepoint incompatibly, once and for all, and for all time. This is a final battle over the meaning of a single bit in IP, and I expect the debates to be as difficult as the ones described in https://www.ietf.org/rfc/ien/ien137.txt - I would really, really, really prefer that they stay technical and not veer into politics, but I have little hope for that. The members of the ecn-sane working group are delighted to finally hear that running code for tcp-prague might land this ietf, and look forward to finally testing the whole l4s/tcpprague/dualpi architecture in conjunction with the flent.org 's and irtt's exhaustive suite of tests and servers in the cloud in the coming months, both against our existing, deployed, fq_codel, fq_pie, cake and pie derived solutions and our new SCE proposal. We hope to finally be able to write new tests for flent in particular, that can show tcpprague off in the ways that are important to those developing it. Flent has some basic dctcp tests, but nothing that can get down below a 20ms sample rate on modern hardware. (currently. It's easy to add tests, flent is written in python) We also hope that more test tools and implementations in ns2 and ns3 show up for tcpprauge and dualpi show up soon also, from members of those projects. Note: sunday's dual-pi linux submission was kicked back from the linux networking developers due to some technical and legal problems with linux net-next HEAD ( https://patchwork.ozlabs.org/patch/1054521/ ) , and I do hope that a corrected version lands soon, so we can safely test it with current versions of OpenWrt, etc. Finally, running code. Will we find consensus? Thx! [1] https://tools.ietf.org/html/rfc8290 [2] sch_cake was available for 3 years out of tree and was mainlined last august, in linux 4.19. It is partially described by "Piece of CAKE: A Comprehensive Queue Management Solution for Home Gateways" " https://arxiv.org/pdf/1804.07617.pdf A second paper describing its fq_codel-derived "cobalt" AQM algorithm is awaiting publication in a peer reviewed journal. It has been part of openwrt and the related sqm-scripts for many years and is widely deployed on multiple commercial products, such as those from eero and evenroute. Cake has a docsis specific mode which we longed for cablelabs to evaluate. On Fri, Mar 15, 2019 at 2:33 AM Bob Briscoe wrote: > Forwarding to tcpm & iccrg - apologies if you were already on one of the > lists that received this. > > Olivier has been working hard on integrating the pieces of a Linux > implementation of TCP Prague, and is close to having a version ported > against the tip of the Linux mainline tree. This is his request for more > people to get involved. > > > Bob > > > Forwarded Message > Subject: [tcpPrague] Implementation and experimentation of TCP Prague/L4S > hackaton at IETF104 > Date: Wed, 6 Mar 2019 10:26:05 + > From: Tilmans, Olivier (Nokia - BE/Antwerp) > > > To: hackat...@ietf.org , > tcppra...@ietf.org > CC: dleb...@google.com , Joakim > Misund , Bob Briscoe > , Quentin De Coninck > , > François Michel > , Mirja Kuehlewind > , > Maxime Piraux , > Olga Albisser , Fabien Duchêne > , De Schepper, > Koen
