subject:"\[Bloat\] incremental deployment, transport and L4S \(Re\: when does the CoDel part of fq

> On 29 Nov 2018, at 13:52, Jonathan Morton  wrote:
> 
>> On 29 Nov, 2018, at 2:06 pm, Michael Welzl  wrote:
>> 
>>> That's my proposal.
>> 
>> - and it's an interesting one. Indeed, I wasn't aware that you're thinking 
>> of a DCTCP-style signal from a string of packets.
>> 
>> Of course, this is hard to get right - there are many possible flavours to 
>> ideas like this ... but yes, interesting!
> 
> I'm glad you think so.  Working title is ELR - Explicit Load Regulation.
> 
> As noted, this needs standardisation effort, which is a bit outside my realm 
> of experience - Cake was a great success, but relied entirely on exploiting 
> existing standards to their logical conclusions.  I think I started writing 
> some material to put in an I-D, but got distracted by something more urgent.

Well - "interesting" is one thing, "better than current proposals" is 
another... I guess this needs lots of evaluations before going anywhere.

> If there's an opportunity to coordinate with relevant people from similar 
> efforts, so much the better.  I wonder, for example, whether the DCTCP folks 
> would be open to supporting a more deployable version of their idea, or 
> whether that would be a political non-starter for them.

I'm not convinced (and I strongly doubt that they would be) that this would 
indeed be more deployable; your idea also includes TCP option changes, which 
have their own deployment trouble... the L4S effort, to me, sounds "easier" to 
deploy  (which is not to say that it's easy to deploy at all; though I did like 
a recent conversation on possibly deploying it with a PEP... that sounded quite 
doable to me).

Cheers,
Michael

___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] incremental deployment, transport and L4S (Re: when does the CoDel part of fq_codel help in the real world?)


On Fri, 30 Nov 2018, Jonathan Morton wrote:

Ah, so you're thinking in terms of link-layers which perform local 
retransmission, like wifi.  So the optimisation is to not delay packets 
"behind" a corrupted packet while the latter is retransmitted.


Yes.

It's possible for a TCP to interpret a reordered packet as missing, 
triggering an end-to-end retransmission which is then discovered to be 
unnecessary.  At the application level, TCP also performs the same HoL 
blocking in response to missing data.  So it's easy to see why links try 
to preserve ordering, even to this extent, but I suspect they typically 
do so on a per-station basis rather than per-flow.


It's a "truth-everybody-knows" in networking that "NEVER RE-ORDER PACKETS 
WITHIN 5-TUPLE FLOW! THERE BE DRAGONS THERE!". I'd also say I see 
enough transport people who says that this should be true generally, if 
nothing else because of legacy.


Personally I think the problem of reordering packets is overblown, and 
that TCPs can cope with occasional missing or reordered packets without 
serious consequences to performance.  So if you add "reordering 
tolerant" to the list of stuff that Diffserv can indicate, you might 
just end up with all traffic being marked that way.  Is that really 
worthwhile?


Question isn't so much about TCP, it's the other things I am worried 
about. TCP handles re-ordering kind of gracefully, other protocols might 
not.


Oddly enough, wifi is now one of the places where FQ is potentially 
easiest to find, with Toke's work reaching the Linux kernel and so many 
wifi routers being Linux based.


Again, even if they're using Linux they will/might have packet 
accelerators that just grab the flow and the kernel never sees it again. 
No FQ_CODEL for that.


An acknowledged problem is overly persistent retries by the ARQ 
mechanism, such that the time horizon for the link-layer retransmission 
often exceeds that of the end-to-end RTO, both for TCP and 
request-response protocols like DNS. I say, retransmit at the link layer 
once or twice, then give up and let the end-hosts sort it out.


I agree, but I also think that it would help some link-layers if the 
re-ordering requirement could be relaxed. However, before that can be 
communicated a lot of study needs to be done to check if this is actually 
true. I've had incidents in my 20 year networking career where it's not 
and applications misbehaved when they were re-ordered.


--
Mikael Abrahamssonemail: swm...@swm.pp.se
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] incremental deployment, transport and L4S (Re: when does the CoDel part of fq_codel help in the real world?)

2018-11-29 Thread Dave Taht

On Wed, Nov 28, 2018 at 11:36 PM Jonathan Morton  wrote:
>
> > On 29 Nov, 2018, at 9:28 am, Mikael Abrahamsson  wrote:
> >
> > This is one thing about L4S, ETC(1) is the last "codepoint" in the header 
> > not used, that can statelessly identify something. If anyone sees a better 
> > way to use it compared to "let's put it in a separate queue and CE-mark it 
> > agressively at very low queue depths and also do not care about re-ordering 
> > so a ARQ L2 can re-order all it wants", then they need to speak up, soon.
>
> You are essentially proposing using ECT(1) to take over an intended function 
> of Diffserv.  In my view, that is the wrong approach.  Better to improve 
> Diffserv to the point where it becomes useful in practice.  Cake has taken 
> steps in that direction, by implementing some reasonable interpretation of 
> some Diffserv codepoints.
>
> My alternative use of ECT(1) is more in keeping with the other codepoints 
> represented by those two bits, to allow ECN to provide more fine-grained 
> information about congestion than it presently does.  The main challenge is 
> communicating the relevant information back to the sender upon receipt, 
> ideally without increasing overhead in the TCP/IP headers.

I felt that using this bit up as a separate indicator of an alternate
algorithm in play for indicating congestion was a pretty good idea...
but no-one was listening at the time.

>  - Jonathan Morton
>
> ___
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat



-- 

Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-205-9740
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] incremental deployment, transport and L4S (Re: when does the CoDel part of fq_codel help in the real world?)

>> I have to ask, why would the network care?  What optimisations can be 
>> obtained by reordering packets *within* a flow, when it's usually just as 
>> easy to deliver them in order?
> 
> Because most implementations aren't flow aware at all and might have 4 
> queues, saying "oh, this single queue is for transports that don't care about 
> ordering" means everything in that queue can just be sent as soon as it can, 
> ignoring HOL caused by ARQ.

Ah, so you're thinking in terms of link-layers which perform local 
retransmission, like wifi.  So the optimisation is to not delay packets 
"behind" a corrupted packet while the latter is retransmitted.

It's possible for a TCP to interpret a reordered packet as missing, triggering 
an end-to-end retransmission which is then discovered to be unnecessary.  At 
the application level, TCP also performs the same HoL blocking in response to 
missing data.  So it's easy to see why links try to preserve ordering, even to 
this extent, but I suspect they typically do so on a per-station basis rather 
than per-flow.

Personally I think the problem of reordering packets is overblown, and that 
TCPs can cope with occasional missing or reordered packets without serious 
consequences to performance.  So if you add "reordering tolerant" to the list 
of stuff that Diffserv can indicate, you might just end up with all traffic 
being marked that way.  Is that really worthwhile?

>> Of course, we already have FQ which reorders packets in *different* flows.  
>> The benefits are obvious in that case.
> 
> FQ is a fringe in real life (speaking as a packet moving monkey). It's just 
> on this mailing list that it's the norm.

Oddly enough, wifi is now one of the places where FQ is potentially easiest to 
find, with Toke's work reaching the Linux kernel and so many wifi routers being 
Linux based.

An acknowledged problem is overly persistent retries by the ARQ mechanism, such 
that the time horizon for the link-layer retransmission often exceeds that of 
the end-to-end RTO, both for TCP and request-response protocols like DNS. I 
say, retransmit at the link layer once or twice, then give up and let the 
end-hosts sort it out.

 - Jonathan Morton

___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] incremental deployment, transport and L4S (Re: when does the CoDel part of fq_codel help in the real world?)

2018-11-29 Thread Bless, Roland (TM)

Hi Michael,

Am 29.11.18 um 13:12 schrieb Michael Welzl:
> I'm answering myself with an add-on thought:
> 
>> On 29 Nov 2018, at 09:08, Michael Welzl  wrote:
>>
>>
>>
>>> On 29 Nov 2018, at 08:46, Mikael Abrahamsson  wrote:
>>>
>>> On Thu, 29 Nov 2018, Jonathan Morton wrote:
>>>
 In my view, that is the wrong approach.  Better to improve Diffserv to the 
 point where it becomes useful in practice.
>>>
>>> I agree, but unfortunately nobody has made me king of the Internet yet so I 
>>> can't just decree it into existance.
>>
>> Well, for what you want (re-ordering tolerance), I would think that the LE 
>> codepoint is suitable. From:
>> https://tools.ietf.org/html/draft-ietf-tsvwg-le-phb-06
>> "there ought to be an expectation that packets of the LE PHB could be 
>> excessively delayed or dropped when any other traffic is present"
>>
>> ... I think it would be strange for an application to expect this, yet not 
>> expect it to happen for only a few individual packets from a stream.
> 
> Actually, maybe this is a problem: the semantics of LE are way broader than 
> "tolerant to re-ordering". What about applications that are 
> reordering-tolerant, yet still latency critical?

Yep, the LE semantics are basically that you're expecting to just
utilize any spare capacity (which may not be available for some longer
periods). Re-ordering of LE-packets shouldn't normally be the case as
packets of a particular flow should all be in the same LE queue.

> E.g., if I use a protocol that can hand over messages out of order (e.g. 
> SCTP, and imagine it running over UDP if that helps), then the benefit of 
> this is typically to get messages delivered faster (without receiver-side HOL 
> blocking)).
> But then, wouldn't it be good to have a way to tell the network "I don't care 
> about ordering" ?
> 
> It seems to me that we'd need a new codepoint for that.

Too few DiffServ codepoints for too many purposes available. :-)
Most of the DiffServ PHBs are observing the recommendation of RFC 2474:
"It is RECOMMENDED that PHB implementations do not introduce any packet
re-ordering within a microflow."

> But, it also seems to me that this couldn't get standardised because that 
> standard would embrace a layer violation (caring about a transport 
> connection), even though that has been implemented for ages.

Just from a logical perspective, a re-ordering property could be
_one_ attribute of a per-hop behavior (PHB), but a PHB
has very likely further properties that specify the packet
forwarding treatment. So probably re-ordering is probably
often orthogonal to other PHB features. But having a
new (best-effort + re-ordering tolerant) PHB could
be useful for some cases...

Regards,
 Roland
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] incremental deployment, transport and L4S (Re: when does the CoDel part of fq_codel help in the real world?)


On Thu, 29 Nov 2018, Jonathan Morton wrote:

I have to ask, why would the network care?  What optimisations can be 
obtained by reordering packets *within* a flow, when it's usually just 
as easy to deliver them in order?


Because most implementations aren't flow aware at all and might have 4 
queues, saying "oh, this single queue is for transports that don't care 
about ordering" means everything in that queue can just be sent as soon as 
it can, ignoring HOL caused by ARQ.


Of course, we already have FQ which reorders packets in *different* 
flows.  The benefits are obvious in that case.


FQ is a fringe in real life (speaking as a packet moving monkey). It's 
just on this mailing list that it's the norm.


--
Mikael Abrahamssonemail: swm...@swm.pp.se
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] incremental deployment, transport and L4S (Re: when does the CoDel part of fq_codel help in the real world?)

> On 29 Nov, 2018, at 2:12 pm, Michael Welzl  wrote:
> 
> But then, wouldn't it be good to have a way to tell the network "I don't care 
> about ordering" ?

I have to ask, why would the network care?  What optimisations can be obtained 
by reordering packets *within* a flow, when it's usually just as easy to 
deliver them in order?

Of course, we already have FQ which reorders packets in *different* flows.  The 
benefits are obvious in that case.

 - Jonathan Morton

___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] incremental deployment, transport and L4S (Re: when does the CoDel part of fq_codel help in the real world?)

> On 29 Nov, 2018, at 2:06 pm, Michael Welzl  wrote:
> 
>> That's my proposal.
> 
> - and it's an interesting one. Indeed, I wasn't aware that you're thinking of 
> a DCTCP-style signal from a string of packets.
> 
> Of course, this is hard to get right - there are many possible flavours to 
> ideas like this ... but yes, interesting!

I'm glad you think so.  Working title is ELR - Explicit Load Regulation.

As noted, this needs standardisation effort, which is a bit outside my realm of 
experience - Cake was a great success, but relied entirely on exploiting 
existing standards to their logical conclusions.  I think I started writing 
some material to put in an I-D, but got distracted by something more urgent.

If there's an opportunity to coordinate with relevant people from similar 
efforts, so much the better.  I wonder, for example, whether the DCTCP folks 
would be open to supporting a more deployable version of their idea, or whether 
that would be a political non-starter for them.

 - Jonathan Morton

___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] incremental deployment, transport and L4S (Re: when does the CoDel part of fq_codel help in the real world?)

I'm answering myself with an add-on thought:

> On 29 Nov 2018, at 09:08, Michael Welzl  wrote:
> 
> 
> 
>> On 29 Nov 2018, at 08:46, Mikael Abrahamsson  wrote:
>> 
>> On Thu, 29 Nov 2018, Jonathan Morton wrote:
>> 
>>> In my view, that is the wrong approach.  Better to improve Diffserv to the 
>>> point where it becomes useful in practice.
>> 
>> I agree, but unfortunately nobody has made me king of the Internet yet so I 
>> can't just decree it into existance.
> 
> Well, for what you want (re-ordering tolerance), I would think that the LE 
> codepoint is suitable. From:
> https://tools.ietf.org/html/draft-ietf-tsvwg-le-phb-06
> "there ought to be an expectation that packets of the LE PHB could be 
> excessively delayed or dropped when any other traffic is present"
> 
> ... I think it would be strange for an application to expect this, yet not 
> expect it to happen for only a few individual packets from a stream.

Actually, maybe this is a problem: the semantics of LE are way broader than 
"tolerant to re-ordering". What about applications that are 
reordering-tolerant, yet still latency critical?
E.g., if I use a protocol that can hand over messages out of order (e.g. SCTP, 
and imagine it running over UDP if that helps), then the benefit of this is 
typically to get messages delivered faster (without receiver-side HOL 
blocking)).
But then, wouldn't it be good to have a way to tell the network "I don't care 
about ordering" ?

It seems to me that we'd need a new codepoint for that.
But, it also seems to me that this couldn't get standardised because that 
standard would embrace a layer violation (caring about a transport connection), 
even though that has been implemented for ages.
:-(

Cheers,
Michael

___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] incremental deployment, transport and L4S (Re: when does the CoDel part of fq_codel help in the real world?)



> On 29 Nov 2018, at 11:30, Jonathan Morton  wrote:
> 
 My alternative use of ECT(1) is more in keeping with the other codepoints 
 represented by those two bits, to allow ECN to provide more fine-grained 
 information about congestion than it presently does.  The main challenge 
 is communicating the relevant information back to the sender upon receipt, 
 ideally without increasing overhead in the TCP/IP headers.
>>> 
>>> You need to go into the IETF process and voice this opinion then, because 
>>> if nobody opposes in the near time then ECT(1) might go to L4S 
>>> interpretation of what is going on. They do have ECN feedback mechanisms in 
>>> their proposal, have you read it? It's a whole suite of documents, 
>>> architecture, AQM proposal, transport proposal, the entire thing.
>>> 
>>> On the other hand, what you want to do and what L4S tries to do might be 
>>> closely related. It doesn't sound too far off.
>> 
>> Indeed I think that the proposal of finer-grain feedback using 2 bits 
>> instead of one is not adding anything to, but in fact strictly weaker than 
>> L4S, where the granularity is in the order of the number of packets that you 
>> sent per RTT, i.e. much higher.
> 
> An important facet you may be missing here is that we don't *only* have 2 
> bits to work with, but a whole sequence of packets carrying these 2-bit 
> codepoints.  We can convey fine-grained information by setting codepoints 
> stochastically or in a pattern, rather than by merely choosing one of the 
> three available (ignoring Not-ECT).  The receiver can then observe the 
> density of codepoints and report that to the sender.
> 
> Which is more-or-less the premise of DCTCP.  However, DCTCP changes the 
> meaning of CE, instead of making use of ECT(1), which I think is the big 
> mistake that makes it undeployable.
> 
> So, from the middlebox perspective, very little changes.  ECN-capable packets 
> still carry ECT(0) or ECT(1).  You still set CE on ECT packets, or drop 
> Non-ECT packets, to signal when a serious level of persistent queue has 
> developed, so that the sender needs to back off a lot.  But if a less serious 
> congestion condition exists, you can now signal *that* by changing some 
> proportion of ECT(0) codepoints to ECT(1), with the intention that senders 
> either reduce their cwnd growth rate, halt growth entirely, or enter a 
> gradual decline.  Those are three things that ECN cannot currently signal.
> 
> This change is invisible to existing, RFC-compliant, deployed middleboxes and 
> endpoints, so should be completely backwards-compatible and incrementally 
> deployable in the network.  (The only thing it breaks is the optional ECN 
> integrity RFC that, according to fairly recent measurements, literally nobody 
> bothered implementing.)
> 
> Through TCP Timestamps, both sender and receiver can know fairly precisely 
> when a round-trip has occurred.  The receiver can use this information to 
> calculate the ratio of ECT(0) and ECT(1) codepoints received in the most 
> recent RTT.  A new TCP Option could replace TCP Timestamps and the two bytes 
> of padding that usually go with it, allowing reporting of this ratio without 
> actually increasing the size of the TCP header.  Large cwnds can be 
> accommodated at the receiver by shifting both counters right until they both 
> fit in a byte each; it is the ratio between them that is significant.
> 
> It is then incumbent on the sender to do something useful with that 
> information.  A reasonable idea would be to aim for a 1:1 ratio via an 
> integrating control loop.  Receipt of even one ECT(1) signal might be 
> considered grounds for exiting slow-start, while exceeding 1:2 ratio should 
> limit growth rate to "Reno linear" semantics (significant for CUBIC), and 
> exceeding 2:1 ratio should trigger a "Reno linear" *decrease* of cwnd.  
> Through all this, a single CE mark (reported in the usual way via ECE and 
> CWR) still has the usual effect of a multiplicative decrease.
> 
> That's my proposal.

- and it's an interesting one. Indeed, I wasn't aware that you're thinking of a 
DCTCP-style signal from a string of packets.

Of course, this is hard to get right - there are many possible flavours to 
ideas like this ... but yes, interesting!

Cheers,
Michael

___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] incremental deployment, transport and L4S (Re: when does the CoDel part of fq_codel help in the real world?)

On Thu, 29 Nov 2018, Sebastian Moeller wrote:

As far as I can tell intel is pushing atom/x86 cores into its
docsis SoCs (puma5/6/7) as well as into the high-end dsl SoCs (formerly
lantiq,
https://www.intel.com/content/www/us/en/smart-home/anywan-grx750-home-gateway-brief.html?wapkw=grx750),
I am quite confident that those also pack enough punch for CPU based
routing at Gbps-rates. In docsis modems these are already rolled-out, I
do not know of any DSL modem/router that uses the GRX750

"10 Gbit/s packet processor".

Game over, again.

Call me naive, but the solution to the impasse at getting a common
definition of diffserv agreed upon is replacing all TCP CC algorithms?
This is replacing changing all endpoints (and network nodes) to honor
diffserve with changing all endpoints to use a different TCP CC. At
least I would call that ambitious (unless L4S offers noticeable
advantages for all participating without being terribly unfair to the
non-participating legacy TCP users*).

L4S proposes a separate queue for the L4S compatible traffic, and some
kind of fair split between L4S and non-L4S traffic. I guess it's kind of
along the lines of my earlier proposals about having some kind of fair
split with 3 queues for PHB LE, BE and the rest. It makes it deployable in
current HW without the worst kind of DDoS downsides imaginable.

The Internet is all about making things incrementally deployable. It's
very frustrating, but that's the way it is. Whatever we want to propose
needs to work so-so with what's already out there and it's ok if it takes
a while before it makes everything better.

I'd like diffserv to work better, but it would take a lot of work in the
operator community to bring it out to where it needs to be. It's not
hopeless though, and I think
https://tools.ietf.org/html/draft-ietf-tsvwg-le-phb-06 is one step in the
right direction. Just the fact that we might have two queues instead of
one in the simplest implementations might help. The first step is to get
ISPs to not bleach diffserv but at least allow 000xxx.

--
Mikael Abrahamssonemail: swm...@swm.pp.se
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] incremental deployment, transport and L4S (Re: when does the CoDel part of fq_codel help in the real world?)

>>> My alternative use of ECT(1) is more in keeping with the other codepoints 
>>> represented by those two bits, to allow ECN to provide more fine-grained 
>>> information about congestion than it presently does.  The main challenge is 
>>> communicating the relevant information back to the sender upon receipt, 
>>> ideally without increasing overhead in the TCP/IP headers.
>> 
>> You need to go into the IETF process and voice this opinion then, because if 
>> nobody opposes in the near time then ECT(1) might go to L4S interpretation 
>> of what is going on. They do have ECN feedback mechanisms in their proposal, 
>> have you read it? It's a whole suite of documents, architecture, AQM 
>> proposal, transport proposal, the entire thing.
>> 
>> On the other hand, what you want to do and what L4S tries to do might be 
>> closely related. It doesn't sound too far off.
> 
> Indeed I think that the proposal of finer-grain feedback using 2 bits instead 
> of one is not adding anything to, but in fact strictly weaker than L4S, where 
> the granularity is in the order of the number of packets that you sent per 
> RTT, i.e. much higher.

An important facet you may be missing here is that we don't *only* have 2 bits 
to work with, but a whole sequence of packets carrying these 2-bit codepoints.  
We can convey fine-grained information by setting codepoints stochastically or 
in a pattern, rather than by merely choosing one of the three available 
(ignoring Not-ECT).  The receiver can then observe the density of codepoints 
and report that to the sender.

Which is more-or-less the premise of DCTCP.  However, DCTCP changes the meaning 
of CE, instead of making use of ECT(1), which I think is the big mistake that 
makes it undeployable.

So, from the middlebox perspective, very little changes.  ECN-capable packets 
still carry ECT(0) or ECT(1).  You still set CE on ECT packets, or drop Non-ECT 
packets, to signal when a serious level of persistent queue has developed, so 
that the sender needs to back off a lot.  But if a less serious congestion 
condition exists, you can now signal *that* by changing some proportion of 
ECT(0) codepoints to ECT(1), with the intention that senders either reduce 
their cwnd growth rate, halt growth entirely, or enter a gradual decline.  
Those are three things that ECN cannot currently signal.

This change is invisible to existing, RFC-compliant, deployed middleboxes and 
endpoints, so should be completely backwards-compatible and incrementally 
deployable in the network.  (The only thing it breaks is the optional ECN 
integrity RFC that, according to fairly recent measurements, literally nobody 
bothered implementing.)

Through TCP Timestamps, both sender and receiver can know fairly precisely when 
a round-trip has occurred.  The receiver can use this information to calculate 
the ratio of ECT(0) and ECT(1) codepoints received in the most recent RTT.  A 
new TCP Option could replace TCP Timestamps and the two bytes of padding that 
usually go with it, allowing reporting of this ratio without actually 
increasing the size of the TCP header.  Large cwnds can be accommodated at the 
receiver by shifting both counters right until they both fit in a byte each; it 
is the ratio between them that is significant.

It is then incumbent on the sender to do something useful with that 
information.  A reasonable idea would be to aim for a 1:1 ratio via an 
integrating control loop.  Receipt of even one ECT(1) signal might be 
considered grounds for exiting slow-start, while exceeding 1:2 ratio should 
limit growth rate to "Reno linear" semantics (significant for CUBIC), and 
exceeding 2:1 ratio should trigger a "Reno linear" *decrease* of cwnd.  Through 
all this, a single CE mark (reported in the usual way via ECE and CWR) still 
has the usual effect of a multiplicative decrease.

That's my proposal.

 - Jonathan Morton


___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] incremental deployment, transport and L4S (Re: when does the CoDel part of fq_codel help in the real world?)

2018-11-29 Thread Sebastian Moeller

Hi Mikael,

> On Nov 29, 2018, at 08:46, Mikael Abrahamsson  wrote:
> 
> On Thu, 29 Nov 2018, Jonathan Morton wrote:
> 
>> You are essentially proposing using ECT(1) to take over an intended function 
>> of Diffserv.
> 
> Well, I am not proposing anything. I am giving people a heads-up that the L4S 
> authors are proposing this.
> 
> But yes, you're right. Diffserv has shown itself to be really hard to 
> incrementally deploy across the Internet, so it's generally bleached mid-path.
> 
>> In my view, that is the wrong approach.  Better to improve Diffserv to the 
>> point where it becomes useful in practice.
> 
> I agree, but unfortunately nobody has made me king of the Internet yet so I 
> can't just decree it into existance.

With your kind of clue, I would happily vote you as (temporary) king of 
the internet. ;)

> 
>> Cake has taken steps in that direction, by implementing some reasonable 
>> interpretation of some Diffserv codepoints.
> 
> Great. I don't know if I've asked this but is CAKE easily implementable in 
> hardware? From what I can tell it's still only Marvell that is trying to put 
> high performance enough CPUs into HGWs to do forwarding in CPU (which can do 
> CAKE), all others still rely on packet accelerators to achieve the desired 
> speeds.

As far as I can tell intel is pushing atom/x86 cores into its docsis 
SoCs (puma5/6/7) as well as into the high-end dsl SoCs (formerly lantiq, 
https://www.intel.com/content/www/us/en/smart-home/anywan-grx750-home-gateway-brief.html?wapkw=grx750),
 I am quite confident that those also pack enough punch for CPU based routing 
at Gbps-rates. In docsis modems these are already rolled-out, I do not know of 
any DSL modem/router that uses the GRX750

> 
>> My alternative use of ECT(1) is more in keeping with the other codepoints 
>> represented by those two bits, to allow ECN to provide more fine-grained 
>> information about congestion than it presently does.  The main challenge is 
>> communicating the relevant information back to the sender upon receipt, 
>> ideally without increasing overhead in the TCP/IP headers.
> 
> You need to go into the IETF process and voice this opinion then, because if 
> nobody opposes in the near time then ECT(1) might go to L4S interpretation of 
> what is going on. They do have ECN feedback mechanisms in their proposal, 
> have you read it? It's a whole suite of documents, architecture, AQM 
> proposal, transport proposal, the entire thing.
> 
> On the other hand, what you want to do and what L4S tries to do might be 
> closely related. It doesn't sound too far off.
> 
> Also, Bob Briscoe works for Cable Labs now, so he will now have silicon 
> behind him. This silicon might go into other things, not just DOCSIS 
> equipment, so if you have use-cases that L4S doesn't do but might do with 
> minor modification, it might be better to join him than to fight him.

Call me naive, but the solution to the impasse at getting a common definition 
of diffserv agreed upon is replacing all TCP CC algorithms? This is replacing 
changing all endpoints (and network nodes) to honor diffserve with changing all 
endpoints to use a different TCP CC. At least I would call that ambitious 
(unless L4S offers noticeable advantages for all participating without being 
terribly unfair to the non-participating legacy TCP users*).

Best Regards
Sebastian

*) Well, being unfair ad out-competing the legacy users would be the best way 
to incentivize everybody to upgrade, but that would also be true for a better 
diffserve scheme...

> 
> -- 
> Mikael Abrahamssonemail: swm...@swm.pp.se
> ___
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat

___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] incremental deployment, transport and L4S (Re: when does the CoDel part of fq_codel help in the real world?)


On Thu, 29 Nov 2018, Jonathan Morton wrote:


I'd say the important bits are only slightly harder than doing the same with 
fq_codel.


Ok, FQ_CODEL is way off to get implemented in HW. I haven't heard anyone 
even discussing it. Have you (or anyone else) heard differently?


I believe much of Cake's perceived CPU overhead is actually down to 
inefficiencies in the Linux network stack.  Using a CPU and some modest 
auxiliary hardware dedicated to moving packets, not tied up in handling 
general-purpose duties, then achieving greater efficiency with 
reasonable hardware costs could be quite easy, without losing the 
flexibility to change algorithms later.


I need to watch the MT7621 packet accelerator talk at the most recent 
OpenWrt summit. I installed OpenWrt 18.06.1 on an Mikrotik RB750vGR3 and 
just clicked my way around in LUCI and enabled flow offload and b00m, it 
now did full gig NAT44 forwarding. It's implemented as a -j FLOWOFFLOAD 
iptables rule. The good thing here might be that we could throw 
unimportant high speed flows off to the accelerator and then just handle 
the time sensitive flows in CPU, and just make sure the CPU has 
preferential access to the media for its time-sensitive flow. That kind of 
approach might make FQ_CODEL deployable even on slow CPU platforms with 
accelerators because you would only run some flows through FQ_CODEL, where 
the bulk high-speed flows would be handed off to acceleration (and we 
guess they don't care about PDV and bufferbloat).


--
Mikael Abrahamssonemail: swm...@swm.pp.se
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] incremental deployment, transport and L4S (Re: when does the CoDel part of fq_codel help in the real world?)

> On 29 Nov, 2018, at 9:46 am, Mikael Abrahamsson  wrote:
> 
> I don't know if I've asked this but is CAKE easily implementable in hardware?

I'd say the important bits are only slightly harder than doing the same with 
fq_codel.  Some of the less important details might be significantly harder, 
and could reasonably be left out.  The Diffserv bit should be nearly trivial to 
put in.

I believe much of Cake's perceived CPU overhead is actually down to 
inefficiencies in the Linux network stack.  Using a CPU and some modest 
auxiliary hardware dedicated to moving packets, not tied up in handling 
general-purpose duties, then achieving greater efficiency with reasonable 
hardware costs could be quite easy, without losing the flexibility to change 
algorithms later.

 - Jonathan Morton

___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] incremental deployment, transport and L4S (Re: when does the CoDel part of fq_codel help in the real world?)