On Mon, Jun 15, 2015 at 5:12 PM, Agarwal, Anil <[email protected]> wrote: > Dave, > > I guess I need to read up on "cake".
some basic doc is at: http://www.bufferbloat.net/projects/codel/wiki/Cake The most important thing in cake at the moment is GRO packet peeling, which turned out desperately needed in all the new router hardware we have encountered. Huge wins there. the other stuff is not fully baked or implemented yet. We are in a bit of a debate about the most troublesome and misunderstood aspect of codel on the list over there. > If you have time, can you simulate an RTT of 600 ms? ok. > With a few queue drain rates from 1 Mbps to 100 Mbps. 10,50,100,200,300 was my planned range, fed by gigE. I can't go much faster than than that. (and even as low as 300 requires GRO offloads) > Would help us "satellite folks" get a better understanding of CoDel > parameters. I have not looked at the very long rtt problem in several years, and since then tcps, in particular have changed muchly (pacing in particular) > > Thanks, > Anil > > -----Original Message----- > From: Dave Taht [mailto:[email protected]] > Sent: Monday, June 15, 2015 7:57 PM > To: Agarwal, Anil > Cc: Simon Barber; Jonathan Morton; [email protected]; Steven Blake > Subject: Re: [aqm] CoDel on high-speed links > > On Mon, Jun 15, 2015 at 4:40 PM, Agarwal, Anil <[email protected]> > wrote: >> I guess this is pointing to the age old problem - what is the right buffer >> size or equivalent delay limit, when packets should be dropped or >> ECN-marked, so that the link is never under-utilized? >> >> For a single TCP connection, the answer is the bandwidth-delay product BDP. >> For large number of connections, it is BDP / sqrt(numConnections). >> >> Hence, one size does not fit all. >> E.g., for RTT of 100 ms or 500 ms, CoDel target delay of 5 or 10 ms is too >> short - when handling a small number of connections. > > The codel recommendation is that the target be set to 5-10% of the typical > interval (RTT), so in the case of a sat link, interval would be 600(?)ms, and > target 5-10 % of that, 30-60ms. > > The recommendation bears testing in this scenario. If you would like to > specify what you expect your (say) 98th percentile real physical RTT, I can > exhaustively simulate that rather rapidly later this week against reno, cdg, > cubic, and westwood and various aqms, against flows from 1 to 100. > > Most of this convo has also missed other advancements in tcp (reno is > thoroughly dead), like PRR. A very good read which incorporates a discussion > of PRR is > > http://folk.uio.no/kennetkl/jonassen_thesis.pdf > > which I plan to finish reading on the plane. > > I am not sure what the pie settings would be for a sat system. > >> Perhaps, there is a need for a design that adapts the queue size (or delay) >> target dynamically by estimating numConnections ! > > Not "perhaps". That is one of the avenues cake is exploring: > > https://lists.bufferbloat.net/pipermail/cake/2015-June/000241.html > >> Anil >> >> >> -----Original Message----- >> From: aqm [mailto:[email protected]] On Behalf Of Simon Barber >> Sent: Monday, June 15, 2015 2:01 PM >> To: Dave Taht >> Cc: Jonathan Morton; [email protected]; Steven Blake >> Subject: Re: [aqm] CoDel on high-speed links >> >> On 6/14/2015 10:26 PM, Dave Taht wrote: >>> On Sun, Jun 14, 2015 at 4:10 PM, Simon Barber <[email protected]> wrote: >>>> Indeed - I believe that Codel will drop too much to allow maximum >>>> bandwidth utilization, when there are very few flows, and RTT is >>>> significantly greater than target. >>> Interval. Not target. Interval defaults to 100ms. Target is 5ms. >>> Dropping behaviors stop when the queue falls below the target. >> >> In this case I specifically mean target, not interval. Dropping stops when >> queue falls below target, but by then it's too late. In the case I'm talking >> about (cwind cut by more than queue length) then a period of link idle >> occurs, and so bandwidth is hurt. It happens repeatedly. >> >>> Range of tests from near zero to 300ms RTT codel does quite well with >>> reno, better with cubic, on single flows. 4 flows, better. fq_codel >>> does better than that on more than X flows in general. >> The effect is not huge, but the bandwidth loss is there. More flows >> significantly reduce the effect, since the other flows keep the link busy. >> This bandwidth reduction effect only happens with very few flows. >> I think TCP Reno will be worse than Cubic, due to it's 50% reduction in >> cwind on drop vs Cubic's 20% reduction - but Cubic's RTT independent >> increase in cwind after the drop may make the effect happen more often with >> larger RTTs. >> >> What results have you seen for codel on single flow for these larger RTTs? >> >>> You can easily do whatever experiments you like with off the shelf >>> hardware and RTTs around half the planet to get the observations you >>> need to confirm your thinking. Remember that a drop tail queue of >>> various sizes has problems of it's own. >>> >>> I have a long overdue rant in progress of being wikified about how to >>> use netem correctly to properly emulate any rtt you like. >>> >>> I note that a main aqm goal is not maximum bandwidth utilization, but >>> maximum bandwidth while still having working congestion avoidance and >>> minimal queue depth so other new flows can rapidly grab their fair >>> share of the link. The bufferbloat problem was the result of wanting >>> maximum bandwidth for single flows. >> Indeed - with many TCP CC algorithms it's just not possible to achieve >> maximum bandwidth utilization with only 5ms induced latency when the RTTs >> are long, and a single queue (no FQ, only drop tail or single queue AQM). >> The multiplicative decrease part of TCP CC simply does not allow it unless >> the decrease is smaller than the queue (PRR might mitigate a little here). >> Now add in FQ and you can have the best of both worlds. >>>> The theory is - with a Reno based CC the cwind gets cut in half on a >>>> drop. If the drop in cwind is greater than the number of packets in >>>> the queue, then the queue will empty out, and the link will then be >>>> idle for a >>> flight + queue. >> When cwind gets cut by N packets, the sender stops sending data while ACKs >> for N data packets are received. If the queue has less than N data packets, >> then it will empty out, resulting in an idle link at that point, and >> eventually at the receiver (hence bandwidth loss). >>>> while. If you want data to keep the data flowing uninterrupted, then >>>> you must have a full unloaded RTT's worth of data in the queue at >>>> that point. A >>> Do the experiment? Recently landed in flent is the ability to monitor >>> queue depth while running another test. >>> >>>> drop will happen, the cwind will be halved (assuming a Reno TCP), >>>> and the sender will stop sending until one (unloaded) RTT's worth of >>>> data has been received. At that point the queue will just hit empty >>>> as the sender starts sending again. >>> And reno is dead. Long live reno! >>> >>>> Simon >>>> >>>> >>>> >>>> On 6/9/2015 10:30 AM, Jonathan Morton wrote: >>>>> >>>>> Wouldn't that be a sign of dropping too much, in contrast to your >>>>> previous post suggesting it wouldn't drop enough? >>>>> >>>>> In practice, statistical multiplexing works just fine with >>>>> fq_codel, and you do in fact get more throughput with multiple >>>>> flows in those cases where a single flow fails to each adequate >>>>> utilisation. >>>>> >>>>> Additionally, utilisation below 100% is really characteristic of >>>>> Reno on any worthwhile AQM queue and significant RTT. Other TCPs, >>>>> particularly CUBIC and Westwood+, do rather better. >>>>> >>>>> - Jonathan Morton >>>>> >>>> _______________________________________________ >>>> aqm mailing list >>>> [email protected] >>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_ma >>>> i >>>> lman_listinfo_aqm&d=AwICAg&c=jcv3orpCsv7C4ly8-ubDob57ycZ4jvhoYZNDBA0 >>>> 6 >>>> fPk&r=FyvaklKYrHaSCPjbBTdviWIW9uSbnxdNSheSGz1Jvq4&m=uxNYRwC4x80YZPJP >>>> Y >>>> o3-lMaVCC_1TNJzTxVGd0F1PSs&s=6OQXBky7MzGz1dBmf7oRhwemCi5a4yAiQRm90WH >>>> X >>>> FIg&e= >>> >>> >> >> _______________________________________________ >> aqm mailing list >> [email protected] >> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mail >> man_listinfo_aqm&d=AwICAg&c=jcv3orpCsv7C4ly8-ubDob57ycZ4jvhoYZNDBA06fP >> k&r=FyvaklKYrHaSCPjbBTdviWIW9uSbnxdNSheSGz1Jvq4&m=uxNYRwC4x80YZPJPYo3- >> lMaVCC_1TNJzTxVGd0F1PSs&s=6OQXBky7MzGz1dBmf7oRhwemCi5a4yAiQRm90WHXFIg& >> e= > > > > -- > Dave Täht > What will it take to vastly improve wifi for everyone? > https://plus.google.com/u/0/explore/makewififast -- Dave Täht What will it take to vastly improve wifi for everyone? https://plus.google.com/u/0/explore/makewififast _______________________________________________ aqm mailing list [email protected] https://www.ietf.org/mailman/listinfo/aqm
