Re: [Cake] [Starlink] How to help?

[Dave Täht via Cake]

We had a ton of email bounce last week.

On Fri, Jun 10, 2022 at 10:21:55AM -0500, Nick Hall via Starlink wrote:
> Hello,
> 
> I've had a Starlink (the round version 1) for around a year and was
> thinking about bufferbloat yesterday and just found your mailing list. I
> just started looking at the archives but I'm gathering that you are looking
> for people to test things and am wondering what I can do to help?

yes.

> I have used flent a little before, and only know the basics, but I can run
> tests if you give me direction for what you need.
> 
> I am not using the provided Starlink router but instead the dish is
> connected directly to my EdgeRouter X which is running the 1.10 series
> EdgeRouter firmware with Cake from
> https://community.ui.com/questions/Cake-compiled-for-the-EdgeRouter-devices/fc1ff27c-f321-4344-8737-fcc755cae8a2
> 
> I see a link to running CAKE with adaptive bandwidth:
> https://forum.openwrt.org/t/cake-w-adaptive-bandwidth/108848
> 
> But this is running OpenWrt. I know OpenWrt can run on the EdgeRouter X but
> I can't spend the level of time to flash the router that it would probably
> need right now. Do you know if that script would be able to run on the
> version of Cake that I currently have running on the EdgeRouter X? Or have
> necessary improvements been made to Cake that my version wouldn't have?

So far as I know the edgerouter version of cake is pretty current.

the script requires some things about timings that egerouter
may not have.

> 
> Thanks,
> 
> Nick

> ___
> Starlink mailing list
> starl...@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/starlink


-- 
Fixing the Internet on a too-tight budget: https://patreon.com/dtaht
Dave Taht, Chief bottlewasher, TekLibre

___
Cake mailing list
Cake@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/cake

Re: [Cake] [LEDE-DEV] Cake SQM killing my DIR-860L - was: [17.01] Kernel: bump to 4.4.51

[Dave Täht]

On 3/2/17 11:51 AM, Stijn Segers wrote:
> Thanks Sebastian, turned out to be a silly syntax error, I have it all
> disabled now. Ethtool -k and ethtool -K printing/requiring different
> stuff doesn't help of course :-)
> 
> I re-enabled SQM, will see how that works out with the offloading disabled.

Would be good to know. I lost a bit of sleep lately (given how badly we
got bit by RCU on the ATF front, I worry about cake... but I can't see
how that would break, there.)

In terms of general "why does shaping use so much cpu"...

I am keen to stress that the core fq_codel algorithm is very lightweight
and barely shows up on traces when used without software rate limiting
and with BQL.

You CAN see a difference in forwarding performance at really high native
rates if you use pfifo and compare it to fq_codel on some platforms -
pfifo-fast is simpler overall. To experiment, you can re-enable
pfifo-fast in scenarios if you want - (tc qdisc add dev whatever pfifo
limit somethingsane, or bfifo something sane)

... however things like nat and firewall rules tend to dominate the
forwarding costs, and fq_codel reduces latency muchly over pfifo), and
the principal use of fq_codel is for sqm (and now wifi).

As for software rate shaping - this is very cpu intensive no matter how
you do it. I wish we didn't have to do it - and with certain (mostly old
DSL) modems that do flow control you don't.

The only one I know that gets this right is the transverse geode that
david woodhouse has. One of my disappointments across the industry is
not seeing BQL roll out universally on any dsl firmwares, starting, oh,
5 years ago.

If we had ethernet devices with a programmable timer (only interrupt me
on 40mbit rate) we could also completely eliminate software rate shaping

anyway my benchmarks are showing that:

cake in it's "besteffort" mode smokes HTB + fq_codel, affording
over 40% more headroom in terms of cpu with bandwidth. (Independent
confirmation across more cpu types is need)

In the default mode, with the new 3 tier classification, wash, nat and
triple-isolate/dual-host/dual-src features - which we hope are going to
help folk deal with torrent better in particular - it's a wash.

cake is a LOT more cpu intense than fq_codel is, especially in its
default modes, which it makes up for by being more unified. Mostly.

If you are running low on cpu and are trying to shape inbound on most of
these low-end mips devices to speeds > 60Mbits, I'd highly recommend
switching to using "besteffort" on that rather than the 3 QoS queue
default. Most ISPs are not classifying traffic well, anyway, and FQ
solves nearly everything, especially per host fq

But none of what I just said applies if there's a bug somewhere else!
GRO has given me fits for years now, and I'm scarred by that.

In terms of cpu costs in cake/fq_codel - dequeue, hashing, and
timestamping show up most on a trace. The rate limiting effort where all
that is happening shows up in softirq dominating the platform.

I have *always* worried that there exists devices (particularly
multi-cores) without a first class high speed internal clock facility,
but thus far haven't had an issue with it (unlike on BSD, which has
internal timings good to only a ms).

As for speeding up hashing, I've been looking over various algorithms to
do that for years now, I'm open to suggestions. The fastest new ones
tend to depend on co-processor support. The fastest I've seen relies on
the CRC32 instruction which is only in some intel platforms.

Cake could certainly use a big round of profiling but it is generally my
hope that we won big with it, in its present form.

I welcome (especially flent) benchmarks of sqm on various architectures
we've not explored fully - notably arm ones -

My hat is off to all that have worked so hard to make this subsystem -
and also all of lede - work so well, in this release.




> Cheers
> 
> Stijn
> 
> 
> ___
> Lede-dev mailing list
> lede-...@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/lede-dev
___
Cake mailing list
Cake@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/cake

Re: [Cake] [Make-wifi-fast] Flent results for point-to-point Wi-Fi on LEDE/OM2P-HS available

[Dave Täht]

On 2/14/17 12:56 AM, Pete Heist wrote:
> 
>> On Jan 31, 2017, at 12:21 AM, Dave Taht  wrote:
>>
>> It would be my hope that 802.11e is off (rrul will show this, and we
>> still do badly with it on)
> 
> Does this mean to try disabling WMM (uci set wireless.default_radio0.wmm='0’)?
> 
> That’s how I took it, as it looks like WMM is a subset of 802.11e. But if I 
> do that, I lose 802.11n support and throughput drops from ~90 Mbps to ~22 
> Mbps. I could get those results out of interest, but it wouldn’t be practical 
> in the field.
> 
> Or is there another way to turn off 802.11e without losing 802.11n support?

you are correct. I misspoke. turning off wmm disables 802.11n. Which I'd
forgotten until I too ran a test with wmm disabled a week or two back.

What I'd done in cerowrt was disable the internal mac80211 QoS
classifier entirely. (-1 line of code). Another way to do it is to apply
an iptables rule to wash away all diffserv markings, but I don't care
for that answer.

try as I might I cannot remember if this was even possible in hostapd or
in the mac layer any other way. I'll look this week.

> Pete
> 
> ___
> Make-wifi-fast mailing list
> make-wifi-f...@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/make-wifi-fast
> 
___
Cake mailing list
Cake@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/cake

Re: [Cake] cake for net-next 4.8

[Dave Täht]

On 10/3/16 2:17 PM, Neil Shepperd wrote:
> Gmail spam team got back to me; they have apparently "fixed the
> problem", which I guess means messages here should stop being marked as
> spam soon. Because gmail's spam filters are top secret business, I
> couldn't tell you anything else :)

It is great to have friends (in places high and low). Thank you!

That said, I almost, but not quite, got the dkim stuff working the other
day - not that I can intuit that was the source of the problem!

> 
> On Fri, 30 Sep 2016 at 17:10 Dave Täht  <mailto:d...@taht.net>> wrote:
> 
> 
> 
> On 9/30/16 1:37 PM, Neil Shepperd wrote:
> > Disabling ipv6 (at least in the mail server, in outgoing direction) is
> > probably the easiest option...
> 
> It looks like the simplest thing I could do to allow inbound while
> stopping outbound ipv6 would be to:
> 
> /etc/postfix/main.cf <http://main.cf>:
> smtp_bind_address6 = ::1
> 
> 
> > I see on most messages here DKIM-Signature headers apparently from
> > gmail: "v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com
> <http://gmail.com>
> > <http://gmail.com>; s=20120113;". These signatures are failing because
> > of the added message footer. No sign of a DKIM-Signature
> > for lists.bufferbloat.net <http://lists.bufferbloat.net>
> <http://lists.bufferbloat.net>. You'd need to
> > set that up in the list MTA.
> 
> Honestly my "email-fu" has declined considerably in recent years.
> Despite the apparent simplicity of this idea, my brain just crashed
> multiple times on setting it up with postfix + mailman 2.
> 
> And thank you for poking so deeply into this, I was A) really annoyed by
> the bloat-list-as-spam thing and B) clueless.
> 
> > On Fri, 30 Sep 2016 at 15:42 Dave Täht  <mailto:d...@taht.net>
> > <mailto:d...@taht.net <mailto:d...@taht.net>>> wrote:
> >
> >
> >
> > On 9/30/16 1:02 AM, Toke Høiland-Jørgensen wrote:
> > > Neil Shepperd  <mailto:nshepp...@gmail.com> <mailto:nshepp...@gmail.com
> <mailto:nshepp...@gmail.com>>>
> > writes:
> > >
> > >> I think I have now accumulated enough spam/nonspam
> classified emails
> > >> to make a statistically signification observation: it seems
> like all
> > >> emails classified as spam from these lists were send from ipv6:
> > >>
> > >> SPF: PASS with IP 2600:3c03:0:0:f03c:91ff:fe61:86ce
> > >>
> > >> All emails from bufferbloat.net <http://bufferbloat.net>
> <http://bufferbloat.net> lists
> > are failing DKIM (because of the
> > >> mailing list footer breaking the DKIM signature) which
> might be worth
> > >> fixing, and failing DMARC because all mailing lists fails DMARC
> > >> (however google does not have a strict DMARC policy so that
> shouldn't
> > >> matter, I hope).
> > >>
> > >> By the way, it's not just you, either. I have emails from
> others on
> > >> these lists in my spam folder.
> > >>
> > >> The distinguishing factor seems to be whether the email was
> sent from
> > >> the lists.bufferbloat.net <http://lists.bufferbloat.net>
> <http://lists.bufferbloat.net> ipv6
> > address. Unless this address
> > >> corresponds to some kind of tunnel broker possibly also used by
> > >> spammers, I can only assume this is some kind of bug (after
> all, it
> > >> was spf validated so the address shouldn't matter at that
> point?).
> > >
> > > Indeed, gmail requires extra measures for IPv6:
> > > https://support.google.com/mail/answer/81126 (scroll down to
> > "Additional
> > > guidelines for IPv6").
> > >
> > > Fixing DKIM might be worthwhile :)
> >
> > But it passes the spf check?? And the reverse lookup is correct.
> >
> > How about I just disable ipv6?
> >
> > Have no idea why dkim doesn't work.
> >
> > >
> > > -Toke
> > > ___
> > > Cake mailing list
> > > Cake@lists.bufferbloat.net
> <mailto:Cake@lists.bufferbloat.net>
> <mailto:Cake@lists.bufferbloat.net <mailto:Cake@lists.bufferbloat.net>>
> > > https://lists.bufferbloat.net/listinfo/cake
> > >
> > ___
> > Cake mailing list
> > Cake@lists.bufferbloat.net <mailto:Cake@lists.bufferbloat.net>
> <mailto:Cake@lists.bufferbloat.net <mailto:Cake@lists.bufferbloat.net>>
> > https://lists.bufferbloat.net/listinfo/cake
> >
> 
___
Cake mailing list
Cake@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/cake

Re: [Cake] cake for net-next 4.8

[Dave Täht]

On 9/30/16 1:37 PM, Neil Shepperd wrote:
> Disabling ipv6 (at least in the mail server, in outgoing direction) is
> probably the easiest option...

It looks like the simplest thing I could do to allow inbound while
stopping outbound ipv6 would be to:

/etc/postfix/main.cf:
smtp_bind_address6 = ::1


> I see on most messages here DKIM-Signature headers apparently from
> gmail: "v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com
> <http://gmail.com>; s=20120113;". These signatures are failing because
> of the added message footer. No sign of a DKIM-Signature
> for lists.bufferbloat.net <http://lists.bufferbloat.net>. You'd need to
> set that up in the list MTA.

Honestly my "email-fu" has declined considerably in recent years.
Despite the apparent simplicity of this idea, my brain just crashed
multiple times on setting it up with postfix + mailman 2.

And thank you for poking so deeply into this, I was A) really annoyed by
the bloat-list-as-spam thing and B) clueless.

> On Fri, 30 Sep 2016 at 15:42 Dave Täht  <mailto:d...@taht.net>> wrote:
> 
> 
> 
> On 9/30/16 1:02 AM, Toke Høiland-Jørgensen wrote:
> > Neil Shepperd mailto:nshepp...@gmail.com>>
> writes:
> >
> >> I think I have now accumulated enough spam/nonspam classified emails
> >> to make a statistically signification observation: it seems like all
> >> emails classified as spam from these lists were send from ipv6:
> >>
> >> SPF: PASS with IP 2600:3c03:0:0:f03c:91ff:fe61:86ce
> >>
> >> All emails from bufferbloat.net <http://bufferbloat.net> lists
> are failing DKIM (because of the
> >> mailing list footer breaking the DKIM signature) which might be worth
> >> fixing, and failing DMARC because all mailing lists fails DMARC
> >> (however google does not have a strict DMARC policy so that shouldn't
> >> matter, I hope).
> >>
> >> By the way, it's not just you, either. I have emails from others on
> >> these lists in my spam folder.
> >>
> >> The distinguishing factor seems to be whether the email was sent from
> >> the lists.bufferbloat.net <http://lists.bufferbloat.net> ipv6
> address. Unless this address
> >> corresponds to some kind of tunnel broker possibly also used by
> >> spammers, I can only assume this is some kind of bug (after all, it
> >> was spf validated so the address shouldn't matter at that point?).
> >
> > Indeed, gmail requires extra measures for IPv6:
> > https://support.google.com/mail/answer/81126 (scroll down to
> "Additional
> > guidelines for IPv6").
> >
> > Fixing DKIM might be worthwhile :)
> 
> But it passes the spf check?? And the reverse lookup is correct.
> 
> How about I just disable ipv6?
> 
> Have no idea why dkim doesn't work.
> 
> >
> > -Toke
> > ___
> > Cake mailing list
> > Cake@lists.bufferbloat.net <mailto:Cake@lists.bufferbloat.net>
> > https://lists.bufferbloat.net/listinfo/cake
> >
> ___
> Cake mailing list
> Cake@lists.bufferbloat.net <mailto:Cake@lists.bufferbloat.net>
> https://lists.bufferbloat.net/listinfo/cake
> 
___
Cake mailing list
Cake@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/cake

Re: [Cake] cake for net-next 4.8

[Dave Täht]

On 9/30/16 1:02 AM, Toke Høiland-Jørgensen wrote:
> Neil Shepperd  writes:
> 
>> I think I have now accumulated enough spam/nonspam classified emails
>> to make a statistically signification observation: it seems like all
>> emails classified as spam from these lists were send from ipv6:
>>
>> SPF: PASS with IP 2600:3c03:0:0:f03c:91ff:fe61:86ce  
>>
>> All emails from bufferbloat.net lists are failing DKIM (because of the
>> mailing list footer breaking the DKIM signature) which might be worth
>> fixing, and failing DMARC because all mailing lists fails DMARC
>> (however google does not have a strict DMARC policy so that shouldn't
>> matter, I hope).
>>
>> By the way, it's not just you, either. I have emails from others on
>> these lists in my spam folder.
>>
>> The distinguishing factor seems to be whether the email was sent from
>> the lists.bufferbloat.net ipv6 address. Unless this address
>> corresponds to some kind of tunnel broker possibly also used by
>> spammers, I can only assume this is some kind of bug (after all, it
>> was spf validated so the address shouldn't matter at that point?).
> 
> Indeed, gmail requires extra measures for IPv6:
> https://support.google.com/mail/answer/81126 (scroll down to "Additional
> guidelines for IPv6").
> 
> Fixing DKIM might be worthwhile :)

But it passes the spf check?? And the reverse lookup is correct.

How about I just disable ipv6?

Have no idea why dkim doesn't work.

> 
> -Toke
> ___
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake
> 
___
Cake mailing list
Cake@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/cake

[Cake] edgerouter gains cake

[Dave Täht]

http://community.ubnt.com/t5/EdgeMAX/Cake-compiled-for-the-ERL/m-p/1679844
___
Cake mailing list
Cake@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/cake

Re: [Cake] CAKE on FTTH in Mauritius

[Dave Täht]

On 7/31/16 10:29 AM, Loganaden Velvindron wrote:
> Thanks, and here's the result with transmission running on another box
> in the house:
> 
> https://www.dslreports.com/speedtest/4583052
> 
> Something that I notice right away is that it seems that CAKE has less
> "burstiness" than fq_codel in simple.qos configuration.
> 
> Btw, I'm using piece of cake as the QoS profile.

Well, I would try to stress it out harder against a more local netperf
server. If you can set one up on a data center in your country I can
toss it into the global dns as "netperf-something.bufferbloat.net"

> 
> 
> 
> On Sun, Jul 31, 2016 at 12:16 PM, Dave Taht  wrote:
>> Pretty impressive result.
>>
>> On Sun, Jul 31, 2016 at 10:13 AM, Loganaden Velvindron
>>  wrote:
>>> Hi folks,
>>>
>>> Got myself an archer c7 v2 with lede: (HEAD, r1185), on a 30Mbit/s
>>> (download) and 4Mbit/s upload.
>>>
>>> here is the result using dslreports:
>>> https://www.dslreports.com/speedtest/4582998.
>>>
>>> It looks to be working fine to me so far.
>>>
>>>  capacity estimate: 3Mbit
>>>  Tin 0
>>>   thresh 3Mbit
>>>   target15.0ms
>>>   interval 300.0ms
>>>   pk_delay   790us
>>>   av_delay   318us
>>>   sp_delay10us
>>>   pkts   23772
>>>   bytes   10352566
>>>   way_inds   0
>>>   way_miss 295
>>>   way_cols   0
>>>   drops 56
>>>   marks  0
>>>   sp_flows   1
>>>   bk_flows   1
>>>   un_flows   0
>>>   max_len 1514
>>> ___
>>> Cake mailing list
>>> Cake@lists.bufferbloat.net
>>> https://lists.bufferbloat.net/listinfo/cake
>>
>>
>>
>> --
>> Dave Täht
>> Let's go make home routers and wifi faster! With better software!
>> http://blog.cerowrt.org
> ___
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake
> 
___
Cake mailing list
Cake@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/cake

Re: [Cake] Cake strange behaviour

[Dave Täht]

On 7/16/16 1:53 PM, Kevin Darbyshire-Bryant wrote:
> 
> 
> On 16/07/16 11:59, Dave Täht wrote:
>> I would repeat the same test with htb+fq_codel.
> 
> Hi Dave,
> 
> That's more challenging than it sounds - reproducing the test scenario
> would require the windows box going back in time.  What could it be
> doing that so far any of the flent tests fail to replicate?  Hmmm, so
> far I've used a local flent server...I wonder if RTT is at play here?

Without extensive testing I have no faith that the current aqm
implementation in cake is actually working. In the case of inbound
traffic, managed at the cpe rather than at the isp, fq alone does not
manage to reduce queue length.

As for the actual characeristics of microsofts new distribution system,
I have heard many reports of it slamming networks, but have no traces
to work from. A suggestion to get some would be to put a freshly
installed windows box on the network and capture all that happens.

> Kevin
> 
>>
>> On 7/16/16 11:35 AM, Kevin Darbyshire-Bryant wrote:
>>> Hi guys,
>>>
>>> Encountering some behaviour that I don't understand.  Line is a 40/10
>>> cake limited to 39000/9840. Overheads 12, 'dual-dsthosts' in ingress,
>>> 'dual-srcshosts' on engress - limiting the on the WAN line. Take a look
>>> at my ping response graph
>>>
>>> http://www.thinkbroadband.com/ping/share/9822cb5160582fa6abee29b60d807766-16-07-2016.html
>>>
>>>
>>>
>>> Around 20:30 I fired up a windows machine that was behind on its updates
>>> so it generated a bit of ingress traffic.  Note the comparatively high
>>> latency (40ms) and stupidly high ping packet loss (50%)  The 3-5ms
>>> steady (blue) latency you can see is a system backup (so egress traffic)
>>> running till around 23:00.
>>>
>>> The really strange bit is that cake stats show it has only dropped 10
>>> (yes 10!) packets.
>>>
>>> I'm not the only person encountering 'interesting' behaviour with regard
>>> to windows updates inducing high latency and high packet loss.  It's as
>>> if cake weren't there managing flows and this is the ISP's rate limiter
>>> in action.
>>>
>>> Kevin
>>>
>>> ___
>>> Cake mailing list
>>> Cake@lists.bufferbloat.net
>>> https://lists.bufferbloat.net/listinfo/cake
>> ___
>> Cake mailing list
>> Cake@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/cake
> 
> ___
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake
___
Cake mailing list
Cake@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/cake

Re: [Cake] Cake strange behaviour

[Dave Täht]

I would repeat the same test with htb+fq_codel.

On 7/16/16 11:35 AM, Kevin Darbyshire-Bryant wrote:
> Hi guys,
> 
> Encountering some behaviour that I don't understand.  Line is a 40/10
> cake limited to 39000/9840. Overheads 12, 'dual-dsthosts' in ingress,
> 'dual-srcshosts' on engress - limiting the on the WAN line. Take a look
> at my ping response graph
> 
> http://www.thinkbroadband.com/ping/share/9822cb5160582fa6abee29b60d807766-16-07-2016.html
> 
> 
> Around 20:30 I fired up a windows machine that was behind on its updates
> so it generated a bit of ingress traffic.  Note the comparatively high
> latency (40ms) and stupidly high ping packet loss (50%)  The 3-5ms
> steady (blue) latency you can see is a system backup (so egress traffic)
> running till around 23:00.
> 
> The really strange bit is that cake stats show it has only dropped 10
> (yes 10!) packets.
> 
> I'm not the only person encountering 'interesting' behaviour with regard
> to windows updates inducing high latency and high packet loss.  It's as
> if cake weren't there managing flows and this is the ISP's rate limiter
> in action.
> 
> Kevin
> 
> ___
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake
___
Cake mailing list
Cake@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/cake

[Cake] conntrack and ipv6

[Dave Täht]

It is generally my hope that ipv6 nat will not be widely deployed.

Firewalls will be stateful instead, and thus there would be no need to
access the conntrack information for ipv6 in cake.

I'm not sure, however, to
what extent ipv6 conntrack is in openwrt today, certainly udp and tcp,
"in" is essentially blocked by default, and needs to be triggered by an
outgoing message. Similarly I'm unfamiliar with the state of ipv6 upnp
and pcp support in openwrt or client applications at present.


On 6/30/16 10:33 AM, Kevin Darbyshire-Bryant wrote:
> 
> 
> On 02/06/16 13:29, Jonathan Morton wrote:
>>> On 2 Jun, 2016, at 14:09, Kevin Darbyshire-Bryant
>>>  wrote:
>>>
>>> Cake uses the flow dissector API to do flow hashing...including per
>>> host flows for dual/triple isolation.  The unfortunate bit is that
>>> the qdisc inevitably gets placed after packets have been NATed on
>>> egress and before they've been de-NATed on ingress.
>>>
>>> When mentioned before Johnathan said "flow dissector ideally needs to
>>> be tweaked to do this" or words to that effect.
>>>
>>> I'd like to progress that idea...the thought of me kernel programming
>>> should horrify everyone but really I'm asking for help in being
>>> pointed in the right direction to ask for help...and go from there :-)
>> I believe Linux does NAT using a “connection tracker” subsystem.  That
>> would contain the necessary data for resolving NAT equivalents.  I
>> don’t know how easy it is to query in a qdisc context, though.
> Imagine my joy of discovering http://fatooh.org/esfq-2.6/  - someone has
> already bl**dy done itand I found it lurking in LEDE as part of a
> patch.
> 
> So there relevant bits are something of the order:
> 
> 
> +#ifdef CONFIG_NET_SCH_ESFQ_NFCT
> +   enum ip_conntrack_info ctinfo;
> +   struct nf_conn *ct = nf_ct_get(skb, &ctinfo);
> +#endif
> 
> +#ifdef CONFIG_NET_SCH_ESFQ_NFCT
> +   /* defaults if there is no conntrack info */
> +   info.ctorigsrc = info.src;
> +   info.ctorigdst = info.dst;
> +   info.ctreplsrc = info.dst;
> +   info.ctrepldst = info.src;
> +   /* collect conntrack info */
> +   if (ct && ct != &nf_conntrack_untracked) {
> +   if (skb->protocol == __constant_htons(ETH_P_IP)) {
> +   info.ctorigsrc =
> ct->tuplehash[IP_CT_DIR_ORIGINAL].tuple.src.u3.ip;
> +   info.ctorigdst =
> ct->tuplehash[IP_CT_DIR_ORIGINAL].tuple.dst.u3.ip;
> +   info.ctreplsrc =
> ct->tuplehash[IP_CT_DIR_REPLY].tuple.src.u3.ip;
> +   info.ctrepldst =
> ct->tuplehash[IP_CT_DIR_REPLY].tuple.dst.u3.ip;
> +   }
> +   else if (skb->protocol == __constant_htons(ETH_P_IPV6)) {
> +   /* Again, hash ipv6 addresses into a single u32. */
> +   info.ctorigsrc =
> jhash2(ct->tuplehash[IP_CT_DIR_ORIGINAL].tuple.src.u3.ip6, 4,
> q->perturbation);
> +   info.ctorigdst =
> jhash2(ct->tuplehash[IP_CT_DIR_ORIGINAL].tuple.dst.u3.ip6, 4,
> q->perturbation);
> +   info.ctreplsrc =
> jhash2(ct->tuplehash[IP_CT_DIR_REPLY].tuple.src.u3.ip6, 4,
> q->perturbation);
> +   info.ctrepldst =
> jhash2(ct->tuplehash[IP_CT_DIR_REPLY].tuple.dst.u3.ip6, 4,
> q->perturbation);
> +   }
> +
> +   }
> +#endif
> 
> I'd rip out the IPv6 conntrack stuff as I'm much more concerned by
> handling IPv4 NAT.  And I'm not sure how to get it into cake's host
> handling yet but
> 
> I can feel an experiment and hackery coming on later today :-)
> 
> Am overjoyed!
> ___
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake
___
Cake mailing list
Cake@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/cake

Re: [Cake] [lede-project/source] Add support for cake qdisc (#72)

[Dave Täht]

I still think squashing is very important, and essentially required by
several rfcs.


On 6/1/16 3:02 AM, Kevin Darbyshire-Bryant wrote:
> To try and keep everyone in the loop this has also been sent to the cake
> list.
> 
> LR;DR - There's a pull request to the LEDE tree to integrate cake as a
> patch to tc as well as a small, separate core package for
> kmod-sched-cake.  This is a good thing.  The pull request is
> https://github.com/lede-project/source/pull/72
> 
> Hannu Nyman is the author of the patch included in the PR - I've
> volunteered to be maintainer until cake really, really, really goes
> upstream.  Lucky me :-)
> 
> Hannu's patch included an improvement in column alignment on tc -s qdisc
> show output.  I have updated Dave's tc-adv tree to include this cosmetic
> change
> https://github.com/dtaht/tc-adv/commit/54794117daef5dd16e0ccec4b821f0c67e6b9ede
> 
> 
> Jonathan made reference to "a particular misfeature that should be
> deleted".  This mis-feature is DSCP washing.  This has been brought up
> before and to that end I put in some pull requests removing said
> facility to Dave's cake & tc-adv repos.  On that previous occasion it
> was decided it could stay but again it has been mentioned as being
> something undesirable.  I dug around, found my previous 'nosquashwash'
> trees, rebased them and have submitted pull requests again.
> 
> https://github.com/dtaht/tc-adv/pull/10
> https://github.com/dtaht/sch_cake/pull/25
> 
> Maybe it will save a little work.  Or if JM happy I will commit them as
> is.  I personally would like this stumbling block removed for better or
> for worse, let's not stumble over it again :-)
> 
> Kevin
> 
> 
> 
> 
> 
> On 31/05/16 19:46, Hannu Nyman wrote:
>> I have amended the commit (and it is rebased to the current LEDE head)
>>
>>   * import the cake commit recommended by @chromi
>> 
>>   * define Kevin as the maintainer for cake, thanks @kdarbyshirebryant
>> 
>>
>> @kdarbyshirebryant  , regarding
>> maintenance:
>>
>>   * The cake part is easily maintainable, as the Makefile here just
>> defines the commit to be fetched from the original cake repo and
>> sets date as the version.
>>   * tc patch may require some manual diff work if the later changes in
>> tc-adv are complex. I actually created my patch a few months ago by
>> making a diff of your tc-adv4.3 branch (that contained tc-adv
>> applied to iproute 4.3) to plain iproute. Then I pruned the
>> non-essential parts (e.g. extra/old cake versions still present in
>> tc-adv, changes to other qdiscs, man pages etc.) from the patch and
>> left only the cake stuff to be compiled. And that was not actually
>> very much. Basically just a new module for tc, small changes to a
>> header file and Makefile. I have since then updated the patch to
>> include the two commits from @chromi  in
>> April.
>>
>> —
>> You are receiving this because you were mentioned.
>> Reply to this email directly, view it on GitHub
>> ,
>> or mute the thread
>> .
>>
>>
> ___
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake
___
Cake mailing list
Cake@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/cake

Re: [Cake] [Codel] Proposing COBALT

[Dave Täht]

1) I am all in favor of continued experimentation and coding in these areas.

2) However I strongly advise the first thing you attempt to do when
futzing with an aqm, is to try it at various RTTs, and then do it at
high bandwidths and low.

Some of the discussion below makes me nervous, in that a point of codel
is to try and catch the next harmonic. There's no smooth ramp up or ramp
down, there' a wave coming sometime in the future that needs to be
smoothed to fill the pipe, not the queue.

My last attempts with cake the way it was had it performing miserably at
longer RTTs (try 50ms) vs codel or fq-codel - as in half the throughput
achieved by codel, at that RTT.

Please test at larger RTTs.

On 5/24/16 8:07 AM, Jonathan Morton wrote:
> 
>> On 24 May, 2016, at 16:47, Jeff Weeks  wrote:
>>
>>> In COBALT, I keep the drop-scheduler running in this phase, but without 
>>> actually dropping packets, and *decrementing* count instead of incrementing 
>>> it; the backoff phase then 
>>> naturally ends when count returns to zero, instead of after an arbitrary 
>>> hard timeout.  The loop simply ensures that count will reduce by the 
>>> correct amount, even if traffic 
>>> temporarily ceases on the queue.  Ideally, this should cause Codel’s count 
>>> value to stabilise where 50% of the time is spent above target sojourn 
>>> time, and 50% below.  (Actual 
>>> behaviour won’t quite be ideal, but it should be closer than before.)
>>
>> I tried this as well, at one point, but can't remember, off-hand, why I 
>> didn't stick with it; will have to see if I can find mention of it in my 
>> notes.
>> What trigger are you using to decrement count?  I initially did a crude 
>> decrement of count every interval, but then you end up with a ramp-down time 
>> which is considerably slower then the ramp-up (and the ramp up is slow to 
>> begin with).
>> I assume you're actually re-calculating the next drop, using the 
>> 1/sqrt(count) but instead of dropping and increasing count, you're simply 
>> decreasing count, so the time to get from 1->N is the same as the time to 
>> get to N->1?
> 
> That’s basically right.  In retrospect, it seems like a very obvious approach 
> to the backoff problem.  :-)
> 
> Of course, due to the “priming” delay and the possibility of the signalling 
> frequency exceeding the packet rate, it’s likely to take *less* time to ramp 
> down than to ramp up; this is why the ramping down is guarded by a while loop.
> 
>>> As another simplification, I eliminated the “primed” state (waiting for 
>>> interval to expire) as an explicit entity, by simply scheduling the first 
>>> drop event to be at now+interval when 
>>> entering the dropping state.  This also eliminates the first_above_time 
>>> variable.  Any packets with sojourn times below target will bump Codel out 
>>> of the dropping state anyway.
>>
>> How do you handle the case where you're scheduled a drop event 100ms in the 
>> future, and we immediately see low latency; is the event descheduled?
>> If not, what if we then see high latency again; can the 
>> still-scheduled-event cause us to start dropping packets earlier than 100ms?
> 
> The first drop event is scheduled by setting the “dropping” flag, ensuring 
> that “count” is nonzero, and setting the “drop_next” timestamp to 
> now+interval.  Any packet below the target sojourn time clears the “dropping” 
> flag, which prevents marking or dropping from occurring - which is why the 
> explicit “primed” state is eliminated.
> 
> Since the timestamp is set in this way whenever the “dropping” flag 
> transitions from cleared to set, there are no spurious drop events.
> 
> The code is in the sch_cake repo if you want to examine the details.  I 
> promise it’s a lot easier to read than the original Codel code.
> 
>  - Jonathan Morton
> 
> ___
> Codel mailing list
> co...@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/codel
> 
___
Cake mailing list
Cake@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/cake

[Cake] cake breaks on net-next head

[Dave Täht]

As I have accumulated enough hardware to do some basic cake testing
again, I spent a bit of time trying to get it to build, and did not
succeed.

There has been some activity in the world (discussed at the netconf
conference - I was not there so I lack details) about modifying how
ingress works, so I assume it has something to do with that (AT_INGRESS
and G_TC_AT are undefined).

patches gladly accepted.
___
Cake mailing list
Cake@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/cake

[Cake] quick review: arokh's feature-full build of openwrt

[Dave Täht]

https://forum.openwrt.org/viewtopic.php?id=50914 as of feb 13th has many
features (like having the luci-ssl gui native, sqm-scripts, and support
for nearly all the platforms I have handy (c7v2, linksys 1200ac, wndr
4800 and 4300))... so I replaced a c7v2 router as my default home
gateway to see how it did. It also has nifty features like multiple vpn
support, mwan (for multiple uplinks), and a tor-specific ssid.

Quick notes:

dns is tunneled through dnscrypt and then to opendns by default, and
feels "chunky".

The ath10k in the archer c7v2 refused to come up with a "world" setting,
I had to move it to country='US'.

The wifi interfaces are bridged. (which is ok at the moment, I wanted a
simple test)

IPv6 "just worked".

The default optional packages pointed to the wrong place - fixed with

cd /etc/opkg
sed -i
s#http://luci.subsignal.org/~trondah/archer_cx/r48717/packages/#http://luci.subsignal.org/~trondah/archer_cx/r48717/packages/#g
di
stfeeds.conf

netperf and Babel were not installed by default, but all the other tools
I rely on like tcpdump were.

It's kernel 4.1.16 (openwrt elsewhere is trying to stabilize on 4.4)

cake works, but is mildly busted and does not have the triple flow
isolation, and no extended stats are visible. The archer C7v2 has a
different default ethernet wan port (eth0 rather than eth1) which caused
me a bit of confusion while setting it up.

# tc -s qdisc show dev eth0
qdisc cake 800a: root refcnt 2 bandwidth 5500Kbit besteffort flows raw
 Sent 196703782 bytes 1022519 pkt (dropped 2131, overlimits 1605810
requeues 0)
 backlog 0b 0p requeues 0

The tc package installed is:

tc-adv - 4.1.0-git-7

While I was away comcast upgraded this connection to 75mbit/5.5mbit. We
basically run out of cpu at this speed with cake and the default
firewall rules.

All and all it's an extremely competent effort, and I am happy to not
have had to build it myself. I will probably "downgrade" to use
comcast's local dnssec enabled dns resolvers and disable dnscrypt, but
all in all it has everything I need on a day to day basis, and it seems
to be nearly a pure superset of everything I regarded as essential in
CeroWrt.

The C7v2 is showing it's age (and there's another issue, which I'll get
to in a sec), so I plan to try this distro on a linksys 1200ac later
this week.

There is another feature-full build of openwrt out there - hannu nyman's
build specific to the wndr3800 and friends (like the ubnt gear), which
is tracking cake developments more closely. I will give that a shot on
my nearly last remaining cerowrt box after confirming the cake
implementation is current.


___
Cake mailing list
Cake@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/cake

Re: [Cake] [aqm] codel with low shape rates

[Dave Täht]

On 1/18/16 2:11 PM, Jeff Weeks wrote:
> Hello all,
> 
> I'm wondering if there's some data on Codel with low shape rates?
> 
> In particular, I'm talking about in the kbps ranges.

We recently did some testing of several codel variants at very low rates
(2mbit/384kbit asymmetric). One flent dataset is here:

http://snapon.cs.kau.se/~d/384k/

There are a couple others. I didn't take the time to create graphs and
collate results before leaving for christmas vacation, the closest
I came to that w/graphs was this post:

https://lists.bufferbloat.net/pipermail/cake/2016-January/001808.html

pie was miserable, also. (it has a 10k estimator needed)

> 
> In my investigation, it seems as though the algorithm can't control latency 
> as effectively.
> 
> For one, the algorithm requires 2 packets in the queue to operate, but at a 
> low shape rate, it's highly likely that all packets in the queue will have 
> high latency, so 'count' will begin to ramp up... but conversely, it's also 
> likely that we drain the queue, and then leave the drop state (and wont 
> re-enter it for at least an interval's worth of time (100ms)).

This is not quite true - it's bytes, not packets. An MTU's worth of
bytes is the std codel limit before switching off.

Additionally, when htb is used, there is at least an htb quantum's worth
of packets that queue also. (the "cake" variant does not have this
problem. After I published the bcake vs cake results above, the cake
code got improved)

> Effectively, we get an oscillation of "in drop state, out of drop state", and 
> we're not in the drop state for long enough to ramp up count, because the 
> queue itself (proportional to the shape rate) isn't very big.

I don't see this, we generally end up with a persistently >1 packet
queue with two or more flows going. I see is a worse problem where the
basic attributes of keeping a connection up and things like slow start,
and with multiple flows, result in count climbing excessively high,
especially with ecn in use.

More research at sub 2mbit speeds is needed. Am pretty happy with things
in the 4mbit to 40gbit range.

> Are there way to combat this, or am I misinterpreting a portion of the 
> algorithm?

The simplest way to get decent results is to set target slightly more
than the MTU at the speed you are running at. e.g. 1Mbit = target 13ms.
This is essentially what the sqm-scripts and cake do - and the results
are still less than fully desirable.

(I would like very much to have an aqm that was knobless at these
speeds. Most of the work on trying to improve matters at these rates is
in the multitude of "cake" variants)

> 
> Thanks,
> Jeff
> 
> 
> 
> /dev/jeff_weeks.x2936
> Sandvine Incorporated
> 
> ___
> aqm mailing list
> a...@ietf.org
> https://www.ietf.org/mailman/listinfo/aqm
> 
___
Cake mailing list
Cake@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/cake