Re: [Bloat] Still seeing bloat with a DOCSIS 3.1 modem
Birdies have told me that it is possible for DOCSIS 3.1 modems to be running in 3.0 mode. Bitch at your ISP. - Jim On Wed, Mar 25, 2020 at 1:01 AM Aaron Wood wrote: > I recently upgraded service from 150up, 10dn Mbps to xfinity's gigabit > (with 35Mbps up) tier, and picked up a DOCSIS 3.1 modem to go with it. > > Flent test results are here: > > https://burntchrome.blogspot.com/2020/03/bufferbloat-with-comcast-gigabit-with.html > > tl/dr; 1000ms of upstream bufferbloat > > But it's DOCSIS 3.1, so why isn't PIE working? Theory: It's in DOCSIS > 3.0 upstream mode based on the status LEDs. Hopefully it will go away if I > can convince it to run in DOCSIS 3.1 mode. > > At the moment, however, my WRT1900AC isn't up to the task of dealing with > these sorts of downstream rates. > > So I'm looking at the apu2, which from this post: > > https://forum.openwrt.org/t/comparative-throughput-testing-including-nat-sqm-wireguard-and-openvpn/44724 > > Will certainly get most of the way there. > > Although > ___ > Bloat mailing list > Bloat@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/bloat > ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] Still seeing bloat with a DOCSIS 3.1 modem
On Wed, Mar 25, 2020 at 8:58 AM Aaron Wood wrote: > > One other thought I've had with this, is that the apu2 is multi-core, and the > i210 is multi-queue. > > Cake/htb aren't, iirc, setup to run on multiple cores (as the rate limiters > then don't talk to each other). But with the correct tuple hashing in the > i210, I _should_ be able to split things and do two cores at 500Mbps each > (with lots of compute left over). > > Obviously, that puts a limit on single-connection rates, but as the number of > connections climb, they should more or less even out (I remember Dave Taht > showing the oddities that happen with say 4 streams and 2 cores, where it's > common to end up with 3 streams on the same core). But assuming that the > hashing function results in even sharing of streams, it should be fairly > balanced (after plotting some binomial distributions with higher "n" values). > Still not perfect, especially since streams aren't likely to all be > elephants. We live with imperfect per core tcp flow behavior already. What I wanted to happen was the "list" ingress improvement to become more generally available ( I can't find the lwn link at the moment). It has. I thought that then we could express a syntax of tc qdisc add dev eth0 ingress cake-mq bandwidth whatever, and it would rock. I figured getting rid of the cost of the existing ifb and tc mirred, and having a fast path preserving each hardware queue, then using rcu to do a sloppy allocate atomic lock for shaped bandwidth and merge every ms or so might be then low-cost enough. Certainly folding everything into a single queue has a cost! I was (before money ran out) prototyping adding a shared shaper to mq at one point (no rcu, just There have been so many other things toss around (bpf?) As for load balancing better, google "RSS++", if you must. > On Wed, Mar 25, 2020 at 4:03 AM Toke Høiland-Jørgensen wrote: >> >> Sebastian Moeller writes: >> >> > Hi Toke, >> > >> > >> >> On Mar 25, 2020, at 09:58, Toke Høiland-Jørgensen wrote: >> >> >> >> Aaron Wood writes: >> >> >> >>> I recently upgraded service from 150up, 10dn Mbps to xfinity's gigabit >> >>> (with 35Mbps up) tier, and picked up a DOCSIS 3.1 modem to go with it. >> >>> >> >>> Flent test results are here: >> >>> https://burntchrome.blogspot.com/2020/03/bufferbloat-with-comcast-gigabit-with.html >> >>> >> >>> tl/dr; 1000ms of upstream bufferbloat >> >>> >> >>> But it's DOCSIS 3.1, so why isn't PIE working? Theory: It's in DOCSIS >> >>> 3.0 >> >>> upstream mode based on the status LEDs. Hopefully it will go away if I >> >>> can >> >>> convince it to run in DOCSIS 3.1 mode. >> >> >> >> I think that while PIE is "mandatory to implement" in DOCSIS 3.1, the >> >> ISP still has to turn it on? So maybe yelling at them will work? (ha!) >> >> >> >>> At the moment, however, my WRT1900AC isn't up to the task of dealing with >> >>> these sorts of downstream rates. >> >>> >> >>> So I'm looking at the apu2, which from this post: >> >>> https://forum.openwrt.org/t/comparative-throughput-testing-including-nat-sqm-wireguard-and-openvpn/44724 >> >>> >> >>> Will certainly get most of the way there. >> >> >> >> My Turris Omnia is doing fine on my 1Gbps connection (although that >> >> hardly suffers from bloat, so I'm not doing any shaping; did try it >> >> though, and it has no problem with running CAKE at 1Gbps). >> > >> > Well, doing local network flent RRUL stress tests indicated that >> > my omnia (at that time with TOS4/Openwrt18) only allowed up to >> > 500/500 Mbps shaping with bi directionally saturating traffic >> > with full MTU-sized packets. So I undirectional CAKE at 1Gbps >> > can work, but under full load, I did not manage that, what did I >> > wrong? >> >> Hmm, not sure I've actually done full bidirectional shaping. And trying >> it now, it does seem to be struggling... >> >> -Toke > > ___ > Bloat mailing list > Bloat@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/bloat -- Make Music, Not War Dave Täht CTO, TekLibre, LLC http://www.teklibre.com Tel: 1-831-435-0729 ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] Still seeing bloat with a DOCSIS 3.1 modem
> > >>> But it's DOCSIS 3.1, so why isn't PIE working? Theory: It's in > DOCSIS 3.0 > >>> upstream mode based on the status LEDs. Hopefully it will go away if > I can > >>> convince it to run in DOCSIS 3.1 mode. > >> > >> I think that while PIE is "mandatory to implement" in DOCSIS 3.1, the > >> ISP still has to turn it on? So maybe yelling at them will work? (ha!) > I've chatted with someone about it, and they seemed to think it's suspicious, but I'm not going to push it further until I have modem showing that it's in DOCSIS 3.1 for upstream. I do need to see if I can sort out what the SB8200's status messages are: CM-STATUS message sent. Event Type Code: 24; Chan ID: 48; DSID: N/A; MAC Addr: N/A; OFDM/OFDMA Profile ID: 2 3.;CM-MAC=xx:xx:xx:xx:xx:xx;CMTS-MAC=xx:xx:xx:xx:xx:xx;CM-QOS=1.1;CM-VER=3.1; > >>> At the moment, however, my WRT1900AC isn't up to the task of dealing > with > >>> these sorts of downstream rates. > >>> > >>> So I'm looking at the apu2, which from this post: > >>> > https://forum.openwrt.org/t/comparative-throughput-testing-including-nat-sqm-wireguard-and-openvpn/44724 > >>> > >>> Will certainly get most of the way there. > >> > >> My Turris Omnia is doing fine on my 1Gbps connection (although that > >> hardly suffers from bloat, so I'm not doing any shaping; did try it > >> though, and it has no problem with running CAKE at 1Gbps). > > > > Well, doing local network flent RRUL stress tests indicated that > > my omnia (at that time with TOS4/Openwrt18) only allowed up to > > 500/500 Mbps shaping with bi directionally saturating traffic > > with full MTU-sized packets. So I undirectional CAKE at 1Gbps > > can work, but under full load, I did not manage that, what did I > > wrong? > > Hmm, not sure I've actually done full bidirectional shaping. And trying > it now, it does seem to be struggling... That's definitely an option for me, as I don't have to worry about a 2Gbps total traffic, only about 1.03Gbps (since cable is so asymmetric). But I'm also not sure I want to go with another ARM box. The small x64 boxes are looking like a much better long-term option. ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] Still seeing bloat with a DOCSIS 3.1 modem
On Tue, Mar 24, 2020 at 11:19 PM Sebastian Moeller wrote: > So, for higher bandwidth plans people started using raspberry pi4bs with > an additional usb3 Ethernet dongle. Its WiFi is not really up for the task > but it does seem to make a mean wired only router, the quad A76 cores seem > to be capable to reliably shape up to 1 gigabit with cpu cycles to spare. > So maybe get one of those and change your old wifi router into a wifi AP? > Using the existing router as an AP is my plan, so this would definitely work (I'm looking to use this as an excuse to split the router and the APs up). > On March 25, 2020 6:29:17 AM GMT+01:00, Matt Taggart > wrote: >> >> On 3/24/20 10:01 PM, Aaron Wood wrote: >> >> I recently got CenturyLink gig fiber and bought one of these: >> >> Qotom Q355G4 >> https://www.amazon.com/gp/product/B077ZWR8Q9 >> >> And it's running OpenWRT 19.07 just fine, boots from a small USB thumbdrive. >> It has no problem with CAKE + piece_of_cake up to 1gbit >> >> Here is a table I made comparing the Qotom models >> >> https://we.riseup.net/lackof/x86-router-candidates#qotom >> >> Thanks! I'll definitely look into those as well. ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] Still seeing bloat with a DOCSIS 3.1 modem
One other thought I've had with this, is that the apu2 is multi-core, and the i210 is multi-queue. Cake/htb aren't, iirc, setup to run on multiple cores (as the rate limiters then don't talk to each other). But with the correct tuple hashing in the i210, I _should_ be able to split things and do two cores at 500Mbps each (with lots of compute left over). Obviously, that puts a limit on single-connection rates, but as the number of connections climb, they should more or less even out (I remember Dave Taht showing the oddities that happen with say 4 streams and 2 cores, where it's common to end up with 3 streams on the same core). But assuming that the hashing function results in even sharing of streams, it should be fairly balanced (after plotting some binomial distributions with higher "n" values). Still not perfect, especially since streams aren't likely to all be elephants. On Wed, Mar 25, 2020 at 4:03 AM Toke Høiland-Jørgensen wrote: > Sebastian Moeller writes: > > > Hi Toke, > > > > > >> On Mar 25, 2020, at 09:58, Toke Høiland-Jørgensen wrote: > >> > >> Aaron Wood writes: > >> > >>> I recently upgraded service from 150up, 10dn Mbps to xfinity's gigabit > >>> (with 35Mbps up) tier, and picked up a DOCSIS 3.1 modem to go with it. > >>> > >>> Flent test results are here: > >>> > https://burntchrome.blogspot.com/2020/03/bufferbloat-with-comcast-gigabit-with.html > >>> > >>> tl/dr; 1000ms of upstream bufferbloat > >>> > >>> But it's DOCSIS 3.1, so why isn't PIE working? Theory: It's in > DOCSIS 3.0 > >>> upstream mode based on the status LEDs. Hopefully it will go away if > I can > >>> convince it to run in DOCSIS 3.1 mode. > >> > >> I think that while PIE is "mandatory to implement" in DOCSIS 3.1, the > >> ISP still has to turn it on? So maybe yelling at them will work? (ha!) > >> > >>> At the moment, however, my WRT1900AC isn't up to the task of dealing > with > >>> these sorts of downstream rates. > >>> > >>> So I'm looking at the apu2, which from this post: > >>> > https://forum.openwrt.org/t/comparative-throughput-testing-including-nat-sqm-wireguard-and-openvpn/44724 > >>> > >>> Will certainly get most of the way there. > >> > >> My Turris Omnia is doing fine on my 1Gbps connection (although that > >> hardly suffers from bloat, so I'm not doing any shaping; did try it > >> though, and it has no problem with running CAKE at 1Gbps). > > > > Well, doing local network flent RRUL stress tests indicated that > > my omnia (at that time with TOS4/Openwrt18) only allowed up to > > 500/500 Mbps shaping with bi directionally saturating traffic > > with full MTU-sized packets. So I undirectional CAKE at 1Gbps > > can work, but under full load, I did not manage that, what did I > > wrong? > > Hmm, not sure I've actually done full bidirectional shaping. And trying > it now, it does seem to be struggling... > > -Toke > ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] Still seeing bloat with a DOCSIS 3.1 modem
So, for higher bandwidth plans people started using raspberry pi4bs with an additional usb3 Ethernet dongle. Its WiFi is not really up for the task but it does seem to make a mean wired only router, the quad A76 cores seem to be capable to reliably shape up to 1 gigabit with cpu cycles to spare. So maybe get one of those and change your old wifi router into a wifi AP? Disclaimer, I have not tried that myself, as I am on a 100/40 plan and already have a router well capable of that speed. Best Regards Sebastian On March 25, 2020 6:29:17 AM GMT+01:00, Matt Taggart wrote: >On 3/24/20 10:01 PM, Aaron Wood wrote: >[snip] >> At the moment, however, my WRT1900AC isn't up to the task of dealing >> with these sorts of downstream rates. >> >> So I'm looking at the apu2, which from this post: >> >https://forum.openwrt.org/t/comparative-throughput-testing-including-nat-sqm-wireguard-and-openvpn/44724 > >I recently got CenturyLink gig fiber and bought one of these: > >Qotom Q355G4 >https://www.amazon.com/gp/product/B077ZWR8Q9 > >And it's running OpenWRT 19.07 just fine, boots from a small USB >thumbdrive. >It has no problem with CAKE + piece_of_cake up to 1gbit > >Here is a table I made comparing the Qotom models > >https://we.riseup.net/lackof/x86-router-candidates#qotom > >(compiled last December, so prices may have changed) > >-- >Matt Taggart >m...@lackof.org >___ >Bloat mailing list >Bloat@lists.bufferbloat.net >https://lists.bufferbloat.net/listinfo/bloat -- Sent from my Android device with K-9 Mail. Please excuse my brevity.___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] Still seeing bloat with a DOCSIS 3.1 modem
Aaron Wood writes: > I recently upgraded service from 150up, 10dn Mbps to xfinity's gigabit > (with 35Mbps up) tier, and picked up a DOCSIS 3.1 modem to go with it. > > Flent test results are here: > https://burntchrome.blogspot.com/2020/03/bufferbloat-with-comcast-gigabit-with.html > > tl/dr; 1000ms of upstream bufferbloat > > But it's DOCSIS 3.1, so why isn't PIE working? Theory: It's in DOCSIS 3.0 > upstream mode based on the status LEDs. Hopefully it will go away if I can > convince it to run in DOCSIS 3.1 mode. I think that while PIE is "mandatory to implement" in DOCSIS 3.1, the ISP still has to turn it on? So maybe yelling at them will work? (ha!) > At the moment, however, my WRT1900AC isn't up to the task of dealing with > these sorts of downstream rates. > > So I'm looking at the apu2, which from this post: > https://forum.openwrt.org/t/comparative-throughput-testing-including-nat-sqm-wireguard-and-openvpn/44724 > > Will certainly get most of the way there. My Turris Omnia is doing fine on my 1Gbps connection (although that hardly suffers from bloat, so I'm not doing any shaping; did try it though, and it has no problem with running CAKE at 1Gbps). -Toke ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] Still seeing bloat with a DOCSIS 3.1 modem
Hi Toke, > On Mar 25, 2020, at 09:58, Toke Høiland-Jørgensen wrote: > > Aaron Wood writes: > >> I recently upgraded service from 150up, 10dn Mbps to xfinity's gigabit >> (with 35Mbps up) tier, and picked up a DOCSIS 3.1 modem to go with it. >> >> Flent test results are here: >> https://burntchrome.blogspot.com/2020/03/bufferbloat-with-comcast-gigabit-with.html >> >> tl/dr; 1000ms of upstream bufferbloat >> >> But it's DOCSIS 3.1, so why isn't PIE working? Theory: It's in DOCSIS 3.0 >> upstream mode based on the status LEDs. Hopefully it will go away if I can >> convince it to run in DOCSIS 3.1 mode. > > I think that while PIE is "mandatory to implement" in DOCSIS 3.1, the > ISP still has to turn it on? So maybe yelling at them will work? (ha!) > >> At the moment, however, my WRT1900AC isn't up to the task of dealing with >> these sorts of downstream rates. >> >> So I'm looking at the apu2, which from this post: >> https://forum.openwrt.org/t/comparative-throughput-testing-including-nat-sqm-wireguard-and-openvpn/44724 >> >> Will certainly get most of the way there. > > My Turris Omnia is doing fine on my 1Gbps connection (although that > hardly suffers from bloat, so I'm not doing any shaping; did try it > though, and it has no problem with running CAKE at 1Gbps). Well, doing local network flent RRUL stress tests indicated that my omnia (at that time with TOS4/Openwrt18) only allowed up to 500/500 Mbps shaping with bi directionally saturating traffic with full MTU-sized packets. So I undirectional CAKE at 1Gbps can work, but under full load, I did not manage that, what did I wrong? Best Regards Sebastian > > -Toke > ___ > Bloat mailing list > Bloat@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/bloat ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] [Cerowrt-devel] OT: Netflix vs 6in4 from HE.net
"David P. Reed" writes: > Thanks, Colin, for the info. Sadly, I learned all about the licensing > of content in the industry back about 20 years ago when I was active > in the battles about Xcasting rights internationally (extending > "broadcast rights" to the Web, which are rights that exist only in the > EU, having to do with protecting broadcasters whose signals are > powerful enough to cross borders of countries, so a whole new, > non-copyright-based Intellectual Property Right was invented. WIPO > wanted to argue that the Web was just like broadcasting across > borders, so web pages should be burdened by Xcasting rights, along > with all other copyrighted things.) > > What I wanted to know was exactly what you just said in passing: that > he.net's address space was entirely blocked by Netflix because it > wasn't accurately geolocated for "region restriction" enforcement. > > Whether I think that is "correct" or "reasonable", I just want to be > able to get Netflix in my US house. Not to be any sort of "pirate" > intentionally trying to break the license. I really just want that > stuff to work as the license between Netflix and content provider > requires (I'm sure the license doesn't say "block he.net"). This can also be achieved by filtering the DNS responses for Netflix. Here's a guide for doing this with Bind and dnsmasq: https://community.ui.com/questions/Blocking-IPv6-traffic-to-Netflix-over-HE-net-tunnel/816b5753-6a86-4781-935e-06f5e972428f#answer/39318121-4ef3-4425-8e20-0c5d39f03937 And here's someone who got annoyed enough to write a Python daemon to do the same thing: https://github.com/cdhowie/netflix-no-ipv6-dns-proxy -Toke ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] Still seeing bloat with a DOCSIS 3.1 modem
Sebastian Moeller writes: > Hi Toke, > > >> On Mar 25, 2020, at 09:58, Toke Høiland-Jørgensen wrote: >> >> Aaron Wood writes: >> >>> I recently upgraded service from 150up, 10dn Mbps to xfinity's gigabit >>> (with 35Mbps up) tier, and picked up a DOCSIS 3.1 modem to go with it. >>> >>> Flent test results are here: >>> https://burntchrome.blogspot.com/2020/03/bufferbloat-with-comcast-gigabit-with.html >>> >>> tl/dr; 1000ms of upstream bufferbloat >>> >>> But it's DOCSIS 3.1, so why isn't PIE working? Theory: It's in DOCSIS 3.0 >>> upstream mode based on the status LEDs. Hopefully it will go away if I can >>> convince it to run in DOCSIS 3.1 mode. >> >> I think that while PIE is "mandatory to implement" in DOCSIS 3.1, the >> ISP still has to turn it on? So maybe yelling at them will work? (ha!) >> >>> At the moment, however, my WRT1900AC isn't up to the task of dealing with >>> these sorts of downstream rates. >>> >>> So I'm looking at the apu2, which from this post: >>> https://forum.openwrt.org/t/comparative-throughput-testing-including-nat-sqm-wireguard-and-openvpn/44724 >>> >>> Will certainly get most of the way there. >> >> My Turris Omnia is doing fine on my 1Gbps connection (although that >> hardly suffers from bloat, so I'm not doing any shaping; did try it >> though, and it has no problem with running CAKE at 1Gbps). > > Well, doing local network flent RRUL stress tests indicated that > my omnia (at that time with TOS4/Openwrt18) only allowed up to > 500/500 Mbps shaping with bi directionally saturating traffic > with full MTU-sized packets. So I undirectional CAKE at 1Gbps > can work, but under full load, I did not manage that, what did I > wrong? Hmm, not sure I've actually done full bidirectional shaping. And trying it now, it does seem to be struggling... -Toke ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
