On Fri, 22 Nov 2019 12:46:18 +0100 Toke Høiland-Jørgensen <[email protected]> wrote:
> "Adam Moffett" <[email protected]> writes: > > >>> > >>> Are there any commercial products already using Cake? > >> > >>Evenroute, eero, ubnt top that list. Evenroute's implementation is > >>superb, the first one that used active line measurements to handle > >>"sag". Anything derived from openwrt (somewhere between 10-30% of the > >>home router market). I'm not sure if preseem is using it or not. > >>dd-wrt. Most other things doing "SQM" are doing it via htb + fq_codel. > >> > >> > > An idea which was floated was to experiment with routing ISP > > customer traffic through a Linux server using cake to improve > > customer experience. Basically like Preseem. My colleague has > > toyed with it a bit in small test cases and was impressed with the > > outcomes. > > > > He's looked closer than I have, but I'm trying to picture how his > > idea would scale. I believe I'm seeing a CLI tool for configuring > > policies. It seems like we'd have to create a middle layer to > > create/update policies for customer's IP address based on > > information obtained from our AAA and CRM systems. I can picture > > some shapes that might take, but I think it would ultimately have > > to revolve around scripting the tc command. There would be > > thousands of policies and a policy would be created/updated > > whenever a subscriber reconnects (e.g. when a DHCP lease renews or > > a RADIUS auth event happens or similar). > > I know several ISPs that do this (route traffic through a Linux box and > shape there). This deployment mode has not been the primary focus of > CAKE, though; the "standard" way to do it is with HTB+FQ-CoDel, which > allows you to set up arbitrarily complex configurations on a single > interface. Yes, I worked for an ISP back in 2005, that route traffic through a Linux box and does shaping. There were a number of scalabilities issues, that we fixed (upstream) and also Open Sources components for others to reuse. I did a public talk in 2008 about how we made it scale: http://people.netfilter.org/hawk/presentations/osd2008/osd2008_iptables_rules_JesperDangaardBrouer.pdf This was before Codel and even-before Bufferbloat was coined/defined. Our setup was HTB+SFQ, with a shaper per customer. This actually solved the bufferbloat problem in-practice for people, as SFQ gave each end-user 128 queues. Today I would not use iptables for this. Instead I would use BPF or nftables 'set' infrastructure. > This can also be made to scale, but there's a central qdisc > lock in Linux that you have to get around to scale to multiple cores. > Fortunately, Jesper has already solved this particular issue: > > https://github.com/netoptimizer/xdp-cpumap-tc The central qdisc TX lock is a major scalability issue. But I've solved in above link, via XDP and TC. This actually runs in production at an ISP. > > Should we even pursue this idea? > > In my own totally non-biased opinion: Yes! :) It would be great. > > Although most staff who would touch this will have studied programming > > in college, I would not qualify any of us as "programmers" per se. My > > biggest concern would be hitting a service affecting problem that we > > can't solve. > > One way to go about this would be to open source the entire solution. > There are already bits and pieces available as open source (such as > Jesper's repository above, and sqm-scripts[0]) that you can build on, > and this way you could enlist community help to solve any issues with > the Linux side. You'd still need to get the data from your internal > systems, of course, but you could define a simple configuration format > that was part of the open source code; then you'll just need to write a > script that grabs customer info from your CRM and outputs the config > format... Yes, this is the advantage of Open Source! :-) If you create this as Open Source, then feel free to reach out to me directly. And I know of several other people (working at ISPs) that would likely be interested in collaborating. As Toke wrote, you can still keep your CRM system proprietary and closed-source, we don't care anyhow ;-) > > Second concern is that many of our equipment vendors already use > > Linux. Even Cisco now in some products. Maybe we'll waste our time > > trying to roll our own solution and then find that a software update > > from a vendor next year gives us everything we needed anyway. I would not hold my breath... and if it does come as a software upgrade, you can expect to pay plenty for it... Maybe I'm the wrong guy to ask, but I really think it is straight forward to implement an ISP Linux router with per customer handling. All the components seems avail as Open Source. (There do seem to be a lack around a DHCP server that can handle this well). > This would be great, of course, and do go and bug your vendors to > solve this problem! Note, however, that just because a system is > running Linux on the control plane, it may be using a > hardware-offloaded data plane that does not have any of the > bufferbloat mitigation features (unless the vendor specifically > implemented them). I'm hoping that *eventually* these things will be > ubiquitous across the industry, but thus far this has seemed to be an > "any decade now" kind of proposition :/ I would prefer, not to waste time on creating bugs for your vendor, but instead actually have the ability to fix the issue myself, or hire some Linux consultant that can... -- Best regards, Jesper Dangaard Brouer MSc.CS, Principal Kernel Engineer at Red Hat LinkedIn: http://www.linkedin.com/in/brouer _______________________________________________ Cake mailing list [email protected] https://lists.bufferbloat.net/listinfo/cake
