Hi Rich, Quote: > > I use one root HTB qdisc and one root (1:) HTB class.
Sounds like a classical case of lock congestion on the TC-root qdisc lock. I have a solution using XDP here[1] combined with TC. Google have also hit the problem, they solved it differently, specific to their use-case. [1] https://github.com/xdp-project/xdp-cpumap-tc On Mon, 7 Jun 2021 13:28:10 -0400 Rich Brown <[email protected]> wrote: > Saw this on the lartc mailing list... For my own information, does > anyone have thoughts, esp. for this quote: > > "... when the speed comes to about 4.5Gbps download (upload is about > 500mbps), chaos kicks in. CPU load goes sky high (all 24x2.4Ghz > physical cores above 90% - 48x2.4Ghz if count that virtualization is > on)..." > > Thanks. > > Rich > > > > Begin forwarded message: > > > > From: "Ethy H. Brito" <[email protected]> > > Subject: Traffic shaping at 10~300mbps at a 10Gbps link > > Date: June 7, 2021 at 12:38:53 PM EDT > > To: lartc <[email protected]> > > > > > > Hi > > > > I am having a hard time trying to shape 3000 users at ceil speeds > > from 10 to 300mbps in a 7/7Gbps link using HTB+SFQ+TC(filter by IP > > hashkey mask) for a few days now tweaking HTB and SFQ parameters > > with no luck so far. > > > > Everything seems right, up 4Gbps overall download speed with > > shaping on. I have no significant packets delay, no dropped packets > > and no high CPU average loads (not more than 20% - htop info) > > > > But when the speed comes to about 4.5Gbps download (upload is about > > 500mbps), chaos kicks in. CPU load goes sky high (all 24x2.4Ghz > > physical cores above 90% - 48x2.4Ghz if count that virtualization > > is on) and as a consequence packets are dropped (as reported by tc > > -s class sh ...), RTT goes above 200ms and a lots of ungry users. > > This goes from about 7PM to 11 PM every day. > > > > If I turn shaping off, everything return to normality immediately > > and peaks of not more than 5Gbps (1 second average) are observed > > and a CPU load of about 5%. So I infer the uplink is not crowded. > > > > > > I use one root HTB qdisc and one root (1:) HTB class. > > > > Then about 20~30 same level (1:xx) inner classes to (sort of) separate the > > users by region > > And under these inner classes, goes the almost 3000 leaves (1:xxxx). > > > > I have one class with about 900 users and this quantity decreases > > by the other inner classes having some of them with just one user. > > > > Is the way I'm using HTB+SFQ+TC suitable for this job? > > > > Since the script that creates the shaping environment is too long I do not > > post it here. > > > > What can I inform you guys to help me solve this? > > Fragments of code, stats, some measurements? What? > > > > Thanks. > > > > Regards > > > > Ethy > -- Best regards, Jesper Dangaard Brouer MSc.CS, Principal Kernel Engineer at Red Hat LinkedIn: http://www.linkedin.com/in/brouer _______________________________________________ Bloat mailing list [email protected] https://lists.bufferbloat.net/listinfo/bloat
