So more specifically, for 500mbit I can use a calculated burst/cburst of 62500 (1000 * 500000 / 8000), here’s the change:
default: 320mbit up / 268mbit down, 3ms latency, 8.8ms tcp rtt burst/cburst 62500: 200mbit up / 480mbit down, 40ms latency, 40ms tcp rtt Aggregate throughput goes from 588mbit to 680mbit, but latency skyrockets. A burst of only 2xMTU doesn’t change much, and 4xMTU already jumps to 30ms latency and only 630mbit aggregate bandwidth. So for this one-armed router setup on this hardware, I don’t see a worthwhile tradeoff of latency for aggregate throughput. Perhaps in other situations, it could be useful. :) > On Dec 31, 2018, at 1:10 AM, Sebastian Moeller <[email protected]> wrote: > > Well the idea would be to scale the buffer to cover, say Xms at the > configured bandwidth, so HTB could deal with CPU stalls up to X-Yms (with Y > << X)... We just switched sqm-scripts to automatically scale the buffering to > 1ms.... > Would be interested to learn whether that would increase HTB's utilisation? > > > On December 30, 2018 11:36:27 PM GMT+01:00, Pete Heist <[email protected]> > wrote: > The experiments I did with those didn’t yield great results, with changing a > value by one MTU sometimes causing sudden throughput or inter-flow latency > increases, with the tradeoffs not being clear. I’m afraid admins could easily > cause problems fiddling with these. Fortunately most customer facing routers > have aggregate bitrates that an APU1 can handle even with default htb, or > cake. I also appreciate that such settings don’t exist in cake… :) > > On Dec 30, 2018, at 10:51 PM, Sebastian Moeller <[email protected]> wrote: > > Hi Pete, > > you might want to have a look at htb's burst and cburst parameters, as these > should allow to trade in latency under load for bandwidth utilization. > > > Best Regards > Sebastian > > On Dec 30, 2018, at 21:42, Pete Heist <[email protected]> wrote: > > It’s a bit more complicated than this. It looks like the htb rate limiter is > different in that as rates increase the actual rate starts to deviate from > the specified rate early on, but it rather gracefully handles the “out of > CPU” situation, where it still maintains control of the queue, just gradually > fails to meet the rate specified by greater and greater percentages. > > Instead of a single flow test with iperf3, here are rates that each limiter > can reach on egress of both apu1a interfaces during an rrul_be test: > > # - max limit on APU for one-armed routing, rrul_be test 4+4 flows (firewall > on): > # - cake: 210mbit > # - htb+fq_codel: 93%@100mbit, 90%@200mbit, 84%@300mbit, 72%@400mbit, > 59%@500mbit > # - hfsc+fq_codel: 310mbit > # - hfsc+cake: 300mbit > > The numbers for cake and hfsc are right before loss of queue, and with htb > the queue isn’t lost even at 500mbit, for example, just the actual rate is > only 59% of what was specified. > > I really need to graph the specified rate vs the actual rate, inter-flow and > intra-flow latency, stepped 25mbit at a time. I think it would be > interesting, so this is on my todo list if there’s time after the ISP config > gets done. > > On Dec 28, 2018, at 1:17 AM, Pete Heist <[email protected]> wrote: > > For whatever reason, I’m seeing the rate limiters in cake and hfsc vastly > outperform htb in the one-armed router configuration I described in my > previous thread. To simplify things, I apply the qdiscs with a single class > only at egress of eth0 on apu1a: > > apu2a <— default VLAN —> apu1a <— VLAN 3300 —> apu2b > > I use iperf3 from apu2a to apu2b and find the rate at which things break > down. Whereas cake and hfsc can both reach around 850mbit, htb is breaking > down at around 200mbit, which seems rather strange. This could be a function > of the older kernel I have to use, the hardware, or maybe htb just isn’t > suited well to this task for some reason. I wish I knew, as I’d rather be > using htb for this task than hfsc (especially given the lockup issue with > cake)... > > —— > > #!/bin/bash > > # point where iperf3 throughput drops below ~93% of theoretical: > # htb: 200mbit > # hfsc: 850mbit > # cake: 850mbit > > IFACE=eth0 > RATE=850mbit > > start_htb() { > stop > tc qdisc add dev $IFACE root handle 1: htb default 1 > tc class add dev $IFACE parent 1: classid 1:1 htb rate $RATE ceil $RATE > tc qdisc add dev $IFACE parent 1:1 handle 10: fq_codel > } > > start_hfsc() { > stop > tc qdisc add dev $IFACE root handle 1: hfsc default 1 > tc class add dev $IFACE parent 1: classid 1:1 hfsc sc rate $RATE ul rate > $RATE > tc qdisc add dev $IFACE parent 1:1 handle 10: fq_codel > } > > start_cake() { > stop > tc qdisc add dev $IFACE root cake bandwidth $RATE > } > > stop() { > tc qdisc del dev $IFACE root &>/dev/null > tc qdisc del dev $IFACE ingress &>/dev/null > } > > "$@“ > —— > > root@apu1a:~/rate_limiters# uname -a > Linux apu1a 3.16.7-ckt9-voyage #1 SMP Thu Apr 23 11:10:44 HKT 2015 i686 > GNU/Linux > > root@apu1a:~/rate_limiters# cat /proc/cpuinfo > processor : 0 > vendor_id : AuthenticAMD > cpu family : 20 > model : 2 > model name : AMD G-T40E Processor > stepping : 0 > microcode : 0x5000101 > cpu MHz : 800.000 > cache size : 512 KB > physical id : 0 > siblings : 2 > core id : 0 > cpu cores : 2 > apicid : 0 > initial apicid : 0 > fdiv_bug : no > f00f_bug : no > coma_bug : no > fpu : yes > fpu_exception : yes > cpuid level : 6 > wp : yes > flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov > pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb > rdtscp lm constant_tsc nonstop_tsc extd_apicid aperfmperf pni monitor ssse3 > cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse > 3dnowprefetch ibs skinit wdt arat hw_pstate npt lbrv svm_lock nrip_save > pausefilter vmmcall > bogomips : 1999.83 > clflush size : 64 > cache_alignment : 64 > address sizes : 36 bits physical, 48 bits virtual > power management: ts ttp tm stc 100mhzsteps hwpstate > > processor : 1 > vendor_id : AuthenticAMD > cpu family : 20 > model : 2 > model name : AMD G-T40E Processor > stepping : 0 > microcode : 0x5000101 > cpu MHz : 800.000 > cache size : 512 KB > physical id : 0 > siblings : 2 > core id : 1 > cpu cores : 2 > apicid : 1 > initial apicid : 1 > fdiv_bug : no > f00f_bug : no > coma_bug : no > fpu : yes > fpu_exception : yes > cpuid level : 6 > wp : yes > flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov > pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb > rdtscp lm constant_tsc nonstop_tsc extd_apicid aperfmperf pni monitor ssse3 > cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse > 3dnowprefetch ibs skinit wdt arat hw_pstate npt lbrv svm_lock nrip_save > pausefilter vmmcall > bogomips : 1999.83 > clflush size : 64 > cache_alignment : 64 > address sizes : 36 bits physical, 48 bits virtual > power management: ts ttp tm stc 100mhzsteps hwpstate > > root@apu1a:~/rate_limiters# ethtool -i eth0 > driver: r8169 > version: 2.3LK-NAPI > firmware-version: rtl_nic/rtl8168e-2.fw > bus-info: 0000:01:00.0 > supports-statistics: yes > supports-test: no > supports-eeprom-access: no > supports-register-dump: yes > supports-priv-flags: no > > Cake mailing list > [email protected] > https://lists.bufferbloat.net/listinfo/cake > <https://lists.bufferbloat.net/listinfo/cake> > > > > -- > Sent from my Android device with K-9 Mail. Please excuse my brevity.
_______________________________________________ Cake mailing list [email protected] https://lists.bufferbloat.net/listinfo/cake
