HI Mark,
> On Dec 17, 2017, at 17:00, Mark Captur <[email protected]> wrote: > > Thanks for the comprehensive and quick reply :) below are some further answers Always happy to help. > > On 17 December 2017 at 16:23, Sebastian Moeller <[email protected]> wrote: > Hi Mark, > > > On Dec 17, 2017, at 11:45, Mark Captur <[email protected]> wrote: > > > > My setup is as follows > > > > vdsl2 modem doing pppoe itself and nat to 10.x.x.x -> lede master eth0.2 > > (wan static ip in modem's DMZ) eth0.1 (lan) doing nat to 192.168.1.x > > > > Here is my current SQM config > > config queue 'eth1' > > option debug_logging '0' > > option verbosity '5' > > option qdisc 'cake' > > option qdisc_advanced '1' > > option ingress_ecn 'ECN' > > option egress_ecn 'NOECN' > > option qdisc_really_really_advanced '1' > > option script 'layer_cake.qos' > > option interface 'eth0.2' > > option enabled '1' > > option eqdisc_opts 'nat rtt 50000 bridged-ptm dual-srchost > > diffserv4' > > option upload '2400' > > option linklayer 'ethernet' > > option overhead '8' > > option squash_dscp '1' > > option squash_ingress '1' > > option iqdisc_opts 'nat rtt 50000 bridged-ptm dual-dsthost' > > option download '0' > > > > config queue > > option debug_logging '0' > > option verbosity '5' > > option download '0' > > option qdisc 'cake' > > option script 'layer_cake.qos' > > option qdisc_advanced '1' > > option squash_dscp '0' > > option squash_ingress '0' > > option ingress_ecn 'ECN' > > option qdisc_really_really_advanced '1' > > option egress_ecn 'ECN' > > option interface 'eth0.1' > > option enabled '1' > > option eqdisc_opts 'nat rtt 50000 bridged-ptm dual-dsthost > > diffserv4' > > option upload '30000' > > option linklayer 'ethernet' > > option overhead '8' > > > > Is the overhead correct? should i use the bridged-ptm keyword (or should i > > use pppoe-ptm). > > The overhead certainly seems confusing. Personally, I dislike the > overhead related compound keywords like *-ptm and would recommend the > following: > 1) remove the bridged-ptm from the eqdisc/iqdisc fields > Done > 2) add "mpu 64" to the eqdisc/iqdisc fields > Done > 3) set overhad to 8+18+4 = 30 bytes (you will need to account for everything > added on the bottleneck, so your packets will be MTU 1492, to leave room for > the PPPoE header that the modem adds). > Dne > 4) DO not set the ptm keyword at all, instead make sure to set the shaper > bandwidth to <= sync bandwidth * 64/65 = sync bandwidth * 0.984615384615 (to > account for ptm's 64/65 encoding _without_ incurring needless operations per > packet). > Done 0.98 (2440 is .98 of 2480) for upstream but had to reduce to .95 for > downstream as other wise ping go very high when line saturated Sure, the 98.46% of sync is a theoretical upper limit and not a really reachable for ingress (but you might want tto try the new "ingress" keyword on eth0.1) > > > root@OpenWrt-Mi3G:~# cat /etc/config/sqm > > config queue 'eth1' > option debug_logging '0' > option verbosity '5' > option qdisc 'cake' > option qdisc_advanced '1' > option ingress_ecn 'ECN' > option egress_ecn 'NOECN' > option qdisc_really_really_advanced '1' > option script 'layer_cake.qos' > option interface 'eth0.2' > option enabled '1' > option linklayer 'ethernet' > option squash_dscp '1' > option squash_ingress '1' > option download '0' > option upload '2440' > option overhead '30' > option iqdisc_opts 'nat rtt 50000 mpu 64 dual-dsthost' > option eqdisc_opts 'nat rtt 50000 mpu 64 dual-srchost diffserv4' > > config queue > option debug_logging '0' > option verbosity '5' > option download '0' > option qdisc 'cake' > option script 'layer_cake.qos' > option qdisc_advanced '1' > option squash_dscp '0' > option squash_ingress '0' > option ingress_ecn 'ECN' > option qdisc_really_really_advanced '1' > option egress_ecn 'ECN' > option interface 'eth0.1' > option enabled '1' > option linklayer 'ethernet' > option overhead '30' > option eqdisc_opts 'nat rtt 50000 mpu 64 dual-dsthost diffserv4' > option upload '30500' > > > 4) tell us about your ISP and plan ;) > ISP is Go Malta VDSL2 70/10, however i'm very far from cabinet so i sync at > 32/2.48 Thanks, I had a look, but i could not easily whether they use a VLAN on their vdsl2 link. Best Regards Sebastian > > > > > I tried using ATM_overhead_detector and get the following output > > Well, yes sorry, this will not work at all for links using PTM; in > theory VDSL links could be using ATM and still be in compliance with the ITU > specs (at lest that is my reading) but fortunately nobody seems to be doing > that. (Also more unfortunately no ISP sseems to be using PTM encoding on ADSL > links, but I digress). > > > > > 869001 lines parsed... > > Found 144921 ping packets in > > /home/mark/src/ATM_overhead_detector/ping_sweep__20171217_075202.txt > > Elapsed time is 972.739 seconds. > > Minimum size of ping payload used: 16 bytes. > > warning: division by zero > > warning: called from > > ATM_overhead_detector at line 201 column 15 > > warning: legend: ignoring extra labels > > Unknown or ambiguous terminal name 'wxt' > > Unknown or ambiguous terminal name 'wxt' > > Saved figure (5) to: > > /home/mark/src/ATM_overhead_detector/ping_sweep__20171217_075202_data.png > > lower bound estimate for one ATM cell RTT based of specified up and > > downlink is 0.052419 ms. > > estimate for one ATM cell RTT based on linear fit of the ping sweep data is > > 0.052419 ms. > > Starting brute-force search for optimal stair fit, might take a while... > > Unknown or ambiguous terminal name 'wxt' > > Unknown or ambiguous terminal name 'wxt' > > Best staircase fit cumulative difference is: 2.6952 > > Best linear fit cumulative difference is: 2.7314 > > Quantized ATM carrier LIKELY (cummulative residual: stair fit 2.6952 linear > > fit 2.7314 > > Yes, that difference is simply too small to be meaningful. I am > currently collecting reference values for these for different technologies > and it seems on real ATM/AAL5 links linear residuals are >= 2 * stair > residuals, but curently the code simply reports the smaller value as more > likely... > > > > remaining ATM cell length after ICMP header is 5 bytes. > > ICMP RTT of a single ATM cell is 0.05871 ms. > > > > Estimated overhead preceding the IP header: 48 bytes > > Saved figure (6) to: > > /home/mark/src/ATM_overhead_detector/ping_sweep__20171217_075202_results.png > > > > According to http://ace-host.stuart.id.au/russell/files/tc/tc-atm/ > > 48 bytes overhead indicate > > Overhead of 0 bytes is not possible so assume 1 full packet (48 bytes) > > overhead... > > Connection: Bridged, LLC/SNAP+FCS RFC-1483/2684 + VLAN tag terminated at > > modem > > Protocol (bytes): Ethernet Header (14), VLAN tag (4), Ethernet PAD [8] (0), > > Ethernet Checksum (4), ATM LLC (3), ATM SNAP (5), ATM pad (2), ATM AAL5 SAR > > (8) : Total 36 > > > > Add the following to both the egress root qdisc: > > A) Assuming the router connects over ethernet to the DSL-modem: > > stab mtu 2048 tsize 128 overhead 48 linklayer atm > > > > Add the following to both the ingress root qdisc: > > > > A) Assuming the router connects over ethernet to the DSL-modem: > > stab mtu 2048 tsize 128 overhead 48 linklayer atm > > > > Elapsed time is 1042.06 seconds. > > Done... > > ans = [](0x0) > > > > > > Could my VDSL2 be using ATM? so i have the wrong overhead set? > > Yes in theory it could, but I am almost 100% certain it is not doing > that (AFAICT ISPs try to get rid of all ATM equipment ASAP). But I also > believe you have the wrong overhead specified... (if you insist on the > keywords try pppoe-ptm). Now, please keep in mind that you properly need to > "model" the encapsulation on the bottleneck link, if your ISP uses a traffic > shaper on the otherside of your link you need to actually model that shapers > settings. As far as I can see, ISPs will model the per packet overhead > roughly correctly, but will reduce the bandwidth from the sync values > reported by the modem, this requires a few iterations of trial and error to > get right. Also some ISPs also use VLAN tags on the bottleneck link... > > Hope this helps... > > Best Regards > Sebastian > > Here is tc output > > root@OpenWrt-Mi3G:~# tc -s qdisc > qdisc noqueue 0: dev lo root refcnt 2 > Sent 0 bytes 0 pkts (dropped 0, overlimits 0) > qdisc fq_codel 0: dev eth0 root refcnt 2 limit 10240p flows 1024 quantum 1514 > target 5.0ms interval 100.0ms memory_limit 4Mb ecn > Sent 13974072314 bytes 14402910 pkts (dropped 0, overlimits 0) > maxpacket 17032 drop_overlimit 0 new_flow_count 132368 ecn_mark 0 > new_flows_len 0 old_flows_len 0 > qdisc noqueue 0: dev br-lan root refcnt 2 > Sent 0 bytes 0 pkts (dropped 0, overlimits 0) > qdisc cake 8069: dev eth0.1 root refcnt 2 bandwidth 31500Kbit diffserv4 > dual-dsthost nat rtt 50.0ms raw total_overhead 14 hard_header_len 14 mpu 64 > Sent 89833020 bytes 59632 pkt (dropped 5, overlimits 44617 requeues 0) > backlog 0b 0p requeues 0 > memory used: 14112b of 4Mb > capacity estimate: 31500Kbit > Bulk Best Effort Video Voice > thresh 1968Kbit 31500Kbit 15750Kbit 7875Kbit > target 9.2ms 2.5ms 2.5ms 2.5ms > interval 56.7ms 50.0ms 50.0ms 50.0ms > pk_delay 7us 4.3ms 3.5ms 3.2ms > av_delay 0us 183us 173us 271us > sp_delay 0us 7us 8us 7us > pkts 4 408 20384 38841 > bytes 717 64851 30954992 58820110 > way_inds 0 0 0 0 > way_miss 4 64 4 11 > way_cols 0 0 0 0 > drops 0 0 5 0 > marks 0 0 0 0 > ack_drop 0 0 0 0 > sp_flows 0 0 0 0 > bk_flows 0 0 1 0 > un_flows 0 0 0 0 > max_len 189 1536 2988 2988 > > qdisc cake 8067: dev eth0.2 root refcnt 2 bandwidth 2440Kbit diffserv4 > dual-srchost nat rtt 50.0ms raw total_overhead 14 hard_header_len 14 mpu 64 > Sent 3780967 bytes 33806 pkt (dropped 130, overlimits 12207 requeues 0) > backlog 10860b 9p requeues 0 > memory used: 116928b of 4Mb > capacity estimate: 2440Kbit > Bulk Best Effort Video Voice > thresh 152496bit 2440Kbit 1220Kbit 610Kbit > target 119.1ms 7.4ms 14.9ms 29.8ms > interval 238.3ms 54.9ms 62.4ms 77.3ms > pk_delay 0us 101.5ms 8.2ms 7us > av_delay 0us 49.7ms 2.6ms 0us > sp_delay 0us 6.8ms 245us 0us > pkts 0 22757 11178 10 > bytes 0 3029896 951692 1096 > way_inds 0 0 0 0 > way_miss 0 61 3 3 > way_cols 0 0 0 0 > drops 0 130 0 0 > marks 0 0 0 0 > ack_drop 0 0 0 0 > sp_flows 0 1 1 0 > bk_flows 0 1 0 0 > un_flows 0 0 0 0 > max_len 0 11700 120 179 > > qdisc noqueue 0: dev wlan1 root refcnt 2 > Sent 0 bytes 0 pkts (dropped 0, overlimits 0) > qdisc noqueue 0: dev wlan0 root refcnt 2 > Sent 0 bytes 0 pkts (dropped 0, overlimits 0) > qdisc fq_codel 0: dev tun0 root refcnt 2 limit 10240p flows 1024 quantum 1500 > target 5.0ms interval 100.0ms memory_limit 4Mb ecn > Sent 304 bytes 4 pkts (dropped 0, overlimits 0) > maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0 > new_flows_len 0 old_flows_len 0 > > Best regards, > Mark > > > > here is my tc -s qdisc output > > > > root@OpenWrt-Mi3G:~# tc -s qdisc > > qdisc noqueue 0: dev lo root refcnt 2 > > Sent 0 bytes 0 pkts (dropped 0, overlimits 0) > > qdisc fq_codel 0: dev eth0 root refcnt 2 limit 10240p flows 1024 quantum > > 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn > > Sent 3460986864 bytes 3651814 pkts (dropped 0, overlimits 0) > > maxpacket 17032 drop_overlimit 0 new_flow_count 29673 ecn_mark 0 > > new_flows_len 0 old_flows_len 0 > > qdisc noqueue 0: dev br-lan root refcnt 2 > > Sent 0 bytes 0 pkts (dropped 0, overlimits 0) > > qdisc cake 8055: dev eth0.1 root refcnt 2 bandwidth 30Mbit diffserv4 > > dual-dsthost nat rtt 50.0ms ptm overhead 22 via-ethernet total_overhead 22 > > hard_header_len 14 > > Sent 20012069 bytes 35472 pkt (dropped 10, overlimits 7587 requeues 0) > > backlog 0b 0p requeues 0 > > memory used: 36684b of 4Mb > > capacity estimate: 30Mbit > > Bulk Best Effort Video Voice > > thresh 1875Kbit 30Mbit 15Mbit 7500Kbit > > target 9.7ms 2.5ms 2.5ms 2.5ms > > interval 57.2ms 50.0ms 50.0ms 50.0ms > > pk_delay 0us 5.6ms 855us 15us > > av_delay 0us 436us 28us 10us > > sp_delay 0us 27us 8us 6us > > pkts 0 33679 457 1346 > > bytes 0 19869595 80517 69940 > > way_inds 0 249 0 0 > > way_miss 0 3099 19 7 > > way_cols 0 0 0 0 > > drops 0 10 0 0 > > marks 0 0 0 0 > > ack_drop 0 0 0 0 > > sp_flows 0 1 1 0 > > bk_flows 0 1 0 0 > > un_flows 0 0 0 0 > > max_len 0 7064 2982 350 > > > > qdisc cake 8053: dev eth0.2 root refcnt 2 bandwidth 2400Kbit diffserv4 > > dual-srchost nat rtt 50.0ms ptm overhead 22 via-ethernet total_overhead 22 > > hard_header_len 14 > > Sent 23218772 bytes 35904 pkt (dropped 205, overlimits 27550 requeues 0) > > backlog 0b 0p requeues 0 > > memory used: 94752b of 4Mb > > capacity estimate: 2400Kbit > > Bulk Best Effort Video Voice > > thresh 150Kbit 2400Kbit 1200Kbit 600Kbit > > target 121.1ms 7.6ms 15.1ms 30.3ms > > interval 242.2ms 55.1ms 62.6ms 77.8ms > > pk_delay 0us 29.5ms 19us 4.4ms > > av_delay 0us 3.8ms 1us 2.3ms > > sp_delay 0us 50us 1us 9us > > pkts 0 34979 31 1099 > > bytes 0 23342354 2930 162457 > > way_inds 0 177 0 0 > > way_miss 0 2003 27 16 > > way_cols 0 0 0 0 > > drops 0 205 0 0 > > marks 0 0 0 0 > > ack_drop 0 0 0 0 > > sp_flows 0 1 0 0 > > bk_flows 0 1 0 0 > > un_flows 0 0 0 0 > > max_len 0 8342 98 598 > > > > qdisc noqueue 0: dev wlan1 root refcnt 2 > > Sent 0 bytes 0 pkts (dropped 0, overlimits 0) > > qdisc noqueue 0: dev wlan0 root refcnt 2 > > Sent 0 bytes 0 pkts (dropped 0, overlimits 0) > > qdisc fq_codel 0: dev tun0 root refcnt 2 limit 10240p flows 1024 quantum > > 1500 target 5.0ms interval 100.0ms memory_limit 4Mb ecn > > Sent 304 bytes 4 pkts (dropped 0, overlimits 0) > > maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0 > > new_flows_len 0 old_flows_len 0 > > > > Thanks, > > Mark > > _______________________________________________ > > Cake mailing list > > [email protected] > > https://lists.bufferbloat.net/listinfo/cake _______________________________________________ Cake mailing list [email protected] https://lists.bufferbloat.net/listinfo/cake
