Re: [Cerowrt-devel] performance numbers from WRT1200AC (Re: Latest build test - new sqm-scripts seem to work; cake overhead 40 didn't)
On Thu, 25 Jun 2015, Toke Høiland-Jørgensen wrote: Dave Taht dave.t...@gmail.com writes: Unfortunately the core piece of metadata I wanted from the router was the qdisc statistics. Didnt parse. Will file bug. My guess is that in this case it's due to the openwrt box missing the tc and ip binaries - those are not installed by default... I did have tc, but I didn't have ip. Do you need ip or ip-full? I installed ip. Or rather, sqm-scripts pulls in the tc binary, I think, but Flent uses the ip binary to get the ip and route information to know from which interfaces to pull the qdisc information from... We'll see if my next test run includes the correct information then. -- Mikael Abrahamssonemail: swm...@swm.pp.se___ Cerowrt-devel mailing list Cerowrt-devel@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cerowrt-devel
Re: [Cerowrt-devel] performance numbers from WRT1200AC (Re: Latest build test - new sqm-scripts seem to work; cake overhead 40 didn't)
On Thu, 25 Jun 2015, Dave Taht wrote: on your host, server, or switch. (you have pfifo_fast on your host) Try fq_codel on host and server (and/or sch_fq) and see what happens. Disable tso/gro/gso on your server/host also. That leaves the switch which I have no insight into. What switch chip is it? (see /etc/config/network) - on the cerowrt project we got less buffering out of the switch by enabling jumbo frames. So the complete physical setup is: Linux laptop cable WRT1200AC cable TP-Link TL-SG3424 L2 switch cable macbook pro thunderbolt port This means there are multiple places buffering can occur, that doesn't have fq_codel. I can't tell what switch chip is being used, it's not on http://techinfodepot.info/wiki/Linksys_WRT1200AC and I don't know where to look to find it. In /etc/config/network there is just br-lan and eth0 and eth1. -- Mikael Abrahamssonemail: swm...@swm.pp.se ___ Cerowrt-devel mailing list Cerowrt-devel@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cerowrt-devel
Re: [Cerowrt-devel] performance numbers from WRT1200AC (Re: Latest build test - new sqm-scripts seem to work; cake overhead 40 didn't)
On Fri, Jun 26, 2015 at 9:36 AM, Mikael Abrahamsson swm...@swm.pp.se wrote: On Fri, 26 Jun 2015, Sebastian Moeller wrote: Tangent: What is the shaper rate the wdr4900 can push with sqm-scripts? (Before your 1200ac results the ppc-soc in the wdr4900 looked like the finest little router platform in the last years, too bad it was ignored by the mass market...) Well, if I set SQM to 500 megabit/s and MSS to 300 I am able to get 70 megabit/s of iperf3 through it at 42k PPS. At default MSS I get 150 megabit/s at 25k PPS through it. This is at 100% sirq. Does this help, or do you want me to do any other tests. I have the WDR4900 powered up and on my desk right now. I am never allergic to somene running a comprehensive flent suite through something, and sticking the results up somewhere. :) There are always more surprises and more things we seem to have to wackamole. -- Mikael Abrahamssonemail: swm...@swm.pp.se ___ Cerowrt-devel mailing list Cerowrt-devel@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cerowrt-devel -- Dave Täht worldwide bufferbloat report: http://www.dslreports.com/speedtest/results/bufferbloat And: What will it take to vastly improve wifi for everyone? https://plus.google.com/u/0/explore/makewififast ___ Cerowrt-devel mailing list Cerowrt-devel@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cerowrt-devel
Re: [Cerowrt-devel] performance numbers from WRT1200AC (Re: Latest build test - new sqm-scripts seem to work; cake overhead 40 didn't)
On Fri, 26 Jun 2015, Jonathan Morton wrote: Hypothesis: this might have to do with the receive path. Some devices might have more capacity than others to buffer inbound packets until the CPU can get around to servicing them. Is there a way to tell? I am better at diagnosing Cisco CPU based routers than Linux ones. I looked in /sys/class/net/ethX/statistics but these drops are not recorded there. -- Mikael Abrahamssonemail: swm...@swm.pp.se ___ Cerowrt-devel mailing list Cerowrt-devel@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cerowrt-devel
Re: [Cerowrt-devel] performance numbers from WRT1200AC (Re: Latest build test - new sqm-scripts seem to work; cake overhead 40 didn't)
On Fri, Jun 26, 2015 at 9:18 AM, Jonathan Morton chromati...@gmail.com wrote: Hypothesis: this might have to do with the receive path. Some devices might have more capacity than others to buffer inbound packets until the CPU can get around to servicing them. *Good* hypothesis. I am certain I have seen this on multiple occasions on other hardware. Hard to confirm. Wet paint... so I finally got off my arse and looked at the driver this morning. given that this is a multi-core box, I would lean towards a smaller napi_poll_weight, which unfortunately is a constant (64) in the code. 4 cores can take interrupts faster. (and I hate napi on routers) I have sometimes longed for an IQL (ingress queue limits) to also handle differences in packet size, dynamically changing the poll weight based on load - increasing it for loads with lots of small packets, decreasing it for lots of big packets. Furthermore this thing is doing software gro (up to 64 packets at a time) which is a LOT of processing at this layer in the stack. Its a two line patch to cut the weight to 16, but I have never managed to get a working build for this platform. - Jonathan Morton ___ Cerowrt-devel mailing list Cerowrt-devel@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cerowrt-devel -- Dave Täht worldwide bufferbloat report: http://www.dslreports.com/speedtest/results/bufferbloat And: What will it take to vastly improve wifi for everyone? https://plus.google.com/u/0/explore/makewififast ___ Cerowrt-devel mailing list Cerowrt-devel@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cerowrt-devel
Re: [Cerowrt-devel] performance numbers from WRT1200AC (Re: Latest build test - new sqm-scripts seem to work; cake overhead 40 didn't)
Hi Mikael, On Jun 26, 2015, at 14:26 , Mikael Abrahamsson swm...@swm.pp.se wrote: On Fri, 26 Jun 2015, Sebastian Moeller wrote: Thanks for the tests, now I know what router to try next (the edgerouterX, which I had eyed as a replacement for the shaper in the wndr3700 tops out at 130K packets per second and hence will not really work that well for a 100/40 Mbps link). I did some more tests and it seems with SQM at 500 megabit/s I start to lose packets at around 250-300k PPS. At these speeds, the rrul_be test is only 85k PPS at 500 megabit/s bidirectional with large packets. Ah, that sounds like there 1200ac is a practical solution today; at 500Mbps there are 500^3/(64*8) = 244 Kpps at the smallest size (since the wire is at 1000 Mbps I think we should and can ignore inter frame gap and preamble, SQM certainly ignores them) so it looks like the router is really close to the theoretical maximum, and with larger packets things get way more relaxed (500^3/(1518*8) = 10 Kpps). Pretty decent for a router using no proprietary offload features. Also, I have an Edgerouter ER-5, but as soon as it does CPU based forwarding it's really weak, easily under 100 megabit/s even with large packets. OpenVPN without encryption is less than 20 megabit/s. Ah, interesting, so neither the affordable edge router lite nor the (similar) ER-5 will cut it unless the offloads are used; I think the newer edgerouterX is rated for slightly higher speeds without offloads, but not nearly close to what your 1200ac does... Btw, the WRT1200AC is now becoming more widely available and it's 150 EUR incl 25% VAT and shipping here in Sweden now. In Germany it still retails for around 200EUR with the cheapest offer at 180EUR (but that is an outlier); guess I should visit Sweden then ;) Btw, I tried WNDR3800 setting it to 100/100 SQM. Yeah, not pretty, huh? It seems to max out around 25-30k PPS, but the difference is that when the CPU is full, it seems to delay/ECN-mark packets because there are no packets lost. When the WRT1200AC runs out of CPU it starts dropping packets. I always have 0 packets lost with the WNDR3800 when doing iperf3 testing. I found this difference interesting, wonder where in the forwarding path the WRT1200AC loses packets? Interesting observation, no idea, but intrigued ;) Best Regards Sebastian -- Mikael Abrahamssonemail: swm...@swm.pp.se ___ Cerowrt-devel mailing list Cerowrt-devel@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cerowrt-devel
Re: [Cerowrt-devel] that latest build I did is looking good
On Jun 25, 2015 10:35 AM, Dave Taht dave.t...@gmail.com wrote: I have been abusing it on a picostation and nanostation now for 48 hours. The archer c7v2 (as a source specific gateway) for a week. A couple wndr3800s. No crashes. Can still trigger the dreaded wifi TX DMA bug, but it seems harder now. DID regularly crash the iwl in one box. [3] Did you update with the latest Qualcomm v5 ath10k firmware and associated mac80211 patches that were committed into upstream openwrt trunk a couple of days ago? My last builds from end of May died several days ago after around a month of uptime. Probably due to the power zone bugs in wifi but I can't check until I get back to Vancouver tonight (currently in Mountain View) My mission was to get to something that I could deploy here at a small scale and just let run for a month while away in the eu, and that was looking dicy there for a while. (I am glad to have basically started in april!) So... we do, finally, have an openwrt build that uses cake, has the minstrel-blues patches, and andrews minimum variance patches, working dnssec (we hope!), and a new version of babel with ecn enabled, has snmp, that does dhcp-pd fairly right, and works with comcast. I also have things (odhcp6c is way better than isc, dibbler, wide) working fairly well with another debian based firewall. If/when new cake or sqm stuff arrives my plan (barring other major bugs elsewhere) is to just incrementally build that and tc-adv out of the above frozen repo. [1] :whew: Have to go climb a couple trees this week. Have 3 source specific gateways up, might get two more. [2] * babel bug - channel diversity routing does not autodetect the actual channel you are on. Setting it manually works, but is a pita to remember to do... fixing it is a mere matter of grokking the iw nl80211 code. * Juliusz does not like having a routing protocol that uses ecn but does not respond to it (yet!), (I agree partially) but me, what I see is routing suffering from congestive failures periodically, and I am most interested in what is going on at precisely the point of congestion... particularly the rtt timestamps now in babel... so I left it in. DID prove that even a minimalistic routing protocol in a fq_codel environment on present day wifi can suffer from significant congestive loss. (or in this case, get CE marked). Distinguishing between path connectivity loss - and congestion - seems worthwhile in a routing protocol. Pull up wireshark on: http://snapon.lab.bufferbloat.net/~d/newrouters/linuxcap.cap use a ipv6.traffic_class.ce == 1 filter in wireshark. An observation: routing stuff that does not use IP (like arp, and I think batman and ISIS) cannot use ecn at all... - I did not bother testing hnetd. Simply not enough time to test it. What I plan to do is just backhaul a few fixed ipv6 prefixes to the core locations that can use them after finishing up: https://github.com/dtaht/ipv6_selfassign - and try to do more to figure out hnetd at ietf, or look over dhcp-pd again for internal use. - did not do package signing [1] If you have any missing packages you need from that repo, tell me now. But the goal is to push up tc-adv and cake into the openwrt mainline repos next running on things like the linksys 1200ac... then get minstrel-blues straightened out... and start on per station queueing... - and I am still not in a position to do another cerowrt-like effort. Where do we go with this? I feel like we are limping along Still, if you want to play along, knowing full well we will be stuck here for months or forever: http://snapon.lab.bufferbloat.net/~cero3/lupin/ar71xx/ [2] a pita in my environment is to get the picos and nanos setup right (unbridged, adhoc, snmp, etc) , so I am doing separate builds for that. [3] Some flent data here: snapon.lab.bufferbloat.net/~d/newrouters.tgz everything with the word linux in the title was a the iwl using linux box. the other tests were driven by OSX. It was interesting/depressing to see how much more airtime the osx box grabbed, while the linux box was quite fair to the AP. Do not have any longer range tests... that awaits tree-climbing. -- Dave Täht worldwide bufferbloat report: http://www.dslreports.com/speedtest/results/bufferbloat And: What will it take to vastly improve wifi for everyone? https://plus.google.com/u/0/explore/makewififast ___ Cerowrt-devel mailing list Cerowrt-devel@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cerowrt-devel ___ Cerowrt-devel mailing list Cerowrt-devel@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cerowrt-devel
Re: [Cerowrt-devel] performance numbers from WRT1200AC (Re: Latest build test - new sqm-scripts seem to work; cake overhead 40 didn't)
On Fri, 26 Jun 2015, Mikael Abrahamsson wrote: Btw, I tried WNDR3800 setting it to 100/100 SQM. It seems to max out around 25-30k PPS, but the difference is that when the CPU is full, it seems to delay/ECN-mark packets because there are no packets lost. When the WRT1200AC runs out of CPU it starts dropping packets. I always have 0 packets lost with the WNDR3800 when doing iperf3 testing. I found this difference interesting, wonder where in the forwarding path the WRT1200AC loses packets? I checked again, and my WDR4900 with same setup doesn't lose packets either. Even at 99% sirq, no packets are lost. WRT1200AC starts to lose packets at 500 megabit/s SQM around MSS 300 and lower. If I turn off gso, tso and gro, I have to go to MSS 600 and above to avoid packet loss. Does flent check for packet loss at all? Perhaps it's something to look into, because with ECN we really don't want to see any packets lost and this might be good to include test results for. -- Mikael Abrahamssonemail: swm...@swm.pp.se ___ Cerowrt-devel mailing list Cerowrt-devel@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cerowrt-devel
Re: [Cerowrt-devel] performance numbers from WRT1200AC (Re: Latest build test - new sqm-scripts seem to work; cake overhead 40 didn't)
On Fri, 26 Jun 2015, Dave Taht wrote: your results are showing basically tail drop behavior. Although I would have expected intrinsic delay on the link to crack 100mbits on the rrul test, not 20ms (which is still high), and you only hit 7 on the single threaded tcp up test, based on what I saw in the driver. turn off sqm, stay at 100mbit, let fq_codel ride, try the rrul_50_up test. Also do a capture to see if CE was ever exerted. http://swm.pp.se/aqm/rrul_50_up-2015-06-27T072819.166452.iperf3_150626_9_100M.flent.gz I have a capture of this as well, it's 1.5gigs. It has lots of mentions of ECT(0) but 0 mention of ECT(1). TOS should be 0x3 when the EC=1 ? I see no such packets. All the packets are 0x2. http://swm.pp.se/aqm/rrul_50up-100M.pcap.bz2 I went back to gig and set SQM to 50 megabit/s bidirectional and looking at tc qdisc -s eth0 I see ecn_mark 10339 on a single queue, none of the others have non-zero ecn_mark. I do not know why my linksys ac1200 build does not work. I generally suspect it is because my big build server is getting ancient. Can you send me your .config file for openwrt? http://swm.pp.se/aqm/wrt1200ac.config This was my first try, I haven't fine tuned it yet, but it works (boots and moves packets anyway :P ) It would be VERY helpful if you pulled from ceropackages-3.14, and added and built kmod-sched-cake and tc-adv and switched to using the ceropackages feed also for luci-app-sqm and sqm-scripts. There are a couple people here that would probably leap on that! :) I will give it a go in the next few days. -- Mikael Abrahamssonemail: swm...@swm.pp.se ___ Cerowrt-devel mailing list Cerowrt-devel@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cerowrt-devel
Re: [Cerowrt-devel] Any chance for the LUCI GUI for cake appearing?
Hi Jim, On Jun 26, 2015, at 19:21 , Jim Gettys j...@freedesktop.org wrote: Inquiring minds would like to know…. I guess, that depends on your definition of “LUCI GUI for cake”; as far as I know you can select cake in the “Queueing Discipline” drop down in the “Queue Discipline” tab of luci-app-sqm. So if the machine has the cake module and a sufficiently recent (out-of-tree) tc binary installed it should just work. Using the “Advanced option strings…” it even should be possible to pass arbitrary strings to cake (without any error checking), so I assume we are all set up for cake testing (or should that be cake tasting?) from the LUCI GUI. But I have not managed to build cake locally, nor have I dared to risk my families internet connectivity by trying one of Dave’s newer openwrt trunk with cake” builds. So these changes linger in the ceropackages-3.10 repository until we get enough tasting to convince Toke to pull them into the openwrt repositories (these changes affect both sqm-scripts and luci-app-sqm). Dave graciously allowed me access to one of his test machines, and the changes allow to set up cake from the GUI, but I only did limited functionality testing. (And we discovered that something was of in that cake worked but did not accept all the keywords it should have, but unless you are on a XDSL-Link you most likely do not care about specifying per-packet-overhead) If you have any specific thing you are missing, let me know and I will try to help. Best Regards Sebastian - Jim ___ Cerowrt-devel mailing list Cerowrt-devel@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cerowrt-devel ___ Cerowrt-devel mailing list Cerowrt-devel@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cerowrt-devel
Re: [Cerowrt-devel] performance numbers from WRT1200AC (Re: Latest build test - new sqm-scripts seem to work; cake overhead 40 didn't)
On Fri, 26 Jun 2015, Dave Taht wrote: Mikael, a simple test of the analysis I just did would be to use ethtool to set your server to 100mbits (ethtool -s your_ethernet_device advertise 0x008 and turn on fq_codel on both the client and server. Hm what do you mean by client and server? Where do you want the queueing to happen? Egress from the WRT1200AC towards the server? Then setting the WAN port of the WRT1200AC to 100 megabit/s would work, yes. -- Mikael Abrahamssonemail: swm...@swm.pp.se ___ Cerowrt-devel mailing list Cerowrt-devel@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cerowrt-devel
Re: [Cerowrt-devel] performance numbers from WRT1200AC (Re: Latest build test - new sqm-scripts seem to work; cake overhead 40 didn't)
On Fri, Jun 26, 2015 at 9:35 AM, Jonathan Morton chromati...@gmail.com wrote: These would be hardware tail drops - there might not be a physical counter recording them. But you could instrument three driver to see whether the receive buffer is full when serviced. from drivers/net/ethernet/marvel/mvneta.c: /* Max number of Rx descriptors */ #define MVNETA_MAX_RXD 128 this is probably too small, especially given the 64 it is willing to wait for. At the same time, it is too large, as there are 8 hardware queues in play here. So you get a huge burst from one flow, it gros it all together aggghh... /* Max number of Tx descriptors */ #define MVNETA_MAX_TXD 532 this really needs BQL. Same problem(s). Only worse. - Jonathan Morton ___ Cerowrt-devel mailing list Cerowrt-devel@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cerowrt-devel -- Dave Täht worldwide bufferbloat report: http://www.dslreports.com/speedtest/results/bufferbloat And: What will it take to vastly improve wifi for everyone? https://plus.google.com/u/0/explore/makewififast ___ Cerowrt-devel mailing list Cerowrt-devel@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cerowrt-devel
Re: [Cerowrt-devel] lacking in BQL in the mvneta, what is the max latency?
On Fri, 26 Jun 2015, Dave Taht wrote: (am I looking at the right driver for the linksys ac1200? mikael? what does lspci and/or dmesg say for both this and the wifi on this platform?) lvneta and mwlwifi according to ethtool -i eth0 and wlan0 respectively. -- Mikael Abrahamssonemail: swm...@swm.pp.se ___ Cerowrt-devel mailing list Cerowrt-devel@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cerowrt-devel
Re: [Cerowrt-devel] performance numbers from WRT1200AC (Re: Latest build test - new sqm-scripts seem to work; cake overhead 40 didn't)
On Fri, 26 Jun 2015, Dave Taht wrote: I am never allergic to somene running a comprehensive flent suite through something, and sticking the results up somewhere. http://swm.pp.se/aqm/wdr4900-150626-9.tar Happy no sneezing! -- Mikael Abrahamssonemail: swm...@swm.pp.se ___ Cerowrt-devel mailing list Cerowrt-devel@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cerowrt-devel
Re: [Cerowrt-devel] Any chance for the LUCI GUI for cake appearing?
On Jun 26, 2015, at 21:48 , Dave Taht dave.t...@gmail.com wrote: It turns out that I am pulling from openwrt for luci-app-sqm, not ceropackages, and overriding it was a headache. Fixed now, see below router# cd /tmp; wget http://snapon.lab.bufferbloat.net/~cero3/luci-app-sqm_3-5_all.ipk ; opkg install ./luci-app-sqm_3-5_all.ipk router # /etc/init.d/uhttpd restart if it breaks, you get to keep both pieces, but it worked for me, and yea! cake in the gui! thx sebastian! Too cite Shaggy “Wasn’t me” ;); the initial commit was not mine and I do not want to claim otherwise. I basically just rearranged the deck-chairs as I tend to think. I seem to be better at the “ love for details” thing than the”big picture”, anyways Best Regards Sebastian I tried - ./scripts feeds uninstall luci-app-sqm; ./scripts feeds install -p cero luci-app-sqm # seemed sane, did not work git rm in the feed itself did not work doing the uninstall, deleting the symlink, and the git rm, nuking the bin/ar71xx/packages/packages/luci-app-sqm* and doing a complete rebuild... eventually did work... but strikes me as a bit more brute force than I wanted. I am not in a position (aformentioned build is my specialized for pico/nano build at the moment) to update the packages db, but I stuck up the correct new package... which you can install via Apologies for the confusion, I simply figured that the -p option would work, and never checked. I also note that I think I am building things compile tuned for the archer mips34c, rather thanthe wndr3800´s 24c. However I kept the unaligned access hacks (not needed on the 34c derived archer, but needed on the 24c wndr3800) Sigh. maintaining 3 separate trees for the platforms we sort of support is a PITA. Then adding in something that can mesh by default on 2 others. and then trying to get anything to compile for another arch entirely and failing... On Fri, Jun 26, 2015 at 11:49 AM, Sebastian Moeller moell...@gmx.de wrote: Hi Jim, On Jun 26, 2015, at 19:21 , Jim Gettys j...@freedesktop.org wrote: Inquiring minds would like to know…. I guess, that depends on your definition of “LUCI GUI for cake”; as far as I know you can select cake in the “Queueing Discipline” drop down in the “Queue Discipline” tab of luci-app-sqm. So if the machine has the cake module and a sufficiently recent (out-of-tree) tc binary installed it should just work. Using the “Advanced option strings…” it even should be possible to pass arbitrary strings to cake (without any error checking), so I assume we are all set up for cake testing (or should that be cake tasting?) from the LUCI GUI. But I have not managed to build cake locally, nor have I dared to risk my families internet connectivity by trying one of Dave’s newer openwrt trunk with cake” builds. So these changes linger in the ceropackages-3.10 repository until we get enough tasting to convince Toke to pull them into the openwrt repositories (these changes affect both sqm-scripts and luci-app-sqm). Dave graciously allowed me access to one of his test machines, and the changes allow to set up cake from the GUI, but I only did limited functionality testing. (And we discovered that something was of in that cake worked but did not accept all the keywords it should have, but unless you are on a XDSL-Link you most likely do not care about specifying per-packet-overhead) If you have any specific thing you are missing, let me know and I will try to help. Best Regards Sebastian - Jim ___ Cerowrt-devel mailing list Cerowrt-devel@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cerowrt-devel ___ Cerowrt-devel mailing list Cerowrt-devel@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cerowrt-devel -- Dave Täht worldwide bufferbloat report: http://www.dslreports.com/speedtest/results/bufferbloat And: What will it take to vastly improve wifi for everyone? https://plus.google.com/u/0/explore/makewififast ___ Cerowrt-devel mailing list Cerowrt-devel@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cerowrt-devel
Re: [Cerowrt-devel] performance numbers from WRT1200AC (Re: Latest build test - new sqm-scripts seem to work; cake overhead 40 didn't)
HI Mikael, On Jun 26, 2015, at 16:49 , Mikael Abrahamsson swm...@swm.pp.se wrote: On Fri, 26 Jun 2015, Mikael Abrahamsson wrote: Btw, I tried WNDR3800 setting it to 100/100 SQM. It seems to max out around 25-30k PPS, but the difference is that when the CPU is full, it seems to delay/ECN-mark packets because there are no packets lost. When the WRT1200AC runs out of CPU it starts dropping packets. I always have 0 packets lost with the WNDR3800 when doing iperf3 testing. I found this difference interesting, wonder where in the forwarding path the WRT1200AC loses packets? I checked again, and my WDR4900 with same setup doesn't lose packets either. Even at 99% sirq, no packets are lost. Tangent: What is the shaper rate the wdr4900 can push with sqm-scripts? (Before your 1200ac results the ppc-soc in the wdr4900 looked like the finest little router platform in the last years, too bad it was ignored by the mass market...) WRT1200AC starts to lose packets at 500 megabit/s SQM around MSS 300 and lower. If I turn off gso, tso and gro, I have to go to MSS 600 and above to avoid packet loss. So the offloads buy roughly a doubling of the achievable packet rate, not bad, but as your results show also knot essential. Best Regards Sebastian Does flent check for packet loss at all? Perhaps it's something to look into, because with ECN we really don't want to see any packets lost and this might be good to include test results for. -- Mikael Abrahamssonemail: swm...@swm.pp.se ___ Cerowrt-devel mailing list Cerowrt-devel@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cerowrt-devel
Re: [Cerowrt-devel] performance numbers from WRT1200AC (Re: Latest build test - new sqm-scripts seem to work; cake overhead 40 didn't)
On Fri, 26 Jun 2015, Sebastian Moeller wrote: Tangent: What is the shaper rate the wdr4900 can push with sqm-scripts? (Before your 1200ac results the ppc-soc in the wdr4900 looked like the finest little router platform in the last years, too bad it was ignored by the mass market...) Well, if I set SQM to 500 megabit/s and MSS to 300 I am able to get 70 megabit/s of iperf3 through it at 42k PPS. At default MSS I get 150 megabit/s at 25k PPS through it. This is at 100% sirq. Does this help, or do you want me to do any other tests. I have the WDR4900 powered up and on my desk right now. -- Mikael Abrahamssonemail: swm...@swm.pp.se ___ Cerowrt-devel mailing list Cerowrt-devel@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cerowrt-devel
Re: [Cerowrt-devel] performance numbers from WRT1200AC (Re: Latest build test - new sqm-scripts seem to work; cake overhead 40 didn't)
These would be hardware tail drops - there might not be a physical counter recording them. But you could instrument three driver to see whether the receive buffer is full when serviced. - Jonathan Morton ___ Cerowrt-devel mailing list Cerowrt-devel@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cerowrt-devel
Re: [Cerowrt-devel] Any chance for the LUCI GUI for cake appearing?
Hi Dave, hi List, On Jun 26, 2015, at 21:48 , Dave Taht dave.t...@gmail.com wrote: It turns out that I am pulling from openwrt for luci-app-sqm, not ceropackages, and overriding it was a headache. Fixed now, see below I guess Toke and I need to find a better way of updating the relevant repository… I had thought that ceropackages-3.0 was a relatively confined and hence “safe” testing group, so my idea was if it does worl for the ceopackages crowd, push it onto the openwrt community; but it seems that step is too large an impedance mismatch. So I will try to find a way with Toke to centralize development a bit. That scares me a bit as, knowing my level of detail, there will be lots of fix-up work; on the other hand, changes without people to test them is somewhat lame as well... router# cd /tmp; wget http://snapon.lab.bufferbloat.net/~cero3/luci-app-sqm_3-5_all.ipk ; opkg install ./luci-app-sqm_3-5_all.ipk That seems recent. I need to remember to bump so,e number on any relevant change. My target is to change on ever bug fixes and feature additions, but I am not in the best position to judge bug fixes truthfully... router # /etc/init.d/uhttpd restart if it breaks, you get to keep both pieces, but it worked for me, and yea! cake in the gui! thx sebastian! Yippi ya ya yippy yippy yeah. I tried - ./scripts feeds uninstall luci-app-sqm; ./scripts feeds install -p cero luci-app-sqm # seemed sane, did not work git rm in the feed itself did not work doing the uninstall, deleting the symlink, and the git rm, nuking the bin/ar71xx/packages/packages/luci-app-sqm* and doing a complete rebuild... eventually did work... but strikes me as a bit more brute force than I wanted. Fair enough, see the point above of Toke and me coming up with a better plan to get relevant changes into tester’s hands quicker... I am not in a position (aformentioned build is my specialized for pico/nano build at the moment) to update the packages db, but I stuck up the correct new package... which you can install via Apologies for the confusion, I simply figured that the -p option would work, and never checked. I also note that I think I am building things compile tuned for the archer mips34c, rather thanthe wndr3800´s 24c. However I kept the unaligned access hacks (not needed on the 34c derived archer, but needed on the 24c wndr3800) Looking at the readily available cpu information, MIPS seems to be severely EOL’d. Mikael’s result with the armada 385 seem way more future proof (even without BQL) Sigh. maintaining 3 separate trees for the platforms we sort of support is a PITA. I am happy to buy a new router to be back on board with your development, but the mips 34c seems a bit underwhelming for the goal of NAT-shapimg roughly 300 Mbps bi-directionally. The edgerouterX’s numbers also do not look too hot; Mikeal’s numbers really made made me reconsider my plan to get another HP priloant microsrver G8 to do duty as PPPoE gateway router and traffic shaper. (The G8 retails fr 240 EUR with 2.3 GHz and 2GB ram, so 200 EUR for a wrt1200ac definitely looks like too much, even though the G* does not do wifi, but I still have the wbdr 3700v2 to serve wireless clients...) Then adding in something that can mesh by default on 2 others. and then trying to get anything to compile for another arch entirely and failing… As much as I would like to participate in that fun, a 1 router household is just not interesting enough I guess ) Best Regards Sebastian On Fri, Jun 26, 2015 at 11:49 AM, Sebastian Moeller moell...@gmx.de wrote: Hi Jim, On Jun 26, 2015, at 19:21 , Jim Gettys j...@freedesktop.org wrote: Inquiring minds would like to know…. I guess, that depends on your definition of “LUCI GUI for cake”; as far as I know you can select cake in the “Queueing Discipline” drop down in the “Queue Discipline” tab of luci-app-sqm. So if the machine has the cake module and a sufficiently recent (out-of-tree) tc binary installed it should just work. Using the “Advanced option strings…” it even should be possible to pass arbitrary strings to cake (without any error checking), so I assume we are all set up for cake testing (or should that be cake tasting?) from the LUCI GUI. But I have not managed to build cake locally, nor have I dared to risk my families internet connectivity by trying one of Dave’s newer openwrt trunk with cake” builds. So these changes linger in the ceropackages-3.10 repository until we get enough tasting to convince Toke to pull them into the openwrt repositories (these changes affect both sqm-scripts and luci-app-sqm). Dave graciously allowed me access to one of his test machines, and the changes allow to set up cake from the GUI, but I only did limited functionality testing. (And we discovered that something was of in that cake worked but
Re: [Cerowrt-devel] performance numbers from WRT1200AC (Re: Latest build test - new sqm-scripts seem to work; cake overhead 40 didn't)
On Fri, Jun 26, 2015 at 11:38 AM, Mikael Abrahamsson swm...@swm.pp.se wrote: On Fri, 26 Jun 2015, Dave Taht wrote: Mikael, a simple test of the analysis I just did would be to use ethtool to set your server to 100mbits (ethtool -s your_ethernet_device advertise 0x008 and turn on fq_codel on both the client and server. Hm what do you mean by client and server? Where do you want the queueing to happen? Egress from the WRT1200AC towards the server? Yes. Then setting the WAN port of the WRT1200AC to 100 megabit/s would work, yes. Yes, but I am unsure from looking at the driver that using ethtool on the egress on the wrt1200ac will actually work, but pretty sure it will work if you set it on the server. feel free to try both. :) -- Mikael Abrahamssonemail: swm...@swm.pp.se -- Dave Täht worldwide bufferbloat report: http://www.dslreports.com/speedtest/results/bufferbloat And: What will it take to vastly improve wifi for everyone? https://plus.google.com/u/0/explore/makewififast ___ Cerowrt-devel mailing list Cerowrt-devel@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cerowrt-devel
Re: [Cerowrt-devel] performance numbers from WRT1200AC (Re: Latest build test - new sqm-scripts seem to work; cake overhead 40 didn't)
On Fri, 26 Jun 2015, Dave Taht wrote: On Fri, Jun 26, 2015 at 11:58 AM, Dave Taht dave.t...@gmail.com wrote: On Fri, Jun 26, 2015 at 11:38 AM, Mikael Abrahamsson swm...@swm.pp.se wrote: On Fri, 26 Jun 2015, Dave Taht wrote: Mikael, a simple test of the analysis I just did would be to use ethtool to set your server to 100mbits (ethtool -s your_ethernet_device advertise 0x008 and turn on fq_codel on both the client and server. Hm what do you mean by client and server? your topology is: client box - router - server forcing the router - server link to 100mbit will push the egress buffering into the router for the rrul_50_up test in particular. No, my topology is client - router - switch - server, that is what made me confused. So if I forced the eth0 router-switch link into 100M I will break the server-client direction (it'll be shallow buffer fifo) but at least client-server direction will exercise fq_codel on the router. -- Mikael Abrahamssonemail: swm...@swm.pp.se ___ Cerowrt-devel mailing list Cerowrt-devel@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cerowrt-devel
Re: [Cerowrt-devel] performance numbers from WRT1200AC (Re: Latest build test - new sqm-scripts seem to work; cake overhead 40 didn't)
On Fri, Jun 26, 2015 at 11:58 AM, Dave Taht dave.t...@gmail.com wrote: On Fri, Jun 26, 2015 at 11:38 AM, Mikael Abrahamsson swm...@swm.pp.se wrote: On Fri, 26 Jun 2015, Dave Taht wrote: Mikael, a simple test of the analysis I just did would be to use ethtool to set your server to 100mbits (ethtool -s your_ethernet_device advertise 0x008 and turn on fq_codel on both the client and server. Hm what do you mean by client and server? your topology is: client box - router - server forcing the router - server link to 100mbit will push the egress buffering into the router for the rrul_50_up test in particular. Where do you want the queueing to happen? Egress from the WRT1200AC towards the server? Yes. Then setting the WAN port of the WRT1200AC to 100 megabit/s would work, yes. Yes, but I am unsure from looking at the driver that using ethtool on the egress on the wrt1200ac will actually work, but pretty sure it will work if you set it on the server. feel free to try both. :) -- Mikael Abrahamssonemail: swm...@swm.pp.se -- Dave Täht worldwide bufferbloat report: http://www.dslreports.com/speedtest/results/bufferbloat And: What will it take to vastly improve wifi for everyone? https://plus.google.com/u/0/explore/makewififast -- Dave Täht worldwide bufferbloat report: http://www.dslreports.com/speedtest/results/bufferbloat And: What will it take to vastly improve wifi for everyone? https://plus.google.com/u/0/explore/makewififast ___ Cerowrt-devel mailing list Cerowrt-devel@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cerowrt-devel
Re: [Cerowrt-devel] performance numbers from WRT1200AC (Re: Latest build test - new sqm-scripts seem to work; cake overhead 40 didn't)
On Fri, Jun 26, 2015 at 12:11 PM, Mikael Abrahamsson swm...@swm.pp.se wrote: On Fri, 26 Jun 2015, Dave Taht wrote: On Fri, Jun 26, 2015 at 11:58 AM, Dave Taht dave.t...@gmail.com wrote: On Fri, Jun 26, 2015 at 11:38 AM, Mikael Abrahamsson swm...@swm.pp.se wrote: On Fri, 26 Jun 2015, Dave Taht wrote: Mikael, a simple test of the analysis I just did would be to use ethtool to set your server to 100mbits (ethtool -s your_ethernet_device advertise 0x008 and turn on fq_codel on both the client and server. Hm what do you mean by client and server? your topology is: client box - router - server forcing the router - server link to 100mbit will push the egress buffering into the router for the rrul_50_up test in particular. No, my topology is client - router - switch - server, that is what made me confused. So if I forced the eth0 router-switch link into 100M I will break the server-client direction (it'll be shallow buffer fifo) but at least client-server direction will exercise fq_codel on the router. well, maybe the driver will take ethtool on the mvneta. try it. :) -- Mikael Abrahamssonemail: swm...@swm.pp.se -- Dave Täht worldwide bufferbloat report: http://www.dslreports.com/speedtest/results/bufferbloat And: What will it take to vastly improve wifi for everyone? https://plus.google.com/u/0/explore/makewififast ___ Cerowrt-devel mailing list Cerowrt-devel@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cerowrt-devel
Re: [Cerowrt-devel] Any chance for the LUCI GUI for cake appearing?
It turns out that I am pulling from openwrt for luci-app-sqm, not ceropackages, and overriding it was a headache. Fixed now, see below router# cd /tmp; wget http://snapon.lab.bufferbloat.net/~cero3/luci-app-sqm_3-5_all.ipk ; opkg install ./luci-app-sqm_3-5_all.ipk router # /etc/init.d/uhttpd restart if it breaks, you get to keep both pieces, but it worked for me, and yea! cake in the gui! thx sebastian! I tried - ./scripts feeds uninstall luci-app-sqm; ./scripts feeds install -p cero luci-app-sqm # seemed sane, did not work git rm in the feed itself did not work doing the uninstall, deleting the symlink, and the git rm, nuking the bin/ar71xx/packages/packages/luci-app-sqm* and doing a complete rebuild... eventually did work... but strikes me as a bit more brute force than I wanted. I am not in a position (aformentioned build is my specialized for pico/nano build at the moment) to update the packages db, but I stuck up the correct new package... which you can install via Apologies for the confusion, I simply figured that the -p option would work, and never checked. I also note that I think I am building things compile tuned for the archer mips34c, rather thanthe wndr3800´s 24c. However I kept the unaligned access hacks (not needed on the 34c derived archer, but needed on the 24c wndr3800) Sigh. maintaining 3 separate trees for the platforms we sort of support is a PITA. Then adding in something that can mesh by default on 2 others. and then trying to get anything to compile for another arch entirely and failing... On Fri, Jun 26, 2015 at 11:49 AM, Sebastian Moeller moell...@gmx.de wrote: Hi Jim, On Jun 26, 2015, at 19:21 , Jim Gettys j...@freedesktop.org wrote: Inquiring minds would like to know…. I guess, that depends on your definition of “LUCI GUI for cake”; as far as I know you can select cake in the “Queueing Discipline” drop down in the “Queue Discipline” tab of luci-app-sqm. So if the machine has the cake module and a sufficiently recent (out-of-tree) tc binary installed it should just work. Using the “Advanced option strings…” it even should be possible to pass arbitrary strings to cake (without any error checking), so I assume we are all set up for cake testing (or should that be cake tasting?) from the LUCI GUI. But I have not managed to build cake locally, nor have I dared to risk my families internet connectivity by trying one of Dave’s newer openwrt trunk with cake” builds. So these changes linger in the ceropackages-3.10 repository until we get enough tasting to convince Toke to pull them into the openwrt repositories (these changes affect both sqm-scripts and luci-app-sqm). Dave graciously allowed me access to one of his test machines, and the changes allow to set up cake from the GUI, but I only did limited functionality testing. (And we discovered that something was of in that cake worked but did not accept all the keywords it should have, but unless you are on a XDSL-Link you most likely do not care about specifying per-packet-overhead) If you have any specific thing you are missing, let me know and I will try to help. Best Regards Sebastian - Jim ___ Cerowrt-devel mailing list Cerowrt-devel@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cerowrt-devel ___ Cerowrt-devel mailing list Cerowrt-devel@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cerowrt-devel -- Dave Täht worldwide bufferbloat report: http://www.dslreports.com/speedtest/results/bufferbloat And: What will it take to vastly improve wifi for everyone? https://plus.google.com/u/0/explore/makewififast ___ Cerowrt-devel mailing list Cerowrt-devel@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cerowrt-devel
Re: [Cerowrt-devel] performance numbers from WRT1200AC (Re: Latest build test - new sqm-scripts seem to work; cake overhead 40 didn't)
Hi Mikael, thanks a lot. On Jun 24, 2015, at 13:31 , Mikael Abrahamsson swm...@swm.pp.se wrote: On Tue, 23 Jun 2015, Sebastian Moeller wrote: As Dave said it would be nice see RRUL data from the same testbed. It would be so nice if flint had a way to send different sized TCP packets… (I guess this might be faked with MSS clamping in the router and relaying on path MTU discovery?) What kind of tests should I run? I have rrul results now, both at the same time as running iperf3. This is interesting. I also like the way iperf3 reports retransmits (BTW way how does it know this, I thought the kernel does the retransmits under the hood out of sight of applications). Since RRUL does not have such a nice report as iperf3 I will not be able to calculate the difference to the theoretical maximum transfer rates (also I lack insight on the ACK traffic). I set iperf3 to run with 10 parallel streams, different MSS per test, and let it run for 30 seconds into the rrul test. The plots are interesting, even though I have a hard time with the logarithmic scale. So in all the tests, iperf3 session stops running half way into the rrul test. I like how the RRUL flows soak up the newly available bandwidth. I set sqm to 500M up and down on eth0, ECN up and down, and not to squash DSCP in any direction. I will have a look at the flint files, but it looks like the 1200ac does 500Mbps bidirectionally with SQM, so this looks like a decent machine for the present and (near?) future. Now I just need to wait until prices come down a tad (I hope that the 1900AC v2 release should drop the 1200ac’s process a bit). Thanks for the tests, now I know what router to try next (the edgerouterX, which I had eyed as a replacement for the shaper in the wndr3700 tops out at 130K packets per second and hence will not really work that well for a 100/40 Mbps link). Best Regards Sebastian http://swm.pp.se/aqm/rrul-wrt1200ac-150624.pdf Tell me if there is more information I can provide or tests to run. -- Mikael Abrahamssonemail: swm...@swm.pp.se ___ Cerowrt-devel mailing list Cerowrt-devel@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cerowrt-devel
Re: [Cerowrt-devel] that latest build I did is looking good
Hi Dave, On Jun 26, 2015, at 00:26 , Dave Taht dave.t...@gmail.com wrote: On Thu, Jun 25, 2015 at 1:15 PM, Sebastian Moeller moell...@gmx.de wrote: Hi Dave, this looks really great, thanks a lot. As I want to take part in the cake party, this looks like the easiest/best way to start. Gui support for cake would be nice. TIA. :) What is missing? As far as I know the latest luci-app-sqm in ceropackages-3.10 should work well with cake, just select cake as qdisc and you should be set. (I want to move cake as a shaper into its own cake.qos script, but still implement the possibility to use cake as leaf qdisc (which might already work out of the box). Currently selecting cake as qdisc will use cake as shaper automatically, which might be a bit surprising). If that does not work let me know, I think I tried on the test machine you gave me access to. The issues were that cake was somehow not accepting the overhead parameter, but it should still work to select cake and have it take over… NOTE these changes are not in the openwrt packages repository yet, as I want some positive testing results before asking Toke to pull them. Currently they seem to not break stuff, so I does not look like they will cause a regression, and hence might be fine to pull just to increase the circle of potential testers/bug-reporters... Best Regards Sebastian In other news I am loving having an easy to parse set of rc_stats_csv to play with: http://pastebin.com/05nZcmVu Not that I can remember what all the fields mean. On Jun 25, 2015, at 19:34 , Dave Taht dave.t...@gmail.com wrote: I have been abusing it on a picostation and nanostation now for 48 hours. The archer c7v2 (as a source specific gateway) for a week. A couple wndr3800s. No crashes. Can still trigger the dreaded wifi TX DMA bug, but it seems harder now. DID regularly crash the iwl in one box. [3] My mission was to get to something that I could deploy here at a small scale and just let run for a month while away in the eu, and that was looking dicy there for a while. (I am glad to have basically started in april!) So... we do, finally, have an openwrt build that uses cake, has the minstrel-blues patches, and andrews minimum variance patches, working dnssec (we hope!), and a new version of babel with ecn enabled, has snmp, that does dhcp-pd fairly right, and works with comcast. I also have things (odhcp6c is way better than isc, dibbler, wide) working fairly well with another debian based firewall. If/when new cake or sqm stuff arrives my plan (barring other major bugs elsewhere) is to just incrementally build that and tc-adv out of the above frozen repo. [1] Question: sqm is from the openwrt repository or from ceropackages-3.10? I ask as the latest changes are only in ceropackages 3.10, since they certainly require testing before being inflicted on the openwrt crowd… Best Regards Sebastian -- Dave Täht worldwide bufferbloat report: http://www.dslreports.com/speedtest/results/bufferbloat And: What will it take to vastly improve wifi for everyone? https://plus.google.com/u/0/explore/makewififast ___ Cerowrt-devel mailing list Cerowrt-devel@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cerowrt-devel -- Dave Täht worldwide bufferbloat report: http://www.dslreports.com/speedtest/results/bufferbloat And: What will it take to vastly improve wifi for everyone? https://plus.google.com/u/0/explore/makewififast ___ Cerowrt-devel mailing list Cerowrt-devel@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cerowrt-devel
Re: [Cerowrt-devel] Any chance for the LUCI GUI for cake appearing?
Honestly, you cannot draw that conclusion on cake yet. The new firmware forwarded at something like 35% better rates than the old.[1] So if you can show a dsireports speedtest on your hardware, on fq_codel, also... that would be confirmation we are winning on cake on your hardware. [1] I have great hope that the new FIB stuff in 4.0 and later will also do magic for forwarding rates. On Fri, Jun 26, 2015 at 4:36 PM, Jim Gettys j...@freedesktop.org wrote: Thank you all for cake... One of my recent pains had been that Comcast doubled my download speed to 100mbps, and CeroWrt using fq_codel/htb couldn't hack anywhere near that speed. Now I have things set using cake, and can do much better, and have my cake and eat it too. http://www.dslreports.com/speedtest/749786 - Jim On Fri, Jun 26, 2015 at 5:04 PM, Sebastian Moeller moell...@gmx.de wrote: On Jun 26, 2015, at 21:48 , Dave Taht dave.t...@gmail.com wrote: It turns out that I am pulling from openwrt for luci-app-sqm, not ceropackages, and overriding it was a headache. Fixed now, see below router# cd /tmp; wget http://snapon.lab.bufferbloat.net/~cero3/luci-app-sqm_3-5_all.ipk ; opkg install ./luci-app-sqm_3-5_all.ipk router # /etc/init.d/uhttpd restart if it breaks, you get to keep both pieces, but it worked for me, and yea! cake in the gui! thx sebastian! Too cite Shaggy “Wasn’t me” ;); the initial commit was not mine and I do not want to claim otherwise. I basically just rearranged the deck-chairs as I tend to think. I seem to be better at the “ love for details” thing than the”big picture”, anyways Best Regards Sebastian I tried - ./scripts feeds uninstall luci-app-sqm; ./scripts feeds install -p cero luci-app-sqm # seemed sane, did not work git rm in the feed itself did not work doing the uninstall, deleting the symlink, and the git rm, nuking the bin/ar71xx/packages/packages/luci-app-sqm* and doing a complete rebuild... eventually did work... but strikes me as a bit more brute force than I wanted. I am not in a position (aformentioned build is my specialized for pico/nano build at the moment) to update the packages db, but I stuck up the correct new package... which you can install via Apologies for the confusion, I simply figured that the -p option would work, and never checked. I also note that I think I am building things compile tuned for the archer mips34c, rather thanthe wndr3800´s 24c. However I kept the unaligned access hacks (not needed on the 34c derived archer, but needed on the 24c wndr3800) Sigh. maintaining 3 separate trees for the platforms we sort of support is a PITA. Then adding in something that can mesh by default on 2 others. and then trying to get anything to compile for another arch entirely and failing... On Fri, Jun 26, 2015 at 11:49 AM, Sebastian Moeller moell...@gmx.de wrote: Hi Jim, On Jun 26, 2015, at 19:21 , Jim Gettys j...@freedesktop.org wrote: Inquiring minds would like to know…. I guess, that depends on your definition of “LUCI GUI for cake”; as far as I know you can select cake in the “Queueing Discipline” drop down in the “Queue Discipline” tab of luci-app-sqm. So if the machine has the cake module and a sufficiently recent (out-of-tree) tc binary installed it should just work. Using the “Advanced option strings…” it even should be possible to pass arbitrary strings to cake (without any error checking), so I assume we are all set up for cake testing (or should that be cake tasting?) from the LUCI GUI. But I have not managed to build cake locally, nor have I dared to risk my families internet connectivity by trying one of Dave’s newer openwrt trunk with cake” builds. So these changes linger in the ceropackages-3.10 repository until we get enough tasting to convince Toke to pull them into the openwrt repositories (these changes affect both sqm-scripts and luci-app-sqm). Dave graciously allowed me access to one of his test machines, and the changes allow to set up cake from the GUI, but I only did limited functionality testing. (And we discovered that something was of in that cake worked but did not accept all the keywords it should have, but unless you are on a XDSL-Link you most likely do not care about specifying per-packet-overhead) If you have any specific thing you are missing, let me know and I will try to help. Best Regards Sebastian - Jim ___ Cerowrt-devel mailing list Cerowrt-devel@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cerowrt-devel ___ Cerowrt-devel mailing list Cerowrt-devel@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cerowrt-devel -- Dave Täht worldwide bufferbloat
Re: [Cerowrt-devel] Any chance for the LUCI GUI for cake appearing?
Thank you all for cake... One of my recent pains had been that Comcast doubled my download speed to 100mbps, and CeroWrt using fq_codel/htb couldn't hack anywhere near that speed. Now I have things set using cake, and can do much better, and have my cake and eat it too. http://www.dslreports.com/speedtest/749786 - Jim On Fri, Jun 26, 2015 at 5:04 PM, Sebastian Moeller moell...@gmx.de wrote: On Jun 26, 2015, at 21:48 , Dave Taht dave.t...@gmail.com wrote: It turns out that I am pulling from openwrt for luci-app-sqm, not ceropackages, and overriding it was a headache. Fixed now, see below router# cd /tmp; wget http://snapon.lab.bufferbloat.net/~cero3/luci-app-sqm_3-5_all.ipk ; opkg install ./luci-app-sqm_3-5_all.ipk router # /etc/init.d/uhttpd restart if it breaks, you get to keep both pieces, but it worked for me, and yea! cake in the gui! thx sebastian! Too cite Shaggy “Wasn’t me” ;); the initial commit was not mine and I do not want to claim otherwise. I basically just rearranged the deck-chairs as I tend to think. I seem to be better at the “ love for details” thing than the”big picture”, anyways Best Regards Sebastian I tried - ./scripts feeds uninstall luci-app-sqm; ./scripts feeds install -p cero luci-app-sqm # seemed sane, did not work git rm in the feed itself did not work doing the uninstall, deleting the symlink, and the git rm, nuking the bin/ar71xx/packages/packages/luci-app-sqm* and doing a complete rebuild... eventually did work... but strikes me as a bit more brute force than I wanted. I am not in a position (aformentioned build is my specialized for pico/nano build at the moment) to update the packages db, but I stuck up the correct new package... which you can install via Apologies for the confusion, I simply figured that the -p option would work, and never checked. I also note that I think I am building things compile tuned for the archer mips34c, rather thanthe wndr3800´s 24c. However I kept the unaligned access hacks (not needed on the 34c derived archer, but needed on the 24c wndr3800) Sigh. maintaining 3 separate trees for the platforms we sort of support is a PITA. Then adding in something that can mesh by default on 2 others. and then trying to get anything to compile for another arch entirely and failing... On Fri, Jun 26, 2015 at 11:49 AM, Sebastian Moeller moell...@gmx.de wrote: Hi Jim, On Jun 26, 2015, at 19:21 , Jim Gettys j...@freedesktop.org wrote: Inquiring minds would like to know…. I guess, that depends on your definition of “LUCI GUI for cake”; as far as I know you can select cake in the “Queueing Discipline” drop down in the “Queue Discipline” tab of luci-app-sqm. So if the machine has the cake module and a sufficiently recent (out-of-tree) tc binary installed it should just work. Using the “Advanced option strings…” it even should be possible to pass arbitrary strings to cake (without any error checking), so I assume we are all set up for cake testing (or should that be cake tasting?) from the LUCI GUI. But I have not managed to build cake locally, nor have I dared to risk my families internet connectivity by trying one of Dave’s newer openwrt trunk with cake” builds. So these changes linger in the ceropackages-3.10 repository until we get enough tasting to convince Toke to pull them into the openwrt repositories (these changes affect both sqm-scripts and luci-app-sqm). Dave graciously allowed me access to one of his test machines, and the changes allow to set up cake from the GUI, but I only did limited functionality testing. (And we discovered that something was of in that cake worked but did not accept all the keywords it should have, but unless you are on a XDSL-Link you most likely do not care about specifying per-packet-overhead) If you have any specific thing you are missing, let me know and I will try to help. Best Regards Sebastian - Jim ___ Cerowrt-devel mailing list Cerowrt-devel@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cerowrt-devel ___ Cerowrt-devel mailing list Cerowrt-devel@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cerowrt-devel -- Dave Täht worldwide bufferbloat report: http://www.dslreports.com/speedtest/results/bufferbloat And: What will it take to vastly improve wifi for everyone? https://plus.google.com/u/0/explore/makewififast ___ Cerowrt-devel mailing list Cerowrt-devel@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cerowrt-devel
Re: [Cerowrt-devel] performance numbers from WRT1200AC (Re: Latest build test - new sqm-scripts seem to work; cake overhead 40 didn't)
On Fri, 26 Jun 2015, Dave Taht wrote: Yes, but I am unsure from looking at the driver that using ethtool on the egress on the wrt1200ac will actually work, but pretty sure it will work if you set it on the server. feel free to try both. :) I set speed 100 on my switch and did some new tests, I left SQM at 500M, don't know what happens then, thought it was worthwile to test? Remember, now I don't have fq_codel in the server-client direction, but there it's an low buffered switchport that is doing the rate adaptation. Btw, now I am running a nightly build that I compiled for myself for the WRT1200AC, so if there is anything you would like me to try to change in the mvneta driver, I can certainly test that. I can do serial console access to the WRT1200AC if needed as well, I already unbricked it once. Btw, disregard the title in the flent files, I just realised I forgot to change the title argument. This is without iperf3, with SQM set to 500M, but eth0 at 100megabit/full duplex. http://swm.pp.se/aqm/wrt1200ac-150627-100m-sqm.tar -- Mikael Abrahamssonemail: swm...@swm.pp.se ___ Cerowrt-devel mailing list Cerowrt-devel@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cerowrt-devel
Re: [Cerowrt-devel] performance numbers from WRT1200AC (Re: Latest build test - new sqm-scripts seem to work; cake overhead 40 didn't)
your results are showing basically tail drop behavior. Although I would have expected intrinsic delay on the link to crack 100mbits on the rrul test, not 20ms (which is still high), and you only hit 7 on the single threaded tcp up test, based on what I saw in the driver. turn off sqm, stay at 100mbit, let fq_codel ride, try the rrul_50_up test. Also do a capture to see if CE was ever exerted. I do not know why my linksys ac1200 build does not work. I generally suspect it is because my big build server is getting ancient. Can you send me your .config file for openwrt? It would be VERY helpful if you pulled from ceropackages-3.14, and added and built kmod-sched-cake and tc-adv and switched to using the ceropackages feed also for luci-app-sqm and sqm-scripts. There are a couple people here that would probably leap on that! :) add to your feeds.conf src-git https://github.com/dtaht/ceropackages-3.10.git ./scripts feeds update ./scripts feeds install kmod-sched-cake tc-adv kmod-sched-fq_pie edit the .config file to make them modules or installed by default make menuconfig then save make I went through hell trying to to figure out how to switch over to the cero versions of these. If yer doing the builds, I could try hacking the BQL. Maybe. In sweden. On Fri, Jun 26, 2015 at 10:03 PM, Mikael Abrahamsson swm...@swm.pp.se wrote: On Fri, 26 Jun 2015, Dave Taht wrote: Yes, but I am unsure from looking at the driver that using ethtool on the egress on the wrt1200ac will actually work, but pretty sure it will work if you set it on the server. feel free to try both. :) I set speed 100 on my switch and did some new tests, I left SQM at 500M, don't know what happens then, thought it was worthwile to test? Remember, now I don't have fq_codel in the server-client direction, but there it's an low buffered switchport that is doing the rate adaptation. Btw, now I am running a nightly build that I compiled for myself for the WRT1200AC, so if there is anything you would like me to try to change in the mvneta driver, I can certainly test that. I can do serial console access to the WRT1200AC if needed as well, I already unbricked it once. Btw, disregard the title in the flent files, I just realised I forgot to change the title argument. This is without iperf3, with SQM set to 500M, but eth0 at 100megabit/full duplex. http://swm.pp.se/aqm/wrt1200ac-150627-100m-sqm.tar -- Mikael Abrahamssonemail: swm...@swm.pp.se -- Dave Täht worldwide bufferbloat report: http://www.dslreports.com/speedtest/results/bufferbloat And: What will it take to vastly improve wifi for everyone? https://plus.google.com/u/0/explore/makewififast ___ Cerowrt-devel mailing list Cerowrt-devel@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cerowrt-devel
Re: [Cerowrt-devel] performance numbers from WRT1200AC (Re: Latest build test - new sqm-scripts seem to work; cake overhead 40 didn't)
On Fri, 26 Jun 2015, Sebastian Moeller wrote: Thanks for the tests, now I know what router to try next (the edgerouterX, which I had eyed as a replacement for the shaper in the wndr3700 tops out at 130K packets per second and hence will not really work that well for a 100/40 Mbps link). I did some more tests and it seems with SQM at 500 megabit/s I start to lose packets at around 250-300k PPS. At these speeds, the rrul_be test is only 85k PPS at 500 megabit/s bidirectional with large packets. Also, I have an Edgerouter ER-5, but as soon as it does CPU based forwarding it's really weak, easily under 100 megabit/s even with large packets. OpenVPN without encryption is less than 20 megabit/s. Btw, the WRT1200AC is now becoming more widely available and it's 150 EUR incl 25% VAT and shipping here in Sweden now. Btw, I tried WNDR3800 setting it to 100/100 SQM. It seems to max out around 25-30k PPS, but the difference is that when the CPU is full, it seems to delay/ECN-mark packets because there are no packets lost. When the WRT1200AC runs out of CPU it starts dropping packets. I always have 0 packets lost with the WNDR3800 when doing iperf3 testing. I found this difference interesting, wonder where in the forwarding path the WRT1200AC loses packets? -- Mikael Abrahamssonemail: swm...@swm.pp.se ___ Cerowrt-devel mailing list Cerowrt-devel@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cerowrt-devel
