Re: [Codel] Using fq_codel with a WiFi uplink to the Internet

[Phineas Gage]

> On Sep 23, 2016, at 1:54 PM, Phineas Gage  wrote:
> 
>>> Option A: We can choose a maximum of 40/40 Mbit with 1:5 aggregation, 
>>> meaning we could get 40 Mbit, but we could also get a lot less at times (8 
>>> Mbit I assume), depending on other network load.
>> 
>> The 1:5 aggregation is worth exploring a bit more deeply.  Usually this 
>> refers to the ratio between the total bandwidth provisioned across all 
>> customers (in some region and some service class) and the minimum backhaul 
>> capacity within and at the far edge of the ISP’s network.  The theory is 
>> that customers tend not to use all of their bandwidth all of the time, 
>> though they do tend to use all of it some of the time.  This theory works 
>> fairly well in practice as long as the traffic patterns are not too 
>> correlated and are distributed over a sufficient number of customers.
>> 
>> In order to be dragged down to 8Mbps, you would have to see all the other 
>> customers sharing your backhaul trying to use 8Mbps or more (on average) at 
>> the same time.  This is unlikely to occur often; consider major sportsball 
>> championship final matches, or national disasters such as one we recently 
>> had an anniversary of, as the most likely triggers.  Under such 
>> circumstances, you’ll need to step in and manually experiment to find out 
>> what works, but shaping at 8Mbps would be a reasonable default reaction.
>> 
>> Of more concern to you is how much external load is needed to pull your 
>> share of the backhaul significantly below 40Mbps - or, alternatively, below 
>> the 20Mbps you can get with the more expensive uncontended service.  This 
>> will vary depending on how many customers the 5:1 average is spread over - 
>> you can’t actually calculate it from the information given.
>> 
>> So the question is whether they have a 40/40 backhaul shared among 5 
>> customers, or a 1G/1G backhaul which they plan to share among up to 125 
>> customers, each with 40/40 service.  I think the latter is a perfectly 
>> reasonable possibility, and would be much more robust than the former option 
>> which would give you a reduction in throughput as soon as *any* of the other 
>> customers on the same backhaul segment did *anything*.
>> 
>> It’s a question worth asking your WISP.  Be ready to point out that you 
>> don’t plan to actually *use* 40Mbps full-time, but that your AQM solution 
>> depends on knowing how much bandwidth is available for when *your* customers 
>> randomly demand it.
> 
> Thanks a lot for this great information, I’ll ask them more about what’s 
> behind their aggregation ratio. As I mentioned in one reply earlier, I may 
> ask for flexibility in our contract, try the aggregated service, then switch 
> to guaranteed service depending on the actual performance of their network. 
> If I can rate limit to 30/30 and fq_codel, and it’s 99% effective, I’d rather 
> take that than settle for a guaranteed 20/20 on-season and 6/6 off-season to 
> get to the same annual contract price.

Following up on the WISP results, I tested the 40 Mbps non-guaranteed 
connection for a week or so, and the truth is that the speed varies much more 
than expected. Although I once saw a 40 Mbps download, that’s more of a black 
swan. Download throughput during Internet rush hour regularly dropped to 5-10 
Mbps, and I once saw 1 Mbps. Upload throughput stays higher. But I can’t see 
how “aggregation 1:5” has any meaning for me, as I saw speeds below 8 Mbps. 
There is probably no way I can effectively control either the upload or 
download queues effectively with this type of service.

Also, they seem to prioritize traffic to speedtest.net, because while the 
graphs to those servers looked pretty good and consistent (I scripted a test 
every 30 minutes using speedtest-cli), throughput reported by using wget to 
various high-powered Ubuntu servers around us in Europe told a different story. 
During the day, 25-30 Mbps was routine, and in the evening, 5-10 Mbps was 
routine, with outliers on either end. This was of course with a cabled client 
and no other traffic from us on the connection. So I’ll be finding out more 
details about the “guaranteed” 10 Mbps service.

Also a couple other comments to anyone interested:

- I saw a post about recent work getting Cake to run on EdgeOS, so I expressed 
interest in that for the EdgeRouter X, which I installed yesterday: 
http://community.ubnt.com/t5/EdgeMAX/Cake-compiled-for-the-ERL/td-p/1679844

- I might make it to the OpenWrt summit (http://openwrtsummit.org/) in Berlin 
next Thursday, 10/13, in case I catch anyone there.

___
Codel mailing list
Codel@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/codel

Re: [Codel] Using fq_codel with a WiFi uplink to the Internet

[Toke Høiland-Jørgensen]

Phineas Gage  writes:

>  On Sep 23, 2016, at 6:31 PM, Toke Høiland-Jørgensen  wrote:
>
>  Phineas Gage  writes:
>
>  On Sep 21, 2016, at 12:32 PM, Dave Taht  wrote:
>  On Wed, Sep 21, 2016 at 2:59 AM, Phineas Gage  wrote:
>
>  Do I have any chance of running fq_codel in the driver on a Mikrotik
>  911-5HnD (firmware 3.30) with Atheros AR9300? If so, I may be able to
>  test it. The camp will be off-season soon until next April for the
>  snowy Czech winter, so it’s a good time for testing, as I also test
>  our meshed OpenWRT APs.
>
>  Can it run LEDE (OpenWrt)? If so, all you need to do is upgrade to
>  current trunk, and you'll be using the FQ-CoDel'ed driver :)
>
> I don’t know for sure, but the specs are so close to this working
> board (https://wiki.openwrt.org/toh/mikrotik/rb91xg_5hpnd) that I bet
> so. Secondly, I have to find out if the ISP will allow it. They will
> probably be more likely to do so if the driver could run on RouterOS
> 6.34.3. I’m guessing that’s not a priority right now. :)

Wikipedia thinks that is based on Linux 3.3.5. Which is ancient. So no.
But in principle it could be done...

>  Q: Would it also be useful to have fq_codel running on our APs? They
>  are Open Mesh OM2P HS’s with "Atheros AR9341 rev 1” chips.
>
>  Most likely, yes. You may also want to include the patches that gives
>  you airtime fairness on those. Keeps slow stations from slowing everyone
>  else down. I have a git tree with those here:
>  https://kau.toke.dk/git/lede/ - it's slightly behind mainline LEDE, so
>  you may want to use that as a base. This is the critical file, in that
>  case:
>  
> https://kau.toke.dk/git/lede/tree/package/kernel/mac80211/patches/347-ath9k-Add-a-per-station-airtime-deficit-scheduler.patch
>
>  I could add it now using “tc", but any level lower than that would
>  require the driver support, obviously. My feeling is that the rate
>  limiting on my Linux bridge puts the queues “mostly” there, and not in
>  the APs or upstream devices.
>
>  Depends on your traffic patterns, of course. But yeah, if all your
>  clients share the same uplink and that has more bandwidth than the
>  AP-to-WiFi link, then that is where the bottleneck would be. But a
>  client with bad reception can end up with an effective rate as low as
>  6.5 Mbps, so not always.
>
> Well, if our uplink goes to 30 Mbps or more, I’ve got repeater nodes
> that connect to their gateways at around that rate and fluctuate, so
> we’re likely to be moving the bottleneck around the camp sometimes if
> our Internet rate goes up. And in this environment, I know for sure
> that there are clients connecting at rates well below 30 Mbps! If
> there were negative MCS indexes, we would be using those.

Indeed. We still have a few kinks to work out to get CoDel to work well
at all bandwidths. But it's not a huge showstopper; what we have now is
still tons better than before.

> Right now, the OpenWRT release we run on the APs comes from Open Mesh.
> Unless I can convince them to build a driver with this patch, I’ll
> have to build and flash my own OpenWRT and give up the use of their
> online dashboard, upgrades and support. This is possible
> (https://wiki.openwrt.org/toh/openmesh/om2p). Moreover, I’m more
> likely to be able to do this on our APs than our point-to-point
> Internet uplink devices, since those are owned by the ISP.

Yeah, I know for a fact that the patches work on those devices; had them
tested on a bunch :)

> Thanks so much for these pointers and your efforts. The airtime
> fairness patch also sounds fantastic. In the main season, there can be
> a lot of contention in our environment at times, like when it starts
> raining and everyone heads to their cabins to get online. I’d love to
> try this out and help you test, but will see if it will be feasible
> for us.

You're very welcome. And testing is always appreciated :)

-Toke
___
Codel mailing list
Codel@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/codel

Re: [Codel] Using fq_codel with a WiFi uplink to the Internet

[Phineas Gage]

> On Sep 23, 2016, at 6:31 PM, Toke Høiland-Jørgensen  wrote:
> 
> Phineas Gage > writes:
> 
>> On Sep 21, 2016, at 12:32 PM, Dave Taht  wrote:
>> On Wed, Sep 21, 2016 at 2:59 AM, Phineas Gage  wrote:
>> 
>> Do I have any chance of running fq_codel in the driver on a Mikrotik
>> 911-5HnD (firmware 3.30) with Atheros AR9300? If so, I may be able to
>> test it. The camp will be off-season soon until next April for the
>> snowy Czech winter, so it’s a good time for testing, as I also test
>> our meshed OpenWRT APs.
> 
> Can it run LEDE (OpenWrt)? If so, all you need to do is upgrade to
> current trunk, and you'll be using the FQ-CoDel'ed driver :)

I don’t know for sure, but the specs are so close to this working board 
(https://wiki.openwrt.org/toh/mikrotik/rb91xg_5hpnd 
) that I bet so. Secondly, 
I have to find out if the ISP will allow it. They will probably be more likely 
to do so if the driver could run on RouterOS 6.34.3. I’m guessing that’s not a 
priority right now. :)

>> Q: Would it also be useful to have fq_codel running on our APs? They
>> are Open Mesh OM2P HS’s with "Atheros AR9341 rev 1” chips.
> 
> Most likely, yes. You may also want to include the patches that gives
> you airtime fairness on those. Keeps slow stations from slowing everyone
> else down. I have a git tree with those here:
> https://kau.toke.dk/git/lede/  - it's slightly 
> behind mainline LEDE, so
> you may want to use that as a base. This is the critical file, in that
> case:
> https://kau.toke.dk/git/lede/tree/package/kernel/mac80211/patches/347-ath9k-Add-a-per-station-airtime-deficit-scheduler.patch
>  
> 
> 
>> I could add it now using “tc", but any level lower than that would
>> require the driver support, obviously. My feeling is that the rate
>> limiting on my Linux bridge puts the queues “mostly” there, and not in
>> the APs or upstream devices.
> 
> Depends on your traffic patterns, of course. But yeah, if all your
> clients share the same uplink and that has more bandwidth than the
> AP-to-WiFi link, then that is where the bottleneck would be. But a
> client with bad reception can end up with an effective rate as low as
> 6.5 Mbps, so not always.

Well, if our uplink goes to 30 Mbps or more, I’ve got repeater nodes that 
connect to their gateways at around that rate and fluctuate, so we’re likely to 
be moving the bottleneck around the camp sometimes if our Internet rate goes 
up. And in this environment, I know for sure that there are clients connecting 
at rates well below 30 Mbps! If there were negative MCS indexes, we would be 
using those.

Right now, the OpenWRT release we run on the APs comes from Open Mesh. Unless I 
can convince them to build a driver with this patch, I’ll have to build and 
flash my own OpenWRT and give up the use of their online dashboard, upgrades 
and support. This is possible (https://wiki.openwrt.org/toh/openmesh/om2p 
). Moreover, I’m more likely to be 
able to do this on our APs than our point-to-point Internet uplink devices, 
since those are owned by the ISP.

Thanks so much for these pointers and your efforts. The airtime fairness patch 
also sounds fantastic. In the main season, there can be a lot of contention in 
our environment at times, like when it starts raining and everyone heads to 
their cabins to get online. I’d love to try this out and help you test, but 
will see if it will be feasible for us.

___
Codel mailing list
Codel@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/codel

Re: [Codel] Using fq_codel with a WiFi uplink to the Internet

[Toke Høiland-Jørgensen]

Phineas Gage  writes:

>  On Sep 21, 2016, at 12:32 PM, Dave Taht  wrote:
>  On Wed, Sep 21, 2016 at 2:59 AM, Phineas Gage  wrote:
>
>  Question #1: Is it still effective to run fq_codel on our edge router when I
>  have a WiFi uplink to the Internet, instead of a cabled connection like
>  ADSL? And related to that, is a high quality point-to-point WiFi connection
>  indistinguishable from a cabled connection as far as fq_codel is concerned?
>
>  It has, until recent developments, been helpful but not as effective
>  as we'd like.
>
>  We have two sets of code in the process of being finalized that should
>  work better
>  for a reasonably fast wifi uplink.
>
>  blog.cerowrt.org/post/fq_codel_on_ath10k/
>
>  
> https://blog.tohojo.dk/2016/06/fixing-the-wifi-performance-anomaly-on-ath9k.html
>
> That looks grand. If I ever see it working, I think I'll be as
> emotional as the OP of the ath10k article. That is, having spent some
> time setting up an fq_codel bridge for our camp, and getting blank
> stares when I talk excitedly about what I’m doing. And yet if I turn
> fq_codel off, I hear pretty quickly, “what’s wrong with the Internet?”
>
> Do I have any chance of running fq_codel in the driver on a Mikrotik
> 911-5HnD (firmware 3.30) with Atheros AR9300? If so, I may be able to
> test it. The camp will be off-season soon until next April for the
> snowy Czech winter, so it’s a good time for testing, as I also test
> our meshed OpenWRT APs.

Can it run LEDE (OpenWrt)? If so, all you need to do is upgrade to
current trunk, and you'll be using the FQ-CoDel'ed driver :)

> Q: Would it also be useful to have fq_codel running on our APs? They
> are Open Mesh OM2P HS’s with "Atheros AR9341 rev 1” chips.

Most likely, yes. You may also want to include the patches that gives
you airtime fairness on those. Keeps slow stations from slowing everyone
else down. I have a git tree with those here:
https://kau.toke.dk/git/lede/ - it's slightly behind mainline LEDE, so
you may want to use that as a base. This is the critical file, in that
case:
https://kau.toke.dk/git/lede/tree/package/kernel/mac80211/patches/347-ath9k-Add-a-per-station-airtime-deficit-scheduler.patch

> I could add it now using “tc", but any level lower than that would
> require the driver support, obviously. My feeling is that the rate
> limiting on my Linux bridge puts the queues “mostly” there, and not in
> the APs or upstream devices.

Depends on your traffic patterns, of course. But yeah, if all your
clients share the same uplink and that has more bandwidth than the
AP-to-WiFi link, then that is where the bottleneck would be. But a
client with bad reception can end up with an effective rate as low as
6.5 Mbps, so not always.

-Toke
___
Codel mailing list
Codel@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/codel

Re: [Codel] Using fq_codel with a WiFi uplink to the Internet

[Phineas Gage]

> On Sep 21, 2016, at 1:38 PM, Jonathan Morton  wrote:
>> On 21 Sep, 2016, at 12:59, Phineas Gage  wrote:
>> 
>> Question #1: Is it still effective to run fq_codel on our edge router when I 
>> have a WiFi uplink to the Internet, instead of a cabled connection like 
>> ADSL? And related to that, is a high quality point-to-point WiFi connection 
>> indistinguishable from a cabled connection as far as fq_codel is concerned?
>> 
>> Question #2: Assuming the answer to Question #1 is an overall "yes", is it 
>> then better to have a guaranteed speed from the ISP, instead of having a 
>> variable but potentially higher speed, so that I can control the queue and 
>> have fq_codel and HTB prioritization work effectively?
> 
> This is actually a pretty interesting pair of questions.  We’ve just been 
> doing quite a lot of work related to integrating fq_codel (or something very 
> like it) into the Linux wifi stack, where it has more information about 
> aggregation and other things, and can therefore make smarter queuing 
> decisions.
> 
> The very best solution would be for your WISP to integrate this work into 
> their hardware at each end of the link.  It may take a little more time until 
> that can happen, as the code is very very fresh and still a little raw.
> 
> Until then, I think something like what you’re already doing should work 
> well.  Normally, fq_codel interacts badly with high-performsnce wifi because 
> it tends to distribute packets alternately to different stations, which tends 
> to prevent effective aggregation, but this is not a concern for a 
> point-to-point link where all your traffic goes to and from one station to 
> another.

So, for example, if one runs fq_codel on a regular AP accessed by multiple 
stations (using “tc”, above the driver level), this is not necessarily a good 
idea? And once the driver work is done and the queues are managed at a lower 
level, will this no longer be the case?

>> Option A: We can choose a maximum of 40/40 Mbit with 1:5 aggregation, 
>> meaning we could get 40 Mbit, but we could also get a lot less at times (8 
>> Mbit I assume), depending on other network load.
> 
> The 1:5 aggregation is worth exploring a bit more deeply.  Usually this 
> refers to the ratio between the total bandwidth provisioned across all 
> customers (in some region and some service class) and the minimum backhaul 
> capacity within and at the far edge of the ISP’s network.  The theory is that 
> customers tend not to use all of their bandwidth all of the time, though they 
> do tend to use all of it some of the time.  This theory works fairly well in 
> practice as long as the traffic patterns are not too correlated and are 
> distributed over a sufficient number of customers.
> 
> In order to be dragged down to 8Mbps, you would have to see all the other 
> customers sharing your backhaul trying to use 8Mbps or more (on average) at 
> the same time.  This is unlikely to occur often; consider major sportsball 
> championship final matches, or national disasters such as one we recently had 
> an anniversary of, as the most likely triggers.  Under such circumstances, 
> you’ll need to step in and manually experiment to find out what works, but 
> shaping at 8Mbps would be a reasonable default reaction.
> 
> Of more concern to you is how much external load is needed to pull your share 
> of the backhaul significantly below 40Mbps - or, alternatively, below the 
> 20Mbps you can get with the more expensive uncontended service.  This will 
> vary depending on how many customers the 5:1 average is spread over - you 
> can’t actually calculate it from the information given.
> 
> So the question is whether they have a 40/40 backhaul shared among 5 
> customers, or a 1G/1G backhaul which they plan to share among up to 125 
> customers, each with 40/40 service.  I think the latter is a perfectly 
> reasonable possibility, and would be much more robust than the former option 
> which would give you a reduction in throughput as soon as *any* of the other 
> customers on the same backhaul segment did *anything*.
> 
> It’s a question worth asking your WISP.  Be ready to point out that you don’t 
> plan to actually *use* 40Mbps full-time, but that your AQM solution depends 
> on knowing how much bandwidth is available for when *your* customers randomly 
> demand it.

Thanks a lot for this great information, I’ll ask them more about what’s behind 
their aggregation ratio. As I mentioned in one reply earlier, I may ask for 
flexibility in our contract, try the aggregated service, then switch to 
guaranteed service depending on the actual performance of their network. If I 
can rate limit to 30/30 and fq_codel, and it’s 99% effective, I’d rather take 
that than settle for a guaranteed 20/20 on-season and 6/6 off-season to get to 
the same annual contract price.

I see that this is a great place for helpful info about fq_codel. I probably 
should have 

Re: [Codel] Using fq_codel with a WiFi uplink to the Internet

[Phineas Gage]

> On Sep 21, 2016, at 12:28 PM, Loganaden Velvindron  
> wrote:
> On Wed, Sep 21, 2016 at 1:59 PM, Phineas Gage  > wrote:
>> I have two questions about using fq_codel on an edge router when the
>> Internet uplink is through point-to-point WiFi:
>> 
>> Question #1: Is it still effective to run fq_codel on our edge router when I
>> have a WiFi uplink to the Internet, instead of a cabled connection like
>> ADSL? And related to that, is a high quality point-to-point WiFi connection
>> indistinguishable from a cabled connection as far as fq_codel is concerned?
> Yes, it is effective to a small extent. However, I would highly
> recommend that you look into make-wifi-fast, and start testing the
> firmware that Toke uploaded.
> 
> See this: http://blog.cerowrt.org/post/crypto_fq_bug/ 
> 

Thanks, this looks great. I’ll see if my WISP is willing to experiment on our 
point-to-point connection, but it would require driver support for an Atheros 
AR9300 on a Mikrotik 911-5HnD (firmware 3.30). That’s on our side. There’s 
another Mikrotik on the other side, but I don’t know its specs yet.

>> Question #2: Assuming the answer to Question #1 is an overall "yes", is it
>> then better to have a guaranteed speed from the ISP, instead of having a
>> variable but potentially higher speed, so that I can control the queue and
>> have fq_codel and HTB prioritization work effectively?
> Guaranteed speed, or at least minimum guaranteed speed for both upload
> and download is a good idea. You don't have to login and tweak each
> time.

Ok, that’s along the lines of what I’m thinking also. I may ask them for 
flexibility in the contract and try the aggregated connection to save money, 
then switch to guaranteed depending on the actual conditions on their network.

>> LAN <=> Linux bridge with fq_codel <=> ADSL Modem 0.4 / 5.0 Mbps <=> DSLAM …
> I have a similar configuration: LAN <=> fq_codel (tp-link archer c7 v2
> with pppoe) <=> FTTH modem (bridge mode)
> 
> May I ask why put the ADSL modem in bridge mode, and let the fq_codel
> box handle pope ?

This way, if the bridge ever fails, I can tell someone even if I’m off-site to 
just “plug in the red cable”, bypassing the bridge. The modem does PPPoE, DHCP 
and DNS caching. The bridge provides better DNS caching, an NTP server, and 
HTB+fq_codel, but if it’s replaced with a cable, the network continues working 
without any configuration changes.

Is there a benefit to running PPPoE on the bridge, other than possibly a bit 
faster PPPoE encapsulation?

>> But now, we have a chance to improve our throughput problem by switching to
>> a point-to-point WiFi uplink that could hit speeds of 30-40 Mbit symmetric
>> (more on the speeds later). We have to decide on starting a contract with
>> them. At the same time, I’ll be switching the bridge to a Ubiquiti
>> EdgeRouter X, which has fq_codel in its kernel, but should have the same
>> effect. It would look something like:
>> 
> Does EdgeRouter X also implement BQL for its network drivers ?

I hadn’t heard of BQL actually, so I read this: 
https://www.bufferbloat.net/projects/bloat/wiki/BQL_enabled_drivers/ 
. I don’t 
think I currently have BQL in my setup, and it’s still effective, maybe because 
I’m doing rate limiting, but that makes me wonder what I might be missing. 
Also, I started this thread for an update on BQL support in the EdgeRouter X:

https://community.ubnt.com/t5/EdgeMAX/EdgeRouter-X-BQL-support-for-ethernet/m-p/1684788#U1684788
 


If it’s as easy to add BQL (4-8 lines of source) as this says: 
https://www.bufferbloat.net/projects/codel/wiki/Best_practices_for_benchmarking_Codel_and_FQ_Codel/
 
,
 I could probably do it. I added a small ioctl for CDDA support to a Linux 
sound card driver some years ago, so have _very_ basic kernel development 
experience.

Q1: In what cases does BQL help, and is it like do be more useful or higher or 
lower bandwidths, or does that make a difference?

Q2: Is there a way I can tell in Linux if a given net driver supports BQL 
without looking at the source?

For the bridge, I’m repurposing an old Mac Mini with 1.25GHz PPC and am using 
its internal adapter (a Sun GEM 100Mbit adapter using the sungem driver) on the 
inside of our network. The outside, which has the fq_codel applied to it, is an 
Apple USB 100Mbit adapter.

Q3: Is it better to run fq_codel on an internal PCI based network adapter, or a 
USB ethernet adapter, or is there no difference?

In case it matters, the kernel has CONFIG_HZ=250. Also, I try to turn off any 
offloads in the interfaces file (to be honest, I don’t know if it makes a 

Re: [Codel] Using fq_codel with a WiFi uplink to the Internet

[Dave Taht]

On Wed, Sep 21, 2016 at 2:59 AM, Phineas Gage  wrote:
> I have two questions about using fq_codel on an edge router when the
> Internet uplink is through point-to-point WiFi:
>
> Question #1: Is it still effective to run fq_codel on our edge router when I
> have a WiFi uplink to the Internet, instead of a cabled connection like
> ADSL? And related to that, is a high quality point-to-point WiFi connection
> indistinguishable from a cabled connection as far as fq_codel is concerned?

It has, until recent developments, been helpful but not as effective
as we'd like.

We have two sets of code in the process of being finalized that should
work better
for a reasonably fast wifi uplink.

blog.cerowrt.org/post/fq_codel_on_ath10k/

https://blog.tohojo.dk/2016/06/fixing-the-wifi-performance-anomaly-on-ath9k.html

Ideally you want to be able to run it on both sides.


> Question #2: Assuming the answer to Question #1 is an overall "yes", is it
> then better to have a guaranteed speed from the ISP, instead of having a
> variable but potentially higher speed, so that I can control the queue and
> have fq_codel and HTB prioritization work effectively?
>
> Background: I manage the network for a camp / conference center that
> supports up to about 120 users (5-10 simultaneous, at times). For Internet,
> we've been using ADSL with 5 Mbit download, 0.4 Mbit upload (remote area, so
> it’s slow). The only thing that keeps it usable is running fq_codel on a
> transparent Linux bridge that sits between the LAN and ADSL modem. On this
> bridge, I restrict the upload and download rate to about 85-90% of maximum,
> and use fq_codel, plus some HTB prioritization rules. It’s extremely
> effective at providing usable latency, so kudos to the fq_codel algorithm
> and implementation. It looks like this:
>
> LAN <=> Linux bridge with fq_codel <=> ADSL Modem 0.4 / 5.0 Mbps <=> DSLAM …
>
> But now, we have a chance to improve our throughput problem by switching to
> a point-to-point WiFi uplink that could hit speeds of 30-40 Mbit symmetric
> (more on the speeds later). We have to decide on starting a contract with
> them. At the same time, I’ll be switching the bridge to a Ubiquiti
> EdgeRouter X, which has fq_codel in its kernel, but should have the same
> effect. It would look something like:
>
> LAN <=> Ubiquiti EdgeRouter X <=> WiFi Client (Mikrotik 802.11n MIMO 2x2)
> <=> WiFi AP from ISP …
>
> We already have a test setup in place. The link rate to the ISP's AP, as
> reported by Mikrotik's admin console, is currently 86.6 Mbps transmit and
> 144.4 Mbps receive, with CCQ (connection quality) at 64% transmit / 99%
> receive. First of all, I'll try to have them get that transmit CCQ up to 99%
> like the receive, to make sure it's a stable link. But I also know that the
> actual Internet throughput will be less than the link rates, and
> speedtest.net results are currently around 30-40 Mbps symmetric.

regrettably no matter how "constant" your provider claims to hold the
connection, in case of bad weather, especially, it is unlikely they
can do so.

>
> Moreover, it's WiFi, and that led to my Question #1. I know that latency and
> throughput can vary, and that there are more queues and more things
> happening with packet aggregation, etc that I don’t understand. Can this
> aspect of the variable latency, throughput and packet transmission
> characteristics of WiFi make fq_codel less effective when used in this way
> on an intermediate bridge or edge router, or does it more depend on the
> quality of the WiFi link, where a high quality point-to-point WiFi uplink to
> a good upstream network (there’s another unknown) is indistinguishable from
> a cabled connection?

Ideally the algorithm should run very close to the wifi mac layer as
per the urls I sent earlier.

>
> My Question #2 came from the fact that I have two options from the ISP:
>
> Option A: We can choose a maximum of 40/40 Mbit with 1:5 aggregation,
> meaning we could get 40 Mbit, but we could also get a lot less at times (8
> Mbit I assume), depending on other network load.
>
> Option B: We can get a guaranteed bandwidth, but it costs more, so the
> maximum throughput we can pay for would be less. We would probably choose
> around 6/6 Mbit off-season, and 20/20 Mbit on-season, as the camp is a
> seasonal business.
>
> My feeling, assuming that the answer to Question #1 is "yes" and I can
> effectively use fq_codel with WiFi at all, is to go with Option B, the
> guaranteed bandwidth. That way, I could set fq_codel to a little less than
> this bandwidth, and hopefully manage buffer bloat and do HTB prioritization
> in the same way I do now. But it depends on the answers to my two questions,
> is fq_codel still effective when using a WiFi uplink in general, and if so,
> is it better to go with a guaranteed bandwidth.

Lacking control of both sides, I would go for the garunteed bandwidth
and try to control it on the ethernet router, or with control of one

Re: [Codel] Using fq_codel with a WiFi uplink to the Internet

[Loganaden Velvindron]

On Wed, Sep 21, 2016 at 1:59 PM, Phineas Gage  wrote:
> I have two questions about using fq_codel on an edge router when the
> Internet uplink is through point-to-point WiFi:
>
> Question #1: Is it still effective to run fq_codel on our edge router when I
> have a WiFi uplink to the Internet, instead of a cabled connection like
> ADSL? And related to that, is a high quality point-to-point WiFi connection
> indistinguishable from a cabled connection as far as fq_codel is concerned?

Yes, it is effective to a small extent. However, I would highly
recommend that you look into make-wifi-fast, and start testing the
firmware that Toke uploaded.

See this: http://blog.cerowrt.org/post/crypto_fq_bug/

(Personally, I would still prefer cables for point to point, but with
the recent efforts of make-wifi-fast, this could change by next year)

>
> Question #2: Assuming the answer to Question #1 is an overall "yes", is it
> then better to have a guaranteed speed from the ISP, instead of having a
> variable but potentially higher speed, so that I can control the queue and
> have fq_codel and HTB prioritization work effectively?
>

Guaranteed speed, or at least minimum guaranteed speed for both upload
and download is a good idea. You don't have to login and tweak each
time.


> Background: I manage the network for a camp / conference center that
> supports up to about 120 users (5-10 simultaneous, at times). For Internet,
> we've been using ADSL with 5 Mbit download, 0.4 Mbit upload (remote area, so
> it’s slow). The only thing that keeps it usable is running fq_codel on a
> transparent Linux bridge that sits between the LAN and ADSL modem. On this
> bridge, I restrict the upload and download rate to about 85-90% of maximum,
> and use fq_codel, plus some HTB prioritization rules. It’s extremely
> effective at providing usable latency, so kudos to the fq_codel algorithm
> and implementation. It looks like this:
>
> LAN <=> Linux bridge with fq_codel <=> ADSL Modem 0.4 / 5.0 Mbps <=> DSLAM …

I have a similar configuration: LAN <=> fq_codel (tp-link archer c7 v2
with pppoe) <=> FTTH modem (bridge mode)

May I ask why put the ADSL modem in bridge mode, and let the fq_codel
box handle pppoe ?


>
> But now, we have a chance to improve our throughput problem by switching to
> a point-to-point WiFi uplink that could hit speeds of 30-40 Mbit symmetric
> (more on the speeds later). We have to decide on starting a contract with
> them. At the same time, I’ll be switching the bridge to a Ubiquiti
> EdgeRouter X, which has fq_codel in its kernel, but should have the same
> effect. It would look something like:
>
Does EdgeRouter X also implement BQL for its network drivers ?

> LAN <=> Ubiquiti EdgeRouter X <=> WiFi Client (Mikrotik 802.11n MIMO 2x2)
> <=> WiFi AP from ISP …
>
> We already have a test setup in place. The link rate to the ISP's AP, as
> reported by Mikrotik's admin console, is currently 86.6 Mbps transmit and
> 144.4 Mbps receive, with CCQ (connection quality) at 64% transmit / 99%
> receive. First of all, I'll try to have them get that transmit CCQ up to 99%
> like the receive, to make sure it's a stable link. But I also know that the
> actual Internet throughput will be less than the link rates, and
> speedtest.net results are currently around 30-40 Mbps symmetric.
>
> Moreover, it's WiFi, and that led to my Question #1. I know that latency and
> throughput can vary, and that there are more queues and more things
> happening with packet aggregation, etc that I don’t understand. Can this
> aspect of the variable latency, throughput and packet transmission
> characteristics of WiFi make fq_codel less effective when used in this way
> on an intermediate bridge or edge router, or does it more depend on the
> quality of the WiFi link, where a high quality point-to-point WiFi uplink to
> a good upstream network (there’s another unknown) is indistinguishable from
> a cabled connection?
>
> My Question #2 came from the fact that I have two options from the ISP:
>
> Option A: We can choose a maximum of 40/40 Mbit with 1:5 aggregation,
> meaning we could get 40 Mbit, but we could also get a lot less at times (8
> Mbit I assume), depending on other network load.
>
> Option B: We can get a guaranteed bandwidth, but it costs more, so the
> maximum throughput we can pay for would be less. We would probably choose
> around 6/6 Mbit off-season, and 20/20 Mbit on-season, as the camp is a
> seasonal business.
>
> My feeling, assuming that the answer to Question #1 is "yes" and I can
> effectively use fq_codel with WiFi at all, is to go with Option B, the
> guaranteed bandwidth. That way, I could set fq_codel to a little less than
> this bandwidth, and hopefully manage buffer bloat and do HTB prioritization
> in the same way I do now. But it depends on the answers to my two questions,
> is fq_codel still effective when using a WiFi uplink in general, and if so,
> is it better to go with a guaranteed bandwidth.