Re: [WISPA] RED queues out, PCQ queues in - anyone else doing this?
On Sun, Apr 10, 2011 at 5:58 PM, Greg Ihnen os10ru...@gmail.com wrote: Rubens, Thanks for the reply! I'm using a 5GHz AirMax back haul (PtMP) to two 2.4GHz APs (All UBNT gear). The 5GHz back haul has never broken a sweat. Our upstream is a 1M/256K high latency connection so there just isn't that much data to move. Satellite, huh ? You will probably gain a lot by forcing users to a transparent proxy. You can do lots of TCP tuning on a server that would be either impossible on some Microsoft TCP/IP stacks or too expensive in support hours to do on the end users machines. Caching also comes to mind. You got me thinking about the ack packets. Besides possibly a queue type, what do you think about prioritizing them high? High priority for ACK packets usually turns into better performance perception on any network. I would try it for sure, but consider the proxy option above for your specific scenario (not the usual WISP one). Rubens WISPA Wants You! Join today! http://signup.wispa.org/ WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/
Re: [WISPA] RED queues out, PCQ queues in - anyone else doing this?
Rubens, Thanks! Yes, I prioritized small ack, syn and fin packets. I think I'm seeing an improvement. Our modem already does caching and I can't control it. It even caches DNS which breaks things like OpenDNS. Happily they fixed the http/s caching so we're not still seeing week old DrudgeReport pages. Greg On Apr 11, 2011, at 9:47 AM, Rubens Kuhl wrote: On Sun, Apr 10, 2011 at 5:58 PM, Greg Ihnen os10ru...@gmail.com wrote: Rubens, Thanks for the reply! I'm using a 5GHz AirMax back haul (PtMP) to two 2.4GHz APs (All UBNT gear). The 5GHz back haul has never broken a sweat. Our upstream is a 1M/256K high latency connection so there just isn't that much data to move. Satellite, huh ? You will probably gain a lot by forcing users to a transparent proxy. You can do lots of TCP tuning on a server that would be either impossible on some Microsoft TCP/IP stacks or too expensive in support hours to do on the end users machines. Caching also comes to mind. You got me thinking about the ack packets. Besides possibly a queue type, what do you think about prioritizing them high? High priority for ACK packets usually turns into better performance perception on any network. I would try it for sure, but consider the proxy option above for your specific scenario (not the usual WISP one). Rubens WISPA Wants You! Join today! http://signup.wispa.org/ WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/ WISPA Wants You! Join today! http://signup.wispa.org/ WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/
Re: [WISPA] RED queues out, PCQ queues in - anyone else doing this?
Rubens, Thanks for the reply! I'm using a 5GHz AirMax back haul (PtMP) to two 2.4GHz APs (All UBNT gear). The 5GHz back haul has never broken a sweat. Our upstream is a 1M/256K high latency connection so there just isn't that much data to move. You got me thinking about the ack packets. Besides possibly a queue type, what do you think about prioritizing them high? Thanks! Greg On Apr 7, 2011, at 10:19 PM, Rubens Kuhl wrote: I was running Butch's script with PCQ queues but I started wondering about buffer bloat (yeah, I follow NANOG too) on the router. I thought about trying RED on the outbound queue since if packets are dropped and resent on our wireless network it's no biggie. Our wireless network is way overkill as far as our bandwidth needs. But I didn't want dropped packets on our inbound side because I didn't want to waste any of our precious satellite bandwidth. So I kept PCQ queues there. Before jumping into the conclusion that your network is overkill for your usage, you should first graph it in RX+TX pps if it's Wi-Fi, or RX pps and TX pps otherwise. Ideally you should also graph airtime % as well, but that's not a MIB-II standard item... AirControl might do it with UBNT gear. It seems like it made things work better but I never know for sure because our satellite bandwidth is oversold and what we get at any given moment is effected by what the other users who are on this same bandwidth are doing. Does anyone else mix queue types like that? Is this a dumb idea? I think it's not dumb, but the cause/effect relations on TCP make choosing which queue type to use in each direction a more complex decision than that. Trying more combinations might be good. One thing I would consider doing is using different queue types on each direction depending on packet size. TCP packets going outbound but have low size are just ACKs of incoming TCP data, and the other way around. non-TCP packets would also have a different QoS strategy as it's usually non-responsive to packet loss or delay variations. Rubens WISPA Wants You! Join today! http://signup.wispa.org/ WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/ WISPA Wants You! Join today! http://signup.wispa.org/ WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/
Re: [WISPA] RED queues out, PCQ queues in - anyone else doing this?
One of the best examples is the impact of half duplex radios, or adaptive speed (modulation) radios, on bandwdith management systems that treat Ideally, adaptive modulation radios should have QoS policies built-in. That is true for Ceragon IP-MAX^2 radios that are aware to EXP MPLS markings, but besides that expection, I don't know radios that do it. for us, for 10 years. But as our network became more congested, half duplex did show to be a challenge for traffic management. It came to a point where Full Duplex licensed links was the only answer, and helped the most. And then our traffic management became more reliable as a result. My point is, its not only the method of traffic management that matters, but also the characteristics of the network. Queuing and QOS will always help make the best of one's network, but it wont fully make up for deficiencies in a physical network. For a growing-up network, I think two half-duplex could be used for better performance. For instance, two OSPF links with unequal inverted costs, so each one will normally have unidiretional traffic, fall-back to bidirectional if one of the links fail. Rubens WISPA Wants You! Join today! http://signup.wispa.org/ WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/
Re: [WISPA] RED queues out, PCQ queues in - anyone else doing this?
On Apr 7, 2011, at 9:53 PM, Butch Evans wrote: On 04/07/2011 06:23 PM, Greg Ihnen wrote: My little network is a wireless network with about 20 user devices (computers, iPads, iPods, Wiis, Blackberries etc). Our upstream is a 1Mbps/256KBps. I was running Butch's script with PCQ queues but I started wondering about buffer bloat (yeah, I follow NANOG too) on the router. I thought about trying RED on the outbound queue since if packets are dropped and resent on our wireless network it's no biggie. Our wireless network is way overkill as far as our bandwidth needs. But I didn't want dropped packets on our inbound side because I didn't want to waste any of our precious satellite bandwidth. So I kept PCQ queues there. It seems like it made things work better but I never know for sure because our satellite bandwidth is oversold and what we get at any given moment is effected by what the other users who are on this same bandwidth are doing. Does anyone else mix queue types like that? Is this a dumb idea? FWIW, the NEWEST version of this script uses RED queues. (just so you know). As for splitting the queues per direction like this, I'm not sure I've ever tried this, but it should perform reasonably well. Butch, Thanks for the reply and for your script! It makes our little internet connection here work so well. It truly makes a night and day difference on a poor connection with a lot of users. Greg -- * Butch Evans * Professional Network Consultation* * http://www.butchevans.com/* Network Engineering * * http://store.wispgear.net/* Wired or Wireless Networks * * http://blog.butchevans.com/ * ImageStream, Mikrotik and MORE! * *NOTE THE NEW PHONE NUMBER: 702-537-0979 * WISPA Wants You! Join today! http://signup.wispa.org/ WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/ WISPA Wants You! Join today! http://signup.wispa.org/ WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/
Re: [WISPA] RED queues out, PCQ queues in - anyone else doing this?
Rubes, Thank you very much! That's great info and ideas. Greg On Apr 7, 2011, at 10:19 PM, Rubens Kuhl wrote: I was running Butch's script with PCQ queues but I started wondering about buffer bloat (yeah, I follow NANOG too) on the router. I thought about trying RED on the outbound queue since if packets are dropped and resent on our wireless network it's no biggie. Our wireless network is way overkill as far as our bandwidth needs. But I didn't want dropped packets on our inbound side because I didn't want to waste any of our precious satellite bandwidth. So I kept PCQ queues there. Before jumping into the conclusion that your network is overkill for your usage, you should first graph it in RX+TX pps if it's Wi-Fi, or RX pps and TX pps otherwise. Ideally you should also graph airtime % as well, but that's not a MIB-II standard item... AirControl might do it with UBNT gear. It seems like it made things work better but I never know for sure because our satellite bandwidth is oversold and what we get at any given moment is effected by what the other users who are on this same bandwidth are doing. Does anyone else mix queue types like that? Is this a dumb idea? I think it's not dumb, but the cause/effect relations on TCP make choosing which queue type to use in each direction a more complex decision than that. Trying more combinations might be good. One thing I would consider doing is using different queue types on each direction depending on packet size. TCP packets going outbound but have low size are just ACKs of incoming TCP data, and the other way around. non-TCP packets would also have a different QoS strategy as it's usually non-responsive to packet loss or delay variations. Rubens WISPA Wants You! Join today! http://signup.wispa.org/ WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/ WISPA Wants You! Join today! http://signup.wispa.org/ WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/
Re: [WISPA] RED queues out, PCQ queues in - anyone else doing this?
There is definately a need for different queue types at different points on the network. Multiple Queue types have been developed because there are different problems to solve for different situations. What I question is when it is necessary to solve a problem. I hardly justify a complete network queueing standard overhaul, just to satisfy the abilty to perform a single stream TCP test to Speak easy at full speed, when most business circuits serve many TCP streams at a time to fill capacity. So it boils down to weighing the scale of how bad the problem is and how badly the customers notices it. There can be a very fine line on which Queueing methods are required for specific cases, and sometimes picking one makes it easier to consistently implement, even if there are some trade offs. On our core routers we've found RED to work well. But we also have other areas where we queue where we use other things, such building routers or customer routers. Tom DeReggi RapidDSL Wireless, Inc IntAirNet- Fixed Wireless Broadband WISPA Wants You! Join today! http://signup.wispa.org/ WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/
Re: [WISPA] RED queues out, PCQ queues in - anyone else doing this?
On 04/08/2011 12:07 PM, Tom DeReggi wrote: There is definately a need for different queue types at different points on the network. Multiple Queue types have been developed because there are different problems to solve for different situations. This is true. What I question is when it is necessary to solve a problem. I hardly justify a complete network queueing standard overhaul, just to satisfy the abilty to perform a single stream TCP test to Speak easy at full speed, when most business circuits serve many TCP streams at a time to fill capacity. This is a very good point. The current trending of internet traffic is geared toward more and more streams. Of the available queue types, only SFQ (or one of it's derivative types) and RED make much sense. FIFO queues, without properly managing the traffic entering those queues, will cause customers to sit up and take notice in a negative way. The problem with SFQ is that the algorithm used to implement this is fairly slow. It doesn't work well under heavy load. More specifically, it falls apart when the volume of traffic is excessive. Mikrotik adds another queue type called PCQ, which is sort of like FIFO queues grouped by some classifier such as source addresses and/or ports OR destination addresses and/or ports OR some wicked combination of all of the above. PCQ is an alternative, as it allows you to set per classification speed limits, but in the end, it is still FIFO per class, which requires very careful crafting of defining traffic types to be sorted into these queues. As to WHEN you need to begin managing network traffic, I personally feel it is ALWAYS necessary. The reality is that even with sufficient bandwidth available, some of that traffic is more sensitive than others to network latencies. Even if you are not creating a QOS policy, you have one. Every interface has an output queue. All a policy does is inform the operating system as to what your preferred order is when placing packets into that queue. There is obviously SOME added latency involved in doing that, but usually that added time can result in a better end user experience. QOS enables you to manage not only high bandwidth use, but high packet rates. You are not limiting packet rates per se, but when that increases (especially on a wireless network), you are more likely to experience collisions. QOS enables you to make those collisions less problematic because you are managing the output side of the interface and know that the important traffic will go first. So it boils down to weighing the scale of how bad the problem is and how badly the customers notices it. There can be a very fine line on which Queueing methods are required for specific cases, and sometimes picking one makes it easier to consistently implement, even if there are some trade offs. On our core routers we've found RED to work well. But we also have other areas where we queue where we use other things, such building routers or customer routers. QOS involves speed limits, but is not always about limiting speeds. A more accurate description of what QOS is would be something like: Management of network traffic policies such that periods of low utilization permit full access to network resources and periods of high utilization will permit preferred access to certain traffic types. Additionally, QOS policies enable an admistrator to level out peaks during very high utilization periods such that all traffic is permitted to pass the policy, but in an orderly fashion. I spent almost 1 minute coming up with that definition, so give me some slack. :-) -- * Butch Evans * Professional Network Consultation* * http://www.butchevans.com/* Network Engineering * * http://store.wispgear.net/* Wired or Wireless Networks * * http://blog.butchevans.com/ * ImageStream, Mikrotik and MORE! * *NOTE THE NEW PHONE NUMBER: 702-537-0979 * WISPA Wants You! Join today! http://signup.wispa.org/ WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/
Re: [WISPA] RED queues out, PCQ queues in - anyone else doing this?
Butch, Nice reply, I agree with everything you said. I agree that the time to begin managing network traffic, is always. The reason I believe this is... 1) any thing one does to help, helps. 2) If you dont have traffic management in place, when you need it, it will be to late. But its also important to recognize that flaws might exist in a network design that cant easilly be avoided, that can undermine an administrators attempt to manage their traffic. What I mean by that is that, some variable beyond one's control might have a higher effect than factors that are in one's control. So because of this, traffic management techniques cant always be 100% relied on, although they help a great deal. One of the best examples is the impact of half duplex radios, or adaptive speed (modulation) radios, on bandwdith management systems that treat outbound and inbound traffic seperately. In many cases, bandwidth management requires you to know the capacity of a link in advance. So, lets say you have a 10mbps half duplex link. If you want to be able to have 10mbps up traffic, you must tell your bandwidth management to allow that 10mbps up. But then what happens if you have 5mbps of download traffic? The upload bandwidth management still thinks it has 10mbps max, but in reality it only has 5mbps available, while 5mbps is downloading. Its impossibly to know in advance the up and down ratio. Or lets look at it a different way... RED gracefully drops packets to slow down throughput as needed. When a half duplex link gets saturated, which direction (up or down traffic) should packets be dropped from? Many of the traffic management methods are based on the assumption that links are full duplex, and get applied to outbound interfaces, therefore a different interface or queue per direction. In some cases, a different router manages the traffic in the other direction, and not able to communicate with the other router.. (example, router on each side of a link, each managing a queue for outbound). Traffic management is much easier to impliment on a full duplex fixed capacity fiber connection, than it is on a variable half duplex wireless connection. Queuing traffic management is not the same as bandwidth management, but none the less, I believe to have demonstrated my point. When our network was less congested, traffic management tools made a huge different for us to maximize the amount of data we could pass smoothly accross our network efficiently. What we found was that half duplex was efficient for RF, and not a disadvantage for wireless. This worked well for us, for 10 years. But as our network became more congested, half duplex did show to be a challenge for traffic management. It came to a point where Full Duplex licensed links was the only answer, and helped the most. And then our traffic management became more reliable as a result. My point is, its not only the method of traffic management that matters, but also the characteristics of the network. Queuing and QOS will always help make the best of one's network, but it wont fully make up for deficiencies in a physical network. Tom DeReggi RapidDSL Wireless, Inc IntAirNet- Fixed Wireless Broadband - Original Message - From: Butch Evans but...@butchevans.com To: wireless@wispa.org Sent: Friday, April 08, 2011 2:53 PM Subject: Re: [WISPA] RED queues out, PCQ queues in - anyone else doing this? On 04/08/2011 12:07 PM, Tom DeReggi wrote: There is definately a need for different queue types at different points on the network. Multiple Queue types have been developed because there are different problems to solve for different situations. This is true. What I question is when it is necessary to solve a problem. I hardly justify a complete network queueing standard overhaul, just to satisfy the abilty to perform a single stream TCP test to Speak easy at full speed, when most business circuits serve many TCP streams at a time to fill capacity. This is a very good point. The current trending of internet traffic is geared toward more and more streams. Of the available queue types, only SFQ (or one of it's derivative types) and RED make much sense. FIFO queues, without properly managing the traffic entering those queues, will cause customers to sit up and take notice in a negative way. The problem with SFQ is that the algorithm used to implement this is fairly slow. It doesn't work well under heavy load. More specifically, it falls apart when the volume of traffic is excessive. Mikrotik adds another queue type called PCQ, which is sort of like FIFO queues grouped by some classifier such as source addresses and/or ports OR destination addresses and/or ports OR some wicked combination of all of the above. PCQ is an alternative, as it allows you to set per classification speed limits, but in the end, it is still FIFO per class, which requires very careful crafting of defining traffic types
[WISPA] RED queues out, PCQ queues in - anyone else doing this?
My little network is a wireless network with about 20 user devices (computers, iPads, iPods, Wiis, Blackberries etc). Our upstream is a 1Mbps/256KBps. I was running Butch's script with PCQ queues but I started wondering about buffer bloat (yeah, I follow NANOG too) on the router. I thought about trying RED on the outbound queue since if packets are dropped and resent on our wireless network it's no biggie. Our wireless network is way overkill as far as our bandwidth needs. But I didn't want dropped packets on our inbound side because I didn't want to waste any of our precious satellite bandwidth. So I kept PCQ queues there. It seems like it made things work better but I never know for sure because our satellite bandwidth is oversold and what we get at any given moment is effected by what the other users who are on this same bandwidth are doing. Does anyone else mix queue types like that? Is this a dumb idea? Thanks! Greg WISPA Wants You! Join today! http://signup.wispa.org/ WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/
Re: [WISPA] RED queues out, PCQ queues in - anyone else doing this?
On 04/07/2011 06:23 PM, Greg Ihnen wrote: My little network is a wireless network with about 20 user devices (computers, iPads, iPods, Wiis, Blackberries etc). Our upstream is a 1Mbps/256KBps. I was running Butch's script with PCQ queues but I started wondering about buffer bloat (yeah, I follow NANOG too) on the router. I thought about trying RED on the outbound queue since if packets are dropped and resent on our wireless network it's no biggie. Our wireless network is way overkill as far as our bandwidth needs. But I didn't want dropped packets on our inbound side because I didn't want to waste any of our precious satellite bandwidth. So I kept PCQ queues there. It seems like it made things work better but I never know for sure because our satellite bandwidth is oversold and what we get at any given moment is effected by what the other users who are on this same bandwidth are doing. Does anyone else mix queue types like that? Is this a dumb idea? FWIW, the NEWEST version of this script uses RED queues. (just so you know). As for splitting the queues per direction like this, I'm not sure I've ever tried this, but it should perform reasonably well. -- * Butch Evans * Professional Network Consultation* * http://www.butchevans.com/* Network Engineering * * http://store.wispgear.net/* Wired or Wireless Networks * * http://blog.butchevans.com/ * ImageStream, Mikrotik and MORE! * *NOTE THE NEW PHONE NUMBER: 702-537-0979 * WISPA Wants You! Join today! http://signup.wispa.org/ WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/
Re: [WISPA] RED queues out, PCQ queues in - anyone else doing this?
I was running Butch's script with PCQ queues but I started wondering about buffer bloat (yeah, I follow NANOG too) on the router. I thought about trying RED on the outbound queue since if packets are dropped and resent on our wireless network it's no biggie. Our wireless network is way overkill as far as our bandwidth needs. But I didn't want dropped packets on our inbound side because I didn't want to waste any of our precious satellite bandwidth. So I kept PCQ queues there. Before jumping into the conclusion that your network is overkill for your usage, you should first graph it in RX+TX pps if it's Wi-Fi, or RX pps and TX pps otherwise. Ideally you should also graph airtime % as well, but that's not a MIB-II standard item... AirControl might do it with UBNT gear. It seems like it made things work better but I never know for sure because our satellite bandwidth is oversold and what we get at any given moment is effected by what the other users who are on this same bandwidth are doing. Does anyone else mix queue types like that? Is this a dumb idea? I think it's not dumb, but the cause/effect relations on TCP make choosing which queue type to use in each direction a more complex decision than that. Trying more combinations might be good. One thing I would consider doing is using different queue types on each direction depending on packet size. TCP packets going outbound but have low size are just ACKs of incoming TCP data, and the other way around. non-TCP packets would also have a different QoS strategy as it's usually non-responsive to packet loss or delay variations. Rubens WISPA Wants You! Join today! http://signup.wispa.org/ WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/
