Re: Working example of bi-directional asymmetric ALTQ + NAT ruleset?
Bingo! Thank you Kelley and (especially again) StuartH. Seems I was confused because the queue directives will work without specifying an explicit interfaceso I assumed it just used the interface given in the most recent altq given above it. But that's not the case - and it seems it works without an interface given **if there are queues defined on only one interface in total in the entire pf.conf**. As soon as you have multiple interfaces, it looks like need to specify the interface explictly in both the altq directive AND the individual queue directives beneath it. Not sure why this feature is helpful - personally I think the syntax checking should be a little tighter - but here's what I have, and it seems to be working beautifully: #cat /etc/pf.conf ... altq on $int_if cbq bandwidth 5000Kb queue { std, slow, fast, tcpack } queue std on $int_if bandwidth 1200Kb priority 1 cbq(default borrow) queue slow on $int_if bandwidth 400Kb priority 0 cbq(ecn) queue fast on $int_if bandwidth 3300Kb priority 2 cbq(borrow) queue tcpack on $int_if bandwidth 100Kb priority 3 cbq(borrow) # altq on $ext_if cbq bandwidth 900Kb queue { std, slow, fast, tcpack } queue std on $ext_if bandwidth 400Kb priority 1 cbq(default borrow) queue slow on $ext_if bandwidth 50Kb priority 0 cbq(ecn) queue fast on $ext_if bandwidth 400Kb priority 2 cbq(borrow) queue tcpack on $ext_if bandwidth 50Kb priority 3 cbq(borrow) ... Many thanks again for the help and discussion, all. This has been instructive and illuminating. -BP- On Jan 12, 12:08=A0pm, s...@spacehopper.org (Stuart Henderson) wrote: On 2011/01/12 08:40, Bonnie Packet wrote: altq on $int_if cbq bandwidth 5000Kb queue { std, slow, fast, tcpack } queue std bandwidth 1200Kb priority 1 cbq(default borrow) you're looking for this format: queue std on $int_if bandwidth 1200Kb priority 1 cbq(default borrow) ..etc.. I don't recall many changes to altq since 3.9 (though of course there have been many other fixes, including some related to DoS issues in certain circumstances).
Re: Working example of bi-directional asymmetric ALTQ + NAT ruleset?
On 2011/01/14 00:24, Bonnie Packet wrote: Bingo! Thank you Kelley and (especially again) StuartH. Seems I was confused because the queue directives will work without specifying an explicit interfaceso I assumed it just used the interface given in the most recent altq given above it. But that's not the case - and it seems it works without an interface given **if there are queues defined on only one interface in total in the entire pf.conf**. As soon as you have multiple interfaces, it looks like need to specify the interface explictly in both the altq directive AND the individual queue directives beneath it. Not sure why this feature is helpful - personally I think the syntax checking should be a little tighter - but here's what I have, and it seems to be working beautifully: It's simpler than that, actually; if you don't specify the interface in the queue directive, it applies to all interfaces that use altq. If you're using the same bandwidth/priority on every interface, you only need list the queues once. I'm not sure what the 3.9 docs said, this is what the current OS has to say about queue...on in pf.conf(5): on interface Specifies the interface the queue operates on. If not given, it operates on all matching interfaces. I've tried to clean up the manual and faq in this area but despite the fact it's not entirely clear as-is, it's been surprisingly difficult to come up with anything better. Glad you got it working (and that we now have some more in the list archives to hopefully help others :-) #cat /etc/pf.conf ... altq on $int_if cbq bandwidth 5000Kb queue { std, slow, fast, tcpack } queue std on $int_if bandwidth 1200Kb priority 1 cbq(default borrow) queue slow on $int_if bandwidth 400Kb priority 0 cbq(ecn) queue fast on $int_if bandwidth 3300Kb priority 2 cbq(borrow) queue tcpack on $int_if bandwidth 100Kb priority 3 cbq(borrow) # altq on $ext_if cbq bandwidth 900Kb queue { std, slow, fast, tcpack } queue std on $ext_if bandwidth 400Kb priority 1 cbq(default borrow) queue slow on $ext_if bandwidth 50Kb priority 0 cbq(ecn) queue fast on $ext_if bandwidth 400Kb priority 2 cbq(borrow) queue tcpack on $ext_if bandwidth 50Kb priority 3 cbq(borrow) ... Many thanks again for the help and discussion, all. This has been instructive and illuminating. -BP- On Jan 12, 12:08=A0pm, s...@spacehopper.org (Stuart Henderson) wrote: On 2011/01/12 08:40, Bonnie Packet wrote: altq on $int_if cbq bandwidth 5000Kb queue { std, slow, fast, tcpack } queue std bandwidth 1200Kb priority 1 cbq(default borrow) you're looking for this format: queue std on $int_if bandwidth 1200Kb priority 1 cbq(default borrow) ..etc.. I don't recall many changes to altq since 3.9 (though of course there have been many other fixes, including some related to DoS issues in certain circumstances).
Re: Working example of bi-directional asymmetric ALTQ + NAT ruleset?
On 01/14/2011 04:48:48 AM, Stuart Henderson wrote: I'm not sure what the 3.9 docs said, this is what the current OS has to say about queue...on in pf.conf(5): on interface Specifies the interface the queue operates on. If not given, it operates on all matching interfaces. That seems clear to me. It's been years since I fiddled with my queuing setup. I'm now motivated to revisit it. Karl k...@meme.com Free Software: You don't pay back, you pay forward. -- Robert A. Heinlein
Re: Working example of bi-directional asymmetric ALTQ + NAT ruleset?
On Jan 11, 2011, at 1:35 AM, Bonnie Packet wrote: Note that I know PF reasonably well and have altq queuing / rate limiting working perfectly already in ONE direction (right now, the high-bandwidth download side) - I just can't figure out how to get it working in both directions, up and down, simultaneously. I've beat my head against TFM and nothing seems applicable - though this can't be an uncommon need...? Are you creating your in/out queues on a single interface? If so, that might be your problem. We have our box set up with both a lan interface and a wan interface. The outbound queues are children of the wan interface, but the inbound queues are children of the lan interface. The folklore you hear about only being able to shape outbound traffic is sort of true; you can only shape the packets as they leave the PF box. However, as packets traverse your box, they come IN, get processed, and then go OUT to be delivered, so you always have the opportunity to shape them. You just need to make sure you're doing it at the right end. Essentially, for inbound traffic (from the internet to your lan), PF queues (and possibly drops) packets *just before* they would be delivered to your boxes on the inside. It's a little tragic (as others have mentioned) since those packets have already consumed your DSL bandwidth only to be dropped, but it's necessary to force the congestion-control to slow the flow for future packets. With a 2-interface 2-queue setup, we've been able to manage asymmetrical bandwidth amounts. We now have a symmetric connection, but I know we had one working back when we were on a 16up/2down cable connection... Jason -- Jason Healy|jhe...@logn.net| http://www.logn.net/
Re: Working example of bi-directional asymmetric ALTQ + NAT ruleset?
If you look at the posting from yesterday, you'll see the attempted setup is on two different interfaces, yes. That's been the case from the start. However, when I try to do the setup suggested by folks here several times now (same queue name repeated on another interface) pf complains bitterly and refuses to load the rules (see below). This is on an old, old OpenBSD 3.9 system - does anyone know if something changed in later releases to allows the the same queue name on multiple interfaces? (I looked through all the change logs up through the current 4.8, but didn't see anything.) Or do I have the syntax wrong? #cat /etc/pf.conf .. altq on $int_if cbq bandwidth 5000Kb queue { std, slow, fast, tcpack } queue std bandwidth 1200Kb priority 1 cbq(default borrow) queue slow bandwidth 400Kb priority 0 cbq(ecn) queue fast bandwidth 3300Kb priority 2 cbq(borrow) queue tcpack bandwidth 100Kb priority 3 cbq(borrow) altq on $ext_if cbq bandwidth 900Kb queue { std, slow, fast, tcpack } queue std bandwidth 400Kb priority 1 cbq(default borrow) queue slow bandwidth 50Kb priority 0 cbq(ecn) queue fast bandwidth 400Kb priority 2 cbq(borrow) queue tcpack bandwidth 50Kb priority 3 cbq(borrow) #./rc.pf queue std already exists on interface rl0 /etc/pf.conf:45: errors in queue definition queue slow already exists on interface rl0 /etc/pf.conf:46: errors in queue definition queue fast already exists on interface rl0 /etc/pf.conf:47: errors in queue definition queue tcpack already exists on interface rl0 /etc/pf.conf:48: errors in queue definition pfctl: Syntax error in config file: pf rules not loaded On Jan 12, 12:01=A0am, jhe...@logn.net (Jason Healy) wrote: On Jan 11, 2011, at 1:35 AM, Bonnie Packet wrote: Note that I know PF reasonably well and have altq queuing / rate limiting working perfectly already in ONE direction (right now, the high-bandwidth download side) - I just can't figure out how to get it working in both directions, up and down, simultaneously. =A0I've beat m= y head against TFM and nothing seems applicable - though this can't be an uncommon need...? Are you creating your in/out queues on a single interface? =A0If so, that= might be your problem. We have our box set up with both a lan interface and a wan interface.= =A0The outbound queues are children of the wan interface, but the inb= ound queues are children of the lan interface. =A0The folklore you hear = about only being able to shape outbound traffic is sort of true; you can = only shape the packets as they leave the PF box. =A0However, as packets tra= verse your box, they come IN, get processed, and then go OUT to be delivere= d, so you always have the opportunity to shape them. =A0You just need to ma= ke sure you're doing it at the right end. Essentially, for inbound traffic (from the internet to your lan), PF qu= eues (and possibly drops) packets *just before* they would be delivered to = your boxes on the inside. =A0It's a little tragic (as others have mentioned= ) since those packets have already consumed your DSL bandwidth only to be d= ropped, but it's necessary to force the congestion-control to slow the flow= for future packets. With a 2-interface 2-queue setup, we've been able to manage asymmetrical = bandwidth amounts. =A0We now have a symmetric connection, but I know we had= one working back when we were on a 16up/2down cable connection... Jason -- Jason Healy =A0 =A0| =A0 =a0jhe...@logn.net =A0 =A0| =A0http://www.logn.n= et/
Re: Working example of bi-directional asymmetric ALTQ + NAT ruleset?
On 2011/01/12 08:40, Bonnie Packet wrote: altq on $int_if cbq bandwidth 5000Kb queue { std, slow, fast, tcpack } queue std bandwidth 1200Kb priority 1 cbq(default borrow) you're looking for this format: queue std on $int_if bandwidth 1200Kb priority 1 cbq(default borrow) ..etc.. I don't recall many changes to altq since 3.9 (though of course there have been many other fixes, including some related to DoS issues in certain circumstances).
Re: Working example of bi-directional asymmetric ALTQ + NAT ruleset?
From the faq: The syntax for the queue directive is: queue name [on interface] bandwidth bw [priority pri] [qlimit qlim] \ scheduler ( sched_options ) { queue_list } Specify on which interface you are assigning each queue. queue bob on fxp0 queue bob on fxp1 Kelley Reynolds President Inside Systems, Inc. On Jan 12, 2011, at 11:40 AM, Bonnie Packet wrote: If you look at the posting from yesterday, you'll see the attempted setup is on two different interfaces, yes. That's been the case from the start. However, when I try to do the setup suggested by folks here several times now (same queue name repeated on another interface) pf complains bitterly and refuses to load the rules (see below). This is on an old, old OpenBSD 3.9 system - does anyone know if something changed in later releases to allows the the same queue name on multiple interfaces? (I looked through all the change logs up through the current 4.8, but didn't see anything.) Or do I have the syntax wrong? #cat /etc/pf.conf .. altq on $int_if cbq bandwidth 5000Kb queue { std, slow, fast, tcpack } queue std bandwidth 1200Kb priority 1 cbq(default borrow) queue slow bandwidth 400Kb priority 0 cbq(ecn) queue fast bandwidth 3300Kb priority 2 cbq(borrow) queue tcpack bandwidth 100Kb priority 3 cbq(borrow) altq on $ext_if cbq bandwidth 900Kb queue { std, slow, fast, tcpack } queue std bandwidth 400Kb priority 1 cbq(default borrow) queue slow bandwidth 50Kb priority 0 cbq(ecn) queue fast bandwidth 400Kb priority 2 cbq(borrow) queue tcpack bandwidth 50Kb priority 3 cbq(borrow) #./rc.pf queue std already exists on interface rl0 /etc/pf.conf:45: errors in queue definition queue slow already exists on interface rl0 /etc/pf.conf:46: errors in queue definition queue fast already exists on interface rl0 /etc/pf.conf:47: errors in queue definition queue tcpack already exists on interface rl0 /etc/pf.conf:48: errors in queue definition pfctl: Syntax error in config file: pf rules not loaded On Jan 12, 12:01=A0am, jhe...@logn.net (Jason Healy) wrote: On Jan 11, 2011, at 1:35 AM, Bonnie Packet wrote: Note that I know PF reasonably well and have altq queuing / rate limiting working perfectly already in ONE direction (right now, the high-bandwidth download side) - I just can't figure out how to get it working in both directions, up and down, simultaneously. =A0I've beat m= y head against TFM and nothing seems applicable - though this can't be an uncommon need...? Are you creating your in/out queues on a single interface? =A0If so, that= might be your problem. We have our box set up with both a lan interface and a wan interface.= =A0The outbound queues are children of the wan interface, but the inb= ound queues are children of the lan interface. =A0The folklore you hear = about only being able to shape outbound traffic is sort of true; you can = only shape the packets as they leave the PF box. =A0However, as packets tra= verse your box, they come IN, get processed, and then go OUT to be delivere= d, so you always have the opportunity to shape them. =A0You just need to ma= ke sure you're doing it at the right end. Essentially, for inbound traffic (from the internet to your lan), PF qu= eues (and possibly drops) packets *just before* they would be delivered to = your boxes on the inside. =A0It's a little tragic (as others have mentioned= ) since those packets have already consumed your DSL bandwidth only to be d= ropped, but it's necessary to force the congestion-control to slow the flow= for future packets. With a 2-interface 2-queue setup, we've been able to manage asymmetrical = bandwidth amounts. =A0We now have a symmetric connection, but I know we had= one working back when we were on a 16up/2down cable connection... Jason -- Jason Healy =A0 =A0| =A0 =a0jhe...@logn.net =A0 =A0| =A0http://www.logn.n= et/
Re: Working example of bi-directional asymmetric ALTQ + NAT ruleset?
On Tue, January 11, 2011 1:35 am, Bonnie Packet wrote: I have an 12mbit down/1mbit up ADSL connection, an OpenBSD router- firewall, and several Net-hungry roommates connecting through it. So...I want to give each roomie a guaranteed bandwidth allotment, but not let them hog the ADSL pipe in either direction, upstream or downstream. I'm trying to wrap my head around how it's possible - if at all! - to set up altq directives both ways, with different sets queues and bandwidth limits, AND do NATting at the same time. The problem with trying to throttle incoming bandwidth is that no matter what you do, you have already received the packets. As long as your internal network is faster than the external network feeding it, throttling incoming is useless: you've got the packets, putting them on your internal network won't slow things down, and dropping them will only cause them to get retransmitted. (And use up *more* of your already-scarce external connection.) Shaping outgoing traffic is useful because you can control which packets make it to the external chokepoint. Coming in, they've already made it through. Daniel T. Staal --- This email copyright the author. Unless otherwise noted, you are expressly allowed to retransmit, quote, or otherwise use the contents for non-commercial purposes. This copyright will expire 5 years after the author's death, or in 30 years, whichever is longer, unless such a period is in excess of local copyright law. ---
Re: Working example of bi-directional asymmetric ALTQ + NAT ruleset?
On 01/11/2011 09:23:48 AM, Daniel Staal wrote: On Tue, January 11, 2011 1:35 am, Bonnie Packet wrote: The problem with trying to throttle incoming bandwidth is that no matter what you do, you have already received the packets. As long as your internal network is faster than the external network feeding it, throttling incoming is useless: you've got the packets, putting them on your internal network won't slow things down, and dropping them will only cause them to get retransmitted. (And use up *more* of your already-scarce external connection.) It is oft-repeated that you can't throttle inbound traffic, and while true in some special cases, not true in most real-world use-cases. If all your traffic was UDP, then yes, your argument holds. But most real-world traffic, especially that which is bulky, is TCP -- which contains it's own throttling mechanism to adjust to downstream changes in bandwidth. So, the sender will slow down transmission if you drop packets. There are may proofs that throttling TCP works, starting with the original paper (Van Jacoson) in 1988 through to the many products today that _do_ manage to reserve enough inbound bandwidth that, e.g., VOIP works reliably. I once promised on this list to setup a test environment and re-prove it but have never gotten around to it and it hardly seems worth bothering. Exactly how well TCP throttling works and how to best make it work in the real world is still up for grabs, but it can be made good enough. It may or may be able to be made to work in the future as there seem to be increasing problems with 'bufferbloat', where equipment manufacturers decide to buffer TCP, which breaks TCP's congestion control and introduces the very latency that throttling is (usually) trying to eliminate. Still, if you're willing to waste enough bandwidth they'll probably always be a way to reserve a bit of unused inbound bandwidth for low-latency needs. Congestion Avoidance and Control. ∗. Van Jacobson†. Lawrence Berkeley Laboratory. Michael J. Karels‡. University of California at Berkeley. November, 1988 http://ee.lbl.gov/papers/congavoid.pdf rfc1323 TCP Extensions for High Performance http://tools.ietf.org/html/rfc1323 The criminal mastermind: bufferbloat! http://gettys.wordpress.com/2010/12/03/introducing-the-criminal- mastermind-bufferbloat/ Karl k...@meme.com Free Software: You don't pay back, you pay forward. -- Robert A. Heinlein
Re: Working example of bi-directional asymmetric ALTQ + NAT ruleset?
Although I respect the theoretical argument that you can't shape/ limit inbound packets, my observations agree with those of with Karl that it's simply not true in the real world. If you read my original posting, I am effectively limiting inbound traffic as far as the user is concerned (inbound =3D=3D the higher-bandwidth download direction from the external Internet). Again, that's working quite well already - whatever mechanism is being used by altq seems completely effective in doing that, at least in my setup. Bumblebees can actually fly, folks. In fact what I'm trying to do now is limit the outbound traffic from the router ( =3D=3D the lower-bandwidth upload direction, TO the external Internet) which I would think would be easier - as I control the router and can in theory ( there's that word again...) shape/limit either at the point of (a) accepting those packets from the internal, NATted network interface or (b) sending those same packets out of the router over the exernal, ADSL-connected interface. That seems very doable; the question is how to manage it simultaneously with the download direction when those packets already part of an established, stateful TCP connection that bypasses the firewall rules. So while the discussion is interesting, it's not in the least germane to the original question (a working bi-directional PF ruleset...) I'm still hoping an example of that is out there somewheresomewhere thanks /BP/ On Jan 11, 11:41=A0am, lanc...@ucolick.org (Kyle Lanclos) wrote: Karl O. Pinc wrote: There are may proofs that throttling TCP works, starting with the original paper (Van Jacoson) in 1988 through to the many products today that _do_ manage to reserve enough inbound bandwidth that, e.g., VOIP works reliably. =A0I once promised on this list to setup a test environment and re-prove it but have never gotten around to it and it hardly seems worth bothering. We have an observatory that sits on the other side of four bonded T1 links, and for political reasons, use pf to shape the traffic *only* on the observatory end of the link. We shape both inbound and outbound traffic; while the outbound traffic is shaped much, much more effectively, the shaping we perform on the inbound traffic is an improvement over no shaping at all. It is only effective for our situation if we deliberately sacrifice a fraction of our inbound bandwidth. In our case, the maximum bandwidth of the bonded T1 link is 6 megabits/second. Our pf configuration uses a maximum bandwidth of 5.5 megabits/second (limit determined empirically)= , and random early detection to drop packets. The other key factor is that the inbound streams we care about tend to use much lower bandwidth than whatever it is that's clogging up our inbound pipe. If all we cared about was raw throughput, the loss of 10% of our total bandwidth may be more of a problem than ungraceful network congestion. --Kyle
Re: Working example of bi-directional asymmetric ALTQ + NAT ruleset?
All you need to do is specify the queue to be the same name in inbound and outbound. Once you label a state/packet as part of a queue, it's sticky. If it's on the way out interface A and it has a queue named 'bob' and you've assigned it to the 'bob' queue, it'll be queued. If you create a queue on interface B called 'bob', it'll be queued that way. As such you can also specify them to have different properties, just call them the same thing. It's worked this way since at least 2005, probably before. Kelley Reynolds President Inside Systems, Inc. On Jan 11, 2011, at 3:46 PM, Bonnie Packet wrote: Although I respect the theoretical argument that you can't shape/ limit inbound packets, my observations agree with those of with Karl that it's simply not true in the real world. If you read my original posting, I am effectively limiting inbound traffic as far as the user is concerned (inbound =3D=3D the higher-bandwidth download direction from the external Internet). Again, that's working quite well already - whatever mechanism is being used by altq seems completely effective in doing that, at least in my setup. Bumblebees can actually fly, folks. In fact what I'm trying to do now is limit the outbound traffic from the router ( =3D=3D the lower-bandwidth upload direction, TO the external Internet) which I would think would be easier - as I control the router and can in theory ( there's that word again...) shape/limit either at the point of (a) accepting those packets from the internal, NATted network interface or (b) sending those same packets out of the router over the exernal, ADSL-connected interface. That seems very doable; the question is how to manage it simultaneously with the download direction when those packets already part of an established, stateful TCP connection that bypasses the firewall rules. So while the discussion is interesting, it's not in the least germane to the original question (a working bi-directional PF ruleset...) I'm still hoping an example of that is out there somewheresomewhere thanks /BP/ On Jan 11, 11:41=A0am, lanc...@ucolick.org (Kyle Lanclos) wrote: Karl O. Pinc wrote: There are may proofs that throttling TCP works, starting with the original paper (Van Jacoson) in 1988 through to the many products today that _do_ manage to reserve enough inbound bandwidth that, e.g., VOIP works reliably. =A0I once promised on this list to setup a test environment and re-prove it but have never gotten around to it and it hardly seems worth bothering. We have an observatory that sits on the other side of four bonded T1 links, and for political reasons, use pf to shape the traffic *only* on the observatory end of the link. We shape both inbound and outbound traffic; while the outbound traffic is shaped much, much more effectively, the shaping we perform on the inbound traffic is an improvement over no shaping at all. It is only effective for our situation if we deliberately sacrifice a fraction of our inbound bandwidth. In our case, the maximum bandwidth of the bonded T1 link is 6 megabits/second. Our pf configuration uses a maximum bandwidth of 5.5 megabits/second (limit determined empirically)= , and random early detection to drop packets. The other key factor is that the inbound streams we care about tend to use much lower bandwidth than whatever it is that's clogging up our inbound pipe. If all we cared about was raw throughput, the loss of 10% of our total bandwidth may be more of a problem than ungraceful network congestion. --Kyle
Re: Working example of bi-directional asymmetric ALTQ + NAT ruleset?
On 2011/01/11 12:46, Bonnie Packet wrote: the question is how to manage it simultaneously with the download direction when those packets already part of an established, stateful TCP connection that bypasses the firewall rules. the PF state is associated with queue by name - you can use the same queue name on more than one interface. in many situations doing this gives you just enough control that you can limit traffic in both directions on a single machine.
Re: Working example of bi-directional asymmetric ALTQ + NAT ruleset?
On Jan 11, 2:44=A0pm, s...@spacehopper.org (Stuart Henderson) wrote: On 2011/01/11 12:46, Bonnie Packet wrote: =A0 =A0 =A0 =A0 the question is how to manage it simultaneously with th= e download direction when those packets already part of an established, stateful TCP connection that bypasses the firewall rules. the PF state is associated with queue by name - you can use the same queue name on more than one interface. in many situations doing this gives you just enough control that you can limit traffic in both directions on a single machine. Stuart: *Ah* thanks - light bulb moment here. The only small problem that the way I want to slice the traffic up may be different in the inbound versus outbound directions (for example, prioritizing TCP acks in one direction matters a whole lot more than the other) - but as you helpfully point out, this still may give me just enough control to do what I need to. So would I have to have all the queues AND subqueues have exactly the same names, just on different interfaces? In other words, would I have to mirror them exactly (except they could have different characteristics/rates)? eg: altq on $internal_if cbq bandwidth 12Mb queue { std_wls, slow_wls, fast_wls, tcp_ack_wls } queue std_wls bandwidth 4Mb priority 1 cbq(default borrow) queue slow_wls bandwidth 1Mb priority 0 cbq(ecn) queue fast_wls bandwidth 6Mb priority 2 cbq(borrow) queue tcp_ack_wls bandwidth 1Mb priority 4 cbq(borrow) altq on $extl_if cbq bandwidth 1Mb queue { std_wls, slow_wls, fast_wls, tcp_ack_wls } queue std_wls bandwidth 300Kb priority 1 cbq(default borrow) queue slow_wls bandwidth 100Kb priority 0 cbq(ecn) queue fast_wls bandwidth 500Kb priority 1 cbq(borrow) queue tcp_ack_wls bandwidth 100Kb priority 3 cbq(borrow) (Note different queue sizes and priorities on external_if vs internal_if queues...) If this way works, it's not quite ideal, but it would certainly be Good Enough. /BP/