Re: queueing/hfsc question
On Wed 04 Dec 2013 05:28:30 GMT, Gabor Berczi wrote:
On Dec 4, 2013, at 12:44 AM, andy wrote:
As Josh and the other guys said, you can use the same queue name on
different interfaces but they must be children.
Nope, they don't have to be, but thanks to your working example I
could deduce what's the deciding factor.
=== Every queue must have the interface defined. ===
And of course it makes sense (dammit), since there is no relation
between altq and queue statements otherwise, despite any fancy
user-specified tabulations in pf.conf :) They actually make this issue
harder to spot.
So this works:
queue _local on $ext bandwidth 2% priority 4 hfsc(upperlimit 100Mb)
queue _local on $int bandwidth 2% priority 4 hfsc(upperlimit 100Mb)
But this does not:
queue _local bandwidth 2% priority 4 hfsc(upperlimit 100Mb)
queue _local bandwidth 2% priority 4 hfsc(upperlimit 100Mb)
Thx all for the help!
Yea we were trying to tell you that from the begging ;) Maybe not as
clearly as we could have..
Anyway, if you use a rule like this one with the working altq example;
pass quick proto { tcp } from { $int_net } to any port { 80, 443 } keep
state (pflow) queue (_wan_web,_wan_pri) set prio (4,6)
I.e. one that does not have an interface on it, it will match the SYN
packet of a new session as it ingresses the firewalls internal
interface, this will create a 'state' entry with the sessions four
tuple and the queue names _wan_web and _wan_pri (for non-ToS and ToS
marked respectively).
As the packet then egress the firewall upstream, post routing, towards
the Internet etc, the matching state will be read and the packet
inserted into the queue on the egress interface with a matching name.
If there is no matching queue name on the egress interface the packet
will be inserted into the queue with the 'default' keyword.
Replies from the Internet will ingress the external interface without
queuing, but as those reply packets egress the firewall on the internal
interface they will be queued downstream there, on the queues on the
internal interface with the same name.
Run 'pftop -s 1 -v queue' on the firewall, then open a web browser on
an internal client PC behind the firewall and you should see the
counters on the two upstream queues and two downstream queues increment.
I've attached a PF flow diagram that can be very useful. I didn't draw
this so cannot take credit, if I could remember who did I'd ask them if
we could put it on the OpenBSD website...
Andy
[demime 1.01d removed an attachment of type image/png which had a name of
OpenBSD_PF_flow.png]
Re: queueing/hfsc question
On Dec 4, 2013, at 10:51 AM, Andy wrote:
Yea we were trying to tell you that from the begging ;) Maybe not
as clearly as we could have..
Yes, in traced back the thread and there were clues I should have
noticed. Oh well.
Anyway, if you use a rule like this one with the working altq example;
pass quick proto { tcp } from { $int_net } to any port { 80, 443 }
keep state (pflow) queue (_wan_web,_wan_pri) set prio (4,6)
I.e. one that does not have an interface on it, it will match the
SYN packet of a new session as it ingresses the firewalls internal
interface
And what happens if the interface is specified? Should it matter?
Run 'pftop -s 1 -v queue' on the firewall, then open a web browser
on an internal client PC behind the firewall and you should see the
counters on the two upstream queues and two downstream queues
increment.
systat queue is better :)
--
G
Re: queueing/hfsc question
On Wed 04 Dec 2013 10:12:58 GMT, Gabor Berczi wrote:
On Dec 4, 2013, at 10:51 AM, Andy wrote:
Yea we were trying to tell you that from the begging ;) Maybe not as
clearly as we could have..
Yes, in traced back the thread and there were clues I should have
noticed. Oh well.
Anyway, if you use a rule like this one with the working altq example;
pass quick proto { tcp } from { $int_net } to any port { 80, 443 }
keep state (pflow) queue (_wan_web,_wan_pri) set prio (4,6)
I.e. one that does not have an interface on it, it will match the SYN
packet of a new session as it ingresses the firewalls internal interface
And what happens if the interface is specified? Should it matter?
If the interface is specified the state created will also detail just
that interface and so packets matching that state can pass only that
same interface.
I.e. You would two rules both;
pass in on $int_if quick proto { tcp } from { $int_net } to any port {
80, 443 } keep
state (pflow) queue (_wan_web,_wan_pri) set prio (4,6)
pass out on $ext_if quick proto { tcp } from { $int_net } to any port {
80, 443 } keep
state (pflow) queue (_wan_web,_wan_pri) set prio (4,6)
First gets you into the firewall, the second gets you out of the
firewall.
The image I sent which shows the processing flow, shows how every
packet is processed twice (ingress processing, kernel processing,
egress processing) etc.
Read through the whole of this;
http://harrykar.blogspot.co.uk/2010/07/openbsd-packet-filteringpf.html
If you find the calomel site be very careful, as some of the examples
are out of date and some of the suggestions made undermine recent
improvements.
Andy
Run 'pftop -s 1 -v queue' on the firewall, then open a web browser on
an internal client PC behind the firewall and you should see the
counters on the two upstream queues and two downstream queues increment.
systat queue is better :)
Re: queueing/hfsc question
Andy a...@brandwatch.com writes: Read through the whole of this; http://harrykar.blogspot.co.uk/2010/07/openbsd-packet-filteringpf.html After a very superficial look, he mentions a lot of useful stuff but unfortunately he's not up to date with then-latest release (OpenBSD 4.7, when the NAT syntax changed) which was released about two months before the date he posted. A post that long takes a while to write, of course, but the 4.7 changes were a big deal at the time and it's a little odd that he hadn't noticed.. - Peter -- Peter N. M. Hansteen, member of the first RFC 1149 implementation team http://bsdly.blogspot.com/ http://www.bsdly.net/ http://www.nuug.no/ Remember to set the evil bit on all malicious network traffic delilah spamd[29949]: 85.152.224.147: disconnected after 42673 seconds.
Re: queueing/hfsc question
On Wed 04 Dec 2013 11:39:28 GMT, Peter N. M. Hansteen wrote: Andy a...@brandwatch.com writes: Read through the whole of this; http://harrykar.blogspot.co.uk/2010/07/openbsd-packet-filteringpf.html After a very superficial look, he mentions a lot of useful stuff but unfortunately he's not up to date with then-latest release (OpenBSD 4.7, when the NAT syntax changed) which was released about two months before the date he posted. A post that long takes a while to write, of course, but the 4.7 changes were a big deal at the time and it's a little odd that he hadn't noticed.. - Peter You are very right, it is out of date. Only the man pages are up to date.. ;) But it is a good starting point for PF concepts/fundamentals (their are many good sites out there for this).
Re: queueing/hfsc question
On Wed, Dec 04, 2013 at 12:11:42AM +, Stuart Henderson wrote:
On 2013-12-03, Ryan Freeman r...@slipgate.org wrote:
On Tue, Dec 03, 2013 at 09:10:32PM +0100, Gabor Berczi wrote:
On Dec 3, 2013, at 8:57 PM, josh Grosse wrote:
The new scheduler will be included in 5.5-release. Users of -
current now, or of 5.5 when it is released, who wish to continue
using altq syntax and an altq scheduler may do so during a
transition period, as described in the link above.
Okay. But what you just described (Child queues are able to use the
same name) doesn't work with ALTQ either. Are you certain it does
with the new sched? (that would mean that it was impossible to do
before)
altq on {$extif} bandwidth 600Kb hfsc queue { pri,http,warez }
queue pri bandwidth 10% priority 7qlimit 500 hfsc(realtime
5% ecn red)
queue http bandwidth 70% priority 5qlimit 500 hfsc(realtime
50% ecn red)
queue warez bandwidth 1% priority 0 qlimit 500 hfsc(upperlimit
90% ecn red)
altq on {$intif} bandwidth 10Kb hfsc queue { local,down }
queue local bandwidth 90% priority 7qlimit 500 hfsc(realtime
5% ecn red)
queue down bandwidth 10% { pri,http,default,warez }
queue pri bandwidth 10% priority 7qlimit 500 hfsc(realtime
5% ecn red)
queue http bandwidth 70% priority 5qlimit 500 hfsc(realtime
50% ecn red)
queue default bandwidth 10% priority 4qlimit 500 hfsc(realtime
5% ecn red)
queue warezbandwidth 1% priority 0 qlimit 500 hfsc(upperlimit
90% ecn red)
queue pri already exists on interface pppoe0
/etc/pf.conf.dual:71: errors in queue definition
queue http already exists on interface pppoe0
/etc/pf.conf.dual:72: errors in queue definition
queue warez already exists on interface pppoe0
try:
altq on {$extif} bandwidth 600Kb hfsc queue { pri,http,warez }
queue extpribandwidth 10% priority 7qlimit 500 hfsc(realtime 5%
ecn red)
queue exthttp bandwidth 70% priority 5qlimit 500 hfsc(realtime
50% ecn red)
queue extwarez bandwidth 1% priority 0 qlimit 500 hfsc(upperlimit 90% ecn
red)
altq on {$intif} bandwidth 10Kb hfsc queue { local,down }
queue local bandwidth 90% priority 7qlimit 500 hfsc(realtime 5%
ecn red)
queue down bandwidth 10% { pri,http,default,warez }
queue intpribandwidth 10% priority 7qlimit 500 hfsc(realtime 5%
ecn red)
queue inthttp bandwidth 70% priority 5qlimit 500 hfsc(realtime
50% ecn red)
queue default bandwidth 10% priority 4qlimit 500 hfsc(realtime 5%
ecn red)
queue intwarez bandwidth 1% priority 0 qlimit 500 hfsc(upperlimit 90% ecn
red)
the only thing i noticed is you're defining pri, http, and warez twice and
confusing pf.
This does not confuse PF at all, but you have to be explicity about
which interfaces they relate to.
Ah, I had once run into this but didn't realize you could be explicit about
the interface on the queue line itself.
cool! cheers :)
-ryan
Re: queueing/hfsc question
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On 12/03, Gabor Berczi wrote:
Greets dudez,
Help.
How can one properly share (on the firewall, not at application
level) the _downstream_ bandwidth between for example a HTTP
download (should get the most bandwidth) and a torrent queue (should
be reduced to the bare minimum except when no HTTP transfer is in
progress)? And at the same time, there must also be an _upstream_
queue too (the usual stuff).
I would do it like this.
altq on $INTERFACE bandwidth $INMbp/s hfsc queue { http, torrent }
queue http bandwidth 70% priority 8 hsfc ( realtime 60% )
queue torrent bandwidth 50% priority 1 hsfc ( upperlimit 95% )
Just apply these rules to appropriate statements in your pf.conf
I can manage the first with a fuckin' ugly hack, coincidentally
reducing the upstream bandwidth to less than a tenth of what it
should be (I don't want to do this). Can do the second, neat and
clean obviously. But can't do both...
--
G
iQEcBAEBAgAGBQJSnf1eAAoJEBdTBdbEk9xoTq8H/1Ai+vGJZIl0plzKU7ns4uWE
YFw7WJ2WJ3Hp5sR2uSVeijv9dT6Zxjoe5f6QdBGXIAIDVypWsIL210LboKbtjCyC
aAk+yBbviKHEp/hOBYnFf2KfzoyWZtD6g1f7ccSKT3h1awND26H2JvDtknU/J/Ig
LpjVIOme3hH76mODJdUO1VKhFEUTPZXGWM3BkCgh1t1gYeg4mWVEUW+cXSi+dHHf
bF/xKM145Rosawi7hQJmqC89JDDKITU2NgfYglPyEyYf6GRcrJ59ZowsB0Yyfj+2
S7+lMmdjyaR4PJw2GbnlrCHt/DfGXeTY+1gytQ1uE5VWbjcZA97bmngrldRtUzQ=
=Ti/N
-END PGP SIGNATURE-
Re: queueing/hfsc question
On Dec 3, 2013, at 4:48 PM, indiunix wrote:
I would do it like this.
altq on $INTERFACE bandwidth $INMbp/s hfsc queue { http, torrent }
queue http bandwidth 70% priority 8 hsfc ( realtime 60% )
queue torrent bandwidth 50% priority 1 hsfc ( upperlimit 95% )
Just apply these rules to appropriate statements in your pf.conf
Yes, but how would you specify the pass/match rules? One TCP
connection can go into only one queue, therefore the other queue will
not take effect.
--
G
Re: queueing/hfsc question
On Tue 03 Dec 2013 16:06:39 GMT, Gabor Berczi wrote:
On Dec 3, 2013, at 4:48 PM, indiunix wrote:
I would do it like this.
altq on $INTERFACE bandwidth $INMbp/s hfsc queue { http, torrent }
queue http bandwidth 70% priority 8 hsfc ( realtime 60% )
queue torrent bandwidth 50% priority 1 hsfc ( upperlimit 95% )
Just apply these rules to appropriate statements in your pf.conf
Yes, but how would you specify the pass/match rules? One TCP
connection can go into only one queue, therefore the other queue will
not take effect.
Google will find lots of examples if you look..
Including http://www.openbsd.org/faq/pf/queueing.html
To add a bit more detail to Indiunix;
altq on $extinterface bandwidth $UpMb hfsc queue { ack, http, torrent }
queue ack on $extinterface bandwidth 10% priority 6 hfsc ( realtime
10% )
queue http on $extinterface bandwidth 80% priority 4 hsfc ( realtime
60% )
queue torrent on $extinterface bandwidth 10% priority 1 hsfc (
linkshare 10%, upperlimit 95%, ecn, default )
altq on $intinterface bandwidth $DownMb hfsc queue { ack, http, torrent
}
queue ack on $intinterface bandwidth 10% priority 6 hfsc ( realtime
10% )
queue http on $intinterface bandwidth 80% priority 4 hsfc ( realtime
60% )
queue torrent on $intinterface bandwidth 10% priority 1 hsfc (
linkshare 10%, upperlimit 95%, ecn, default )
pass quick proto { tcp } from { 10.0.0.0/8 } to { !10.0.0.0/8 } port {
80, 443 } queue (http,ack) set prio (4,6)
pass quick proto { tcp } from { ext_trusted } to { int_ssh_servers
} port { ssh } queue (torrent,ack) set prio (2,5)
pass quick from { 10.0.0.0/8 } to { !10.0.0.0/8 } queue (torrent,ack)
BUT, this is using 'altq' which you shouldn't use anymore (will die in
next release). Instead you should be using the new queue structure;
http://quigon.bsws.de/papers/2012/eurobsdcon/
http://www.youtube.com/watch?v=CMv90lDHhB8
PS; This altq example is of the top of my head so will have errors but
should get you a bit further..
Google for more...
Andy
Re: queueing/hfsc question
On Dec 3, 2013, at 7:15 PM, Andy wrote: queue ack on $extinterface bandwidth 10% priority 6 hfsc ( realtime 10% ) queue ack on $intinterface bandwidth 10% priority 6 hfsc ( realtime 10% ) One queue can't be on two interfaces. queue XYZ already exists on interface pppoe0 Google for more... I use duckduckgo, and don't condone using product names or companies as verbs, sorry :) -- G
Re: queueing/hfsc question
On 2013-12-03 13:46, Gabor Berczi wrote: On Dec 3, 2013, at 7:15 PM, Andy wrote: queue ack on $extinterface bandwidth 10% priority 6 hfsc ( realtime 10% ) queue ack on $intinterface bandwidth 10% priority 6 hfsc ( realtime 10% ) One queue can't be on two interfaces. queue XYZ already exists on interface pppoe0 Child queues are able to use the same name. Here's an example, using the new queuing syntax, where the std and pig queue names are assigned to queues on different interfaces with different settings: queue up on $external_nic bandwidth 5M burst 100M for 800ms queue pig parent up bandwidth 2500K queue std parent up bandwidth 2500K default queue internal on $internal_nic bandwidth 100M queue local parent internal bandwidth 75M queue down parent internal bandwidth 25M burst 100M for 1600ms queue pig parent down bandwidth 12500K queue std parent down bandwidth 12500K default
Re: queueing/hfsc question
On Dec 3, 2013, at 8:03 PM, josh Grosse wrote: On 2013-12-03 13:46, Gabor Berczi wrote: On Dec 3, 2013, at 7:15 PM, Andy wrote: queue ack on $extinterface bandwidth 10% priority 6 hfsc ( realtime 10% ) queue ack on $intinterface bandwidth 10% priority 6 hfsc ( realtime 10% ) One queue can't be on two interfaces. queue XYZ already exists on interface pppoe0 Child queues are able to use the same name. Here's an example, using the new queuing syntax, where the std and pig queue names are assigned to queues on different interfaces with different settings: /etc/pf.conf.test:4: unknown unit M /etc/pf.conf.test:5: queue pig has no parent /etc/pf.conf.test:5: errors in queue definition /etc/pf.conf.test:6: queue std has no parent /etc/pf.conf.test:6: errors in queue definition /etc/pf.conf.test:8: unknown unit M /etc/pf.conf.test:9: queue local has no parent /etc/pf.conf.test:9: errors in queue definition /etc/pf.conf.test:10: queue down has no parent /etc/pf.conf.test:10: errors in queue definition /etc/pf.conf.test:11: queue pig has no parent /etc/pf.conf.test:11: errors in queue definition /etc/pf.conf.test:12: queue std has no parent /etc/pf.conf.test:12: errors in queue definition
Re: queueing/hfsc question
On Dec 3, 2013, at 7:15 PM, Andy wrote: Including http://www.openbsd.org/faq/pf/queueing.html This page doesn't even mention neither HFSC nor any new scheduler... -- G
Re: queueing/hfsc question
On Tue Dec 3 2013 20:32, Gabor Berczi wrote: On Dec 3, 2013, at 8:03 PM, josh Grosse wrote: On 2013-12-03 13:46, Gabor Berczi wrote: On Dec 3, 2013, at 7:15 PM, Andy wrote: queue ack on $extinterface bandwidth 10% priority 6 hfsc ( realtime 10% ) queue ack on $intinterface bandwidth 10% priority 6 hfsc ( realtime 10% ) One queue can't be on two interfaces. queue XYZ already exists on interface pppoe0 Child queues are able to use the same name. Here's an example, using the new queuing syntax, where the std and pig queue names are assigned to queues on different interfaces with different settings: /etc/pf.conf.test:4: unknown unit M /etc/pf.conf.test:5: queue pig has no parent /etc/pf.conf.test:5: errors in queue definition /etc/pf.conf.test:6: queue std has no parent /etc/pf.conf.test:6: errors in queue definition /etc/pf.conf.test:8: unknown unit M /etc/pf.conf.test:9: queue local has no parent /etc/pf.conf.test:9: errors in queue definition /etc/pf.conf.test:10: queue down has no parent /etc/pf.conf.test:10: errors in queue definition /etc/pf.conf.test:11: queue pig has no parent /etc/pf.conf.test:11: errors in queue definition /etc/pf.conf.test:12: queue std has no parent /etc/pf.conf.test:12: errors in queue definition You need to run -current for this.
Re: queueing/hfsc question
On Tue Dec 3 2013 20:40, Gabor Berczi wrote: On Dec 3, 2013, at 7:15 PM, Andy wrote: Including http://www.openbsd.org/faq/pf/queueing.html This page doesn't even mention neither HFSC nor any new scheduler... Please watch the video record of Henning's talk on the new queueing subsystem. And yes, the new queueing subsystem is not yet included in one of the releases. This will be the case with 5.5. It's in -current, though.
Re: queueing/hfsc question
On 2013-12-03 14:40, Gabor Berczi wrote: This page doesn't even mention neither HFSC nor any new scheduler... The new queueing system is HFSC, with a simpler syntax. It became available to -current users in October, per http://www.openbsd.org/faq/current.html#20131012 The new scheduler will be included in 5.5-release. Users of -current now, or of 5.5 when it is released, who wish to continue using altq syntax and an altq scheduler may do so during a transition period, as described in the link above.
Re: queueing/hfsc question
On Dec 3, 2013, at 8:57 PM, josh Grosse wrote:
The new scheduler will be included in 5.5-release. Users of -
current now, or of 5.5 when it is released, who wish to continue
using altq syntax and an altq scheduler may do so during a
transition period, as described in the link above.
Okay. But what you just described (Child queues are able to use the
same name) doesn't work with ALTQ either. Are you certain it does
with the new sched? (that would mean that it was impossible to do
before)
altq on {$extif} bandwidth 600Kb hfsc queue { pri,http,warez }
queue pri bandwidth 10% priority 7qlimit 500 hfsc
(realtime 5% ecn red)
queue http bandwidth 70% priority 5qlimit 500 hfsc
(realtime 50% ecn red)
queue warez bandwidth 1% priority 0 qlimit 500 hfsc(upperlimit
90% ecn red)
altq on {$intif} bandwidth 10Kb hfsc queue { local,down }
queue local bandwidth 90% priority 7qlimit 500 hfsc
(realtime 5% ecn red)
queue down bandwidth 10% { pri,http,default,warez }
queue pri bandwidth 10% priority 7qlimit 500 hfsc
(realtime 5% ecn red)
queue http bandwidth 70% priority 5qlimit 500 hfsc
(realtime 50% ecn red)
queue default bandwidth 10% priority 4qlimit 500 hfsc
(realtime 5% ecn red)
queue warezbandwidth 1% priority 0 qlimit 500 hfsc(upperlimit
90% ecn red)
queue pri already exists on interface pppoe0
/etc/pf.conf.dual:71: errors in queue definition
queue http already exists on interface pppoe0
/etc/pf.conf.dual:72: errors in queue definition
queue warez already exists on interface pppoe0
--
G
Re: queueing/hfsc question
On Tue, Dec 03, 2013 at 09:10:32PM +0100, Gabor Berczi wrote:
On Dec 3, 2013, at 8:57 PM, josh Grosse wrote:
The new scheduler will be included in 5.5-release. Users of -
current now, or of 5.5 when it is released, who wish to continue
using altq syntax and an altq scheduler may do so during a
transition period, as described in the link above.
Okay. But what you just described (Child queues are able to use the
same name) doesn't work with ALTQ either. Are you certain it does
with the new sched? (that would mean that it was impossible to do
before)
altq on {$extif} bandwidth 600Kb hfsc queue { pri,http,warez }
queue pri bandwidth 10% priority 7qlimit 500 hfsc(realtime
5% ecn red)
queue http bandwidth 70% priority 5qlimit 500 hfsc(realtime
50% ecn red)
queue warez bandwidth 1% priority 0 qlimit 500 hfsc(upperlimit
90% ecn red)
altq on {$intif} bandwidth 10Kb hfsc queue { local,down }
queue local bandwidth 90% priority 7qlimit 500 hfsc(realtime
5% ecn red)
queue down bandwidth 10% { pri,http,default,warez }
queue pri bandwidth 10% priority 7qlimit 500 hfsc(realtime
5% ecn red)
queue http bandwidth 70% priority 5qlimit 500 hfsc(realtime
50% ecn red)
queue default bandwidth 10% priority 4qlimit 500 hfsc(realtime
5% ecn red)
queue warezbandwidth 1% priority 0 qlimit 500 hfsc(upperlimit
90% ecn red)
queue pri already exists on interface pppoe0
/etc/pf.conf.dual:71: errors in queue definition
queue http already exists on interface pppoe0
/etc/pf.conf.dual:72: errors in queue definition
queue warez already exists on interface pppoe0
try:
altq on {$extif} bandwidth 600Kb hfsc queue { pri,http,warez }
queue extpribandwidth 10% priority 7qlimit 500 hfsc(realtime 5% ecn
red)
queue exthttp bandwidth 70% priority 5qlimit 500 hfsc(realtime 50%
ecn red)
queue extwarez bandwidth 1% priority 0 qlimit 500 hfsc(upperlimit 90% ecn red)
altq on {$intif} bandwidth 10Kb hfsc queue { local,down }
queue local bandwidth 90% priority 7qlimit 500 hfsc(realtime 5% ecn
red)
queue down bandwidth 10% { pri,http,default,warez }
queue intpribandwidth 10% priority 7qlimit 500 hfsc(realtime 5% ecn
red)
queue inthttp bandwidth 70% priority 5qlimit 500 hfsc(realtime 50%
ecn red)
queue default bandwidth 10% priority 4qlimit 500 hfsc(realtime 5% ecn
red)
queue intwarez bandwidth 1% priority 0 qlimit 500 hfsc(upperlimit 90% ecn red)
the only thing i noticed is you're defining pri, http, and warez twice and
confusing pf.
-ryan
--
G
Re: queueing/hfsc question
On 2013-12-03 15:10, Gabor Berczi wrote: Okay. But what you just described (Child queues are able to use the same name) doesn't work with ALTQ either. Are you certain it does with the new sched? (that would mean that it was impossible to do before) The PF User's guide shows a cbq example where the queue name bulk is assigned twice. Stuart Henderson mentioned the technique here: http://marc.info/?l=openbsd-pfm=129478585825865w=2 I used cbq with this method until switching to the new scheduler. The example rules that I posted in this thread (where the child queues are defined with multiple interfaces) are rules that I'm using with the new scheduler.
Re: queueing/hfsc question
On Tue, 3 Dec 2013 21:10:32 +0100, Gabor Berczi gabuc...@gabucino.be
wrote:
On Dec 3, 2013, at 8:57 PM, josh Grosse wrote:
The new scheduler will be included in 5.5-release. Users of -
current now, or of 5.5 when it is released, who wish to continue
using altq syntax and an altq scheduler may do so during a
transition period, as described in the link above.
Okay. But what you just described (Child queues are able to use the
same name) doesn't work with ALTQ either. Are you certain it does
with the new sched? (that would mean that it was impossible to do
before)
altq on {$extif} bandwidth 600Kb hfsc queue { pri,http,warez }
queue pri bandwidth 10% priority 7qlimit 500 hfsc
(realtime 5% ecn red)
queue http bandwidth 70% priority 5qlimit 500 hfsc
(realtime 50% ecn red)
queue warez bandwidth 1% priority 0 qlimit 500 hfsc(upperlimit
90% ecn red)
altq on {$intif} bandwidth 10Kb hfsc queue { local,down }
queue local bandwidth 90% priority 7qlimit 500 hfsc
(realtime 5% ecn red)
queue down bandwidth 10% { pri,http,default,warez }
queue pri bandwidth 10% priority 7qlimit 500 hfsc
(realtime 5% ecn red)
queue http bandwidth 70% priority 5qlimit 500 hfsc
(realtime 50% ecn red)
queue default bandwidth 10% priority 4qlimit 500 hfsc
(realtime 5% ecn red)
queue warezbandwidth 1% priority 0 qlimit 500 hfsc(upperlimit
90% ecn red)
queue pri already exists on interface pppoe0
/etc/pf.conf.dual:71: errors in queue definition
queue http already exists on interface pppoe0
/etc/pf.conf.dual:72: errors in queue definition
queue warez already exists on interface pppoe0
As Josh and the other guys said, you can use the same queue name on
different interfaces but they must be children. Here's the queues from one
of my firewalls;
# WAN egress Queues
altq on $if_vpls bandwidth 1500Mb hfsc queue { _local, _wan }
queue _local on $if_vpls bandwidth 2% priority 4 hfsc(upperlimit
100Mb) { _local_kernel, _local_data }
queue _local_kernel on $if_vpls bandwidth 1% priority 6
hfsc(realtime 1%, linkshare 10%)
queue _local_data on $if_vpls bandwidth 99% priority 0
hfsc(linkshare 80%)
queue _wan on $if_vpls bandwidth 98% priority 15 hfsc(linkshare 98%) {
_wan_pri, _wan_int, _wan_web, _wan_dflt }
queue _wan_pri on $if_vpls bandwidth 20% priority 6 qlimit 100
hfsc(realtime 20%, linkshare 20%)
queue _wan_int on $if_vpls bandwidth 5% priority 5 qlimit 100
hfsc(realtime 5%, linkshare 5%)
queue _wan_web on $if_vpls bandwidth 50% priority 4 qlimit 100
hfsc(realtime(10%, 1000, 1%), linkshare 50%, ecn)
queue _wan_dflt on $if_vpls bandwidth 5% priority 2 qlimit 100
hfsc(realtime(10%, 1000, 1%), linkshare 5%, ecn, default)
# LAN egress Queues
altq on $if_lan bandwidth 1500Mb hfsc queue { _local, _wan }
queue _local on $if_lan bandwidth 2% priority 4 hfsc(upperlimit 100Mb)
{ _local_kernel, _local_data }
queue _local_kernel on $if_lan bandwidth 1% priority 6
hfsc(realtime 1%, linkshare 10%)
queue _local_data on $if_lan bandwidth 99% priority 0
hfsc(linkshare 80%)
queue _wan on $if_lan bandwidth 98% priority 15 hfsc(linkshare 98%) {
_wan_pri, _wan_int, _wan_web, _wan_dflt }
queue _wan_pri on $if_lan bandwidth 10% priority 6 qlimit 100
hfsc(realtime 10%, linkshare 10%)
queue _wan_int on $if_lan bandwidth 5% priority 5 qlimit 100
hfsc(realtime 5%, linkshare 5%)
queue _wan_web on $if_lan bandwidth 60% priority 4 qlimit 100
hfsc(realtime(10%, 1000, 1%), linkshare 60%, ecn)
queue _wan_dflt on $if_lan bandwidth 5% priority 2 qlimit 100
hfsc(realtime(10%, 1000, 1%), linkshare 5%, ecn, default)
pass quick proto { tcp } from { $int_net } to any port { 80, 443 } keep
state (pflow) queue (_wan_web,_wan_pri) set prio (4,6)
WARNING; Don't set your qlimit to 500 unless you really know what you are
doing. 500 is huge!
The bugger your buckets, the longer you'll just store packets for until
their is bandwidth available to send them. This means the packet delay
variance on sessions in queues with high qlimits will be very high.
You control TCP by dropping packets, so dropping packets is not a bad
thing. ECN is much better though.. I.e. Dropping makes the sender stop
sending you so much traffic to allow your queues to clear and find the
balanced max throughput whilst keeping latency at a min..
I use qlimit 100 here as this firewall has a cluster of web crawlers
behind it and so the jitter doesn't matter. Office firewalls however which
have VoIP queues have qlimit set to 30...
Another tip, the sum of the first property to realtime cannot be greater
than 80% of the interface parent..
Anyway, learn the new queue stuff.. I still need to read all the man pages
and notes on it myself yet.
Andy
Re: queueing/hfsc question
On 2013-12-03, Ryan Freeman r...@slipgate.org wrote:
On Tue, Dec 03, 2013 at 09:10:32PM +0100, Gabor Berczi wrote:
On Dec 3, 2013, at 8:57 PM, josh Grosse wrote:
The new scheduler will be included in 5.5-release. Users of -
current now, or of 5.5 when it is released, who wish to continue
using altq syntax and an altq scheduler may do so during a
transition period, as described in the link above.
Okay. But what you just described (Child queues are able to use the
same name) doesn't work with ALTQ either. Are you certain it does
with the new sched? (that would mean that it was impossible to do
before)
altq on {$extif} bandwidth 600Kb hfsc queue { pri,http,warez }
queue pri bandwidth 10% priority 7qlimit 500 hfsc(realtime
5% ecn red)
queue http bandwidth 70% priority 5qlimit 500 hfsc(realtime
50% ecn red)
queue warez bandwidth 1% priority 0 qlimit 500 hfsc(upperlimit
90% ecn red)
altq on {$intif} bandwidth 10Kb hfsc queue { local,down }
queue local bandwidth 90% priority 7qlimit 500 hfsc(realtime
5% ecn red)
queue down bandwidth 10% { pri,http,default,warez }
queue pri bandwidth 10% priority 7qlimit 500 hfsc(realtime
5% ecn red)
queue http bandwidth 70% priority 5qlimit 500 hfsc(realtime
50% ecn red)
queue default bandwidth 10% priority 4qlimit 500 hfsc(realtime
5% ecn red)
queue warezbandwidth 1% priority 0 qlimit 500 hfsc(upperlimit
90% ecn red)
queue pri already exists on interface pppoe0
/etc/pf.conf.dual:71: errors in queue definition
queue http already exists on interface pppoe0
/etc/pf.conf.dual:72: errors in queue definition
queue warez already exists on interface pppoe0
try:
altq on {$extif} bandwidth 600Kb hfsc queue { pri,http,warez }
queue extpribandwidth 10% priority 7qlimit 500 hfsc(realtime 5%
ecn red)
queue exthttp bandwidth 70% priority 5qlimit 500 hfsc(realtime 50%
ecn red)
queue extwarez bandwidth 1% priority 0 qlimit 500 hfsc(upperlimit 90% ecn
red)
altq on {$intif} bandwidth 10Kb hfsc queue { local,down }
queue local bandwidth 90% priority 7qlimit 500 hfsc(realtime 5%
ecn red)
queue down bandwidth 10% { pri,http,default,warez }
queue intpribandwidth 10% priority 7qlimit 500 hfsc(realtime 5%
ecn red)
queue inthttp bandwidth 70% priority 5qlimit 500 hfsc(realtime 50%
ecn red)
queue default bandwidth 10% priority 4qlimit 500 hfsc(realtime 5%
ecn red)
queue intwarez bandwidth 1% priority 0 qlimit 500 hfsc(upperlimit 90% ecn
red)
the only thing i noticed is you're defining pri, http, and warez twice and
confusing pf.
This does not confuse PF at all, but you have to be explicity about
which interfaces they relate to.
Re: queueing/hfsc question
On 2013-12-03, Gabor Berczi gabuc...@gabucino.be wrote: On Dec 3, 2013, at 7:15 PM, Andy wrote: Including http://www.openbsd.org/faq/pf/queueing.html This page doesn't even mention neither HFSC nor any new scheduler... The FAQ does not cover -current, it covers the last release. (Feel free to ask about this again when 5.5 is out if the faq isn't updated though. ;)
Re: queueing/hfsc question
On Dec 3, 2013, at 9:43 PM, Ryan Freeman wrote: try: queue extpribandwidth 10% priority 7qlimit 500 hfsc (realtime 5% ecn red) queue intpribandwidth 10% priority 7qlimit 500 hfsc (realtime 5% ecn red) Yes, but as I said, only one of these can be used (can't pass into two queues). I'd like to shape both up-, and downstream sides of TCP connections. -- G
Re: queueing/hfsc question
On Dec 4, 2013, at 12:44 AM, andy wrote: As Josh and the other guys said, you can use the same queue name on different interfaces but they must be children. Nope, they don't have to be, but thanks to your working example I could deduce what's the deciding factor. === Every queue must have the interface defined. === And of course it makes sense (dammit), since there is no relation between altq and queue statements otherwise, despite any fancy user-specified tabulations in pf.conf :) They actually make this issue harder to spot. So this works: queue _local on $ext bandwidth 2% priority 4 hfsc(upperlimit 100Mb) queue _local on $int bandwidth 2% priority 4 hfsc(upperlimit 100Mb) But this does not: queue _local bandwidth 2% priority 4 hfsc(upperlimit 100Mb) queue _local bandwidth 2% priority 4 hfsc(upperlimit 100Mb) Thx all for the help! -- G
