On 03/04/2013 10:17 PM, Martin Pelikan wrote:
Hello. Thanks for your reply.
I need to guaranty bandwidth for selected host (abu, ali) and
pass all other traffic to bulk queue, but I have a lot of filter rules
and don't know how to do it. I have applied queues to some pass rules
and lost connection to Internet and to ssh (22555)
Follow notes inline, in the config.
My biggest advice (I've done the same mistake so many times myself) in
building a firewall ruleset is to go one step at a time. Don't try to
write the whole ruleset all at once, and then loading it and expecting
it to work right away.
The same applies to queueing. Add two queues, default one big enough,
start using them both and observe "systat queues 1". If it worked, go
and add another one, and so forth.
If you've lost your connectivity to ssh, first find out which step in
the process did it. pflog(4) is quite handy (match ... log ...).
table<tlv_qnap> { 192.168.2.200 }
table<tlk_proxmox> { 192.168.5.201 }
table<tlv_proxmox> { 192.168.2.201 }
table<tlv_mentor> { 192.168.2.205 }
table<tlv_bugzilla> { 192.168.2.206 }
table<macintosh> { 192.168.5.73 }
table<ogo> { 192.168.5.36 }
table<msn> { 192.168.5.44 }
table<sma> { 192.168.5.210 }
table<presentation> { 192.168.5.13 }
Actually, I believe creating tables for just one host can be wasteful of
resources (if you're planning to add hosts dynamically to them, then it
is okay). pfctl(8) automatically creates tables when one rule appears
seven or more times but just with one address changed. In cases of a
single host, macros are better, because the kernel sees directly that
one IP adddress and doesn't have to look it up in a table (which is a
different memory location that doesn't have to be present in CPU caches
and therefore consuming more CPU time).
But on 20 Mbit/s gateway CPU power shouldn't bother you.
table<private> { 0.0.0.0/8, 10.0.0.0/8, 14.0.0.0/8, \
127.0.0.0/8, 128.0.0.0/16, 169.254.0.0/16, \
172.16.0.0/12, 191.255.0.0/16, 192.0.2.0/24, \
192.168.0.0/16, 240.0.0.0/4, 255.255.255.0/24 }
One, you're missing 100.64/10, which is the new CGN private range.
Two, such a table be better marked const, so you accidentally don't add
something unexpected in it.
Three, 128.0/16 has some allocated bits in it. There are lots of books
suggesting people block "martian" IPv4 ranges (the valid ones being
0.0.0.0/8 and few others), but some of them have been allocated since
the books were released.
You may want to read http://tools.ietf.org/html/rfc5735 .
block quick proto tcp flags /S
block quick proto tcp flags A/A
I've seen people being told that playing with rules to various TCP flag
combinations usually leads to the firewall misbehaving and that pf(4) is
doing most of the sanity checks already by itself. Are you sure you
really need these rules for anything in particular?
Did your internet connection work without them?
altq on $ext_if hfsc bandwidth $upstream queue { root_out }
queue root_out on $ext_if bandwidth 100% hfsc {ack, dns, manager, bulk}
queue dns on $ext_if priority 7 bandwidth 5% qlimit 500 hfsc (realtime 5%)
queue ack on $ext_if priority 6 bandwidth 10% qlimit 500 hfsc (realtime 10%)
queue manager on $ext_if priority 5 bandwidth 20% qlimit 500 hfsc
(realtime 10% upperlimit 95%)
queue bulk on $ext_if priority 1 bandwidth 40% qlimit 500 hfsc (default,
red realtime 20% upperlimit 95%)
altq on $int_if hfsc bandwidth $downstream queue { root_in}
queue root_in on $int_if bandwidth 100% hfsc {ack, dns, manager, bulk}
queue dns on $int_if priority 7 bandwidth 5% qlimit 500 hfsc (realtime 5%)
queue ack on $int_if priority 6 bandwidth 10% qlimit 500 hfsc (realtime 10%)
queue manager on $int_if priority 5 bandwidth 20% qlimit 500 hfsc
(realtime 10% upperlimit 95%)
queue bulk on $int_if priority 1 bandwidth 40% qlimit 500 hfsc (default,
red realtime 20% upperlimit 95%)
It occurs to me these two are exactly the same. I think you can make it
a lot shorter by writing it at once, like so:
altq on $ext_if hfsc bandwidth $upstream queue { ack dns manager bulk }
altq on $int_if hfsc bandwidth $downstream queue { ack dns manager bulk }
queue ack bandwidth 10% qlimit 500 priority 6 hfsc(realtime 10%)
queue dns bandwidth 5% qlimit 500 priority 7 hfsc(realtime 5%)
queue manager bandwidth 20% qlimit 500 priority 5 hfsc(...)
...
And it will create two of each of them automatically (you can check
pfctl -vvsq if it matched your expectations). You can always
differentiate them per interface if you want. But keeping the file
minimal in size is good for readability after a time period when you
forget about what you did. Also qlimit of 500 is a little too high
(I use 150 on 200+ Mbit/s 50kpps gateways and it is more than enough).
#in
pass in on $ext_if inet proto tcp from any to em1 port 22555
This alone should match every time you connect. Also note the rule is
"to em1 port 22555", which means you can only connect to that TCP port
ONLY on the addresses that were on em1 at the time the config was loaded.
"to (em1) port 22555" is handy for DHCP client on em1, and "to self port
22555" (or "to (self) port 22555", which is slower but better to read in
pfctl -sr) is probably what you wanted. See 'self' keyword in pf.conf(5).
Every rule has counters counting packets and bytes it matched. "systat
rules 1" or "pfctl -vvsr" will tell you and you'll get your SSH working.
pass in on $int_if inet proto tcp from {<ali>,<abu> } to any port
$portstuff queue manager
pass in on $int_if inet proto tcp from<tlk_lan> to any port $portstuff
queue bulk
I obviously don't know your situation, but almost all the time (and if a
queue "manager" is for your human manager ;-)) providing second queue
for priorized traffic is very helpful, especially on low bandwidth easy
to clog up links, because your TCP ACKs won't wait in the queue and
interactive traffic such as ssh(1) marked by lowdelay ToS will be better
responding.
So "pass in ... queue (manager, ack)"
Didn't read the rest, but that should work.
--
Martin Pelikan
Hello Martin, thanks again for you comments.
I have carefully checked rules again, but without allying your notes and
now it works.
What I don't fully understand - it's a definition of *bandwidth* in
child queue:
queue manager on $int_if priority 5 bandwidth 20% qlimit 500 hfsc
(realtime 10% upperlimit 95%)
because *realtime* guarantee minimal bandwidth and *upperlimit*
guarantee maximum bandwidth.
So bandwidth may be equals any value or not?
QUEUE BW SCH PRIO PKTS BYTES DROP_P
DROP_B QLEN BORROW SUSPEN P/S B/S
root_em1 19M hfsc 0 0 0 0
0 0 0 0
root_out 19M hfsc 0 0
0 0 0 0 0
dns 969K hfsc 7 44088 4014298
0 0 0 0.6 75
ack 1939K hfsc 6 3553970 235418K
0 0 0 58 3393
manager 3879K hfsc 5 15889 5096767
0 0 0 0 0
bulk 7759K hfsc 2793002 816322K 5
6930 0 36 10769
root_em0 19M hfsc 0 0 0 0
0 0 0 0
root_in 19M hfsc 0 0
0 0 0 0 0
dns 969K hfsc 7 0 0
0 0 0 0 0
ack 1939K hfsc 6 1204458 83360616
0 0 0 24 1445
manager 3879K hfsc 5 40033 28603739
0 0 0 0 0
bulk 7759K hfsc 6854076 6902931K 291
417072 0 87 68120
# $OpenBSD: pf.conf,v 1.50 2011/04/28 00:19:42 mikeb Exp $
# See pf.conf(5) for syntx and examples.
# Remember to set net.inet.ip.forwarding=1 and/or net.inet6.ip6.forwarding=1
# in /etc/sysctl.conf if packets are to be forwarded between interfaces.
ext_if = "em1"
#wifi_if = "rum0"
int_if = "em0"
upstream="19.4Mb"
downstream="19.4Mb"
portstuff = "{ smtps, 5190, submission, pop3, pop3s, imap, imaps, www,
https, 1863, 1935, 3322, 3389, 5222, 5900, 5938, 8200 }"
#portstuffwww = "{ smtps, 445, 5190, submission, pop3, pop3s, imap,
imaps, www, https, 1863, 1935, 3322, 3389, 5222, 9100 }"
table <firewall> const { self }
table <tlk_lan> { 192.168.5.0/24 }
#table <tlk_wifi> { 192.168.55.0/24 }
table <tlv_lan> { 192.168.2.0/24 }
table <tlv_wifi> { 192.168.22.0/24 }
table <tlv_gw> { x.x.x.x }
table <tlv_gw_int> { 192.168.2.80 }
table <admin> { 192.168.5.1, 192.168.5.20 }
table <dns> { x.x.x.x, x.x.x.x, 8.8.8.8, x.x.x.x }
table <tlk_vpn> { 192.168.99.0/24 }
table <tlv_vpn> { 192.168.88.0/24 }
table <pptp_vpn> { 192.168.66.0/24 }
table <tlk_redmine> { 192.168.5.245 }
table <tlv_redmine> { 192.168.2.245 }
table <tlk_mysql> { 192.168.5.248 }
table <tlv_mysql> { 192.168.2.248 }
table <tlk_scm> { 192.168.5.251 }
table <tlv_scm> { 192.168.2.251 }
table <tlv_twiki> { 192.168.2.247 }
table <tlk_lic> { 192.168.5.246 }
table <tlv_lic> { 192.168.2.246 }
table <tlk_qnap> { 192.168.5.200 }
table <tlv_qnap> { 192.168.2.200 }
table <tlk_proxmox> { 192.168.5.201 }
table <tlv_proxmox> { 192.168.2.201 }
table <tlv_mentor> { 192.168.2.205 }
table <tlv_bugzilla> { 192.168.2.206 }
table <macintosh> { 192.168.5.73 }
table <ogo> { 192.168.5.36 }
table <msn> { 192.168.5.44 }
table <sma> { 192.168.5.210 }
table <presentation> { 192.168.5.13 }
table <ali> { 192.168.5.11, 192.168.5.51 }
table <abu> { 192.168.5.52, 192.168.5.53 }
table <psu> { 192.168.5.17, 192.168.5.50 }
table <agcoclient> { 192.168.5.15, 192.168.5.32, 192.168.5.34, \
192.168.5.35, 192.168.5.41, 192.168.5.42, 192.168.5.49,
192.168.5.72 }
table <agco> { x.x.x.x, x.x.x.x }
table <private> { 0.0.0.0/8, 10.0.0.0/8, 14.0.0.0/8, \
127.0.0.0/8, 128.0.0.0/16, 169.254.0.0/16, \
172.16.0.0/12, 191.255.0.0/16, 192.0.2.0/24, \
192.168.0.0/16, 240.0.0.0/4, 255.255.255.0/24 }
table <bruteforce> persist
#table <advertisement> file "/etc/advertisement"
table <spamd-white> persist
table <spamd> persist
#table <spamd-bypass> file "/etc/mail/spamd.bypass"
#table <spamd-black> file "/etc/mail/spamd.black"
set skip on { lo, enc0 }
set loginterface em1
set timeout { frag 20, tcp.established 3600 }
set block-policy return
antispoof quick for { em1 }
match in all scrub (no-df)
anchor "ftp-proxy/*"
#nat
match out on $ext_if inet proto tcp from { <tlk_lan>, <tlk_wifi>,
<pptp_vpn> } to any nat-to em1
match out on $ext_if inet proto udp from { <tlk_lan>, <tlk_wifi> } to
<agco> nat-to em1
match out on $ext_if inet from <admin> to any nat-to em1
match out on $ext_if inet proto gre from <sma> to any nat-to em1
match out on $ext_if inet proto tcp from <sma> to any port pptp nat-to em1
#rdr
match in on $ext_if inet proto tcp from any to em1 port 3690 rdr-to
<tlk_scm> port www
match in on $ext_if inet proto tcp from any to em1 port { 16881, 65001 }
rdr-to <admin>
match in on $ext_if inet proto tcp from any to em1 port 65002 rdr-to
<presentation>
match in on $ext_if inet proto tcp from any to em1 port 65003 rdr-to <ali>
match in on $ext_if inet proto tcp from any to em1 port 65004 rdr-to <abu>
match in on $ext_if inet proto udp from any to em1 port 27015 rdr-to
192.168.5.244
match in on $ext_if inet proto tcp from any to em1 port 8080 rdr-to
192.168.5.244 port www
#block in quick on $int_if from any to <advertisement>
block quick proto tcp flags /S
block quick proto tcp flags A/A
block in quick on $ext_if from { <bruteforce>, <private>, <spamd-black>
} to any
block out quick on $ext_if from any to <private>
#block in quick on $int_if inet proto tcp from { !<twikimail>,
!<twikitest> } to any port smtp
block all
#queuening
altq on $ext_if hfsc bandwidth $upstream queue { root_out }
queue root_out on $ext_if bandwidth 100% hfsc {ack, dns, manager, bulk}
queue dns on $ext_if priority 7 bandwidth 5% qlimit 50 hfsc (realtime 5%)
queue ack on $ext_if priority 6 bandwidth 10% qlimit 50 hfsc (realtime 10%)
queue manager on $ext_if priority 5 bandwidth 20% qlimit 500 hfsc
(realtime 10% upperlimit 95%)
queue bulk on $ext_if priority 1 bandwidth 40% qlimit 500 hfsc (default,
red realtime 20% upperlimit 95%)
altq on $int_if hfsc bandwidth $downstream queue { root_in }
queue root_in on $int_if bandwidth 100% hfsc {ack, dns, manager, bulk}
queue dns on $int_if priority 7 bandwidth 5% qlimit 50 hfsc (realtime 5%)
queue ack on $int_if priority 6 bandwidth 10% qlimit 50 hfsc (realtime 10%)
queue manager on $int_if priority 5 bandwidth 20% qlimit 500 hfsc
(realtime 10% upperlimit 95%)
queue bulk on $int_if priority 1 bandwidth 40% qlimit 500 hfsc (default,
red realtime 20% upperlimit 95%)
#in
pass in on $ext_if inet proto tcp from any to em1 port 22555
pass in on $ext_if proto esp from <tlv_gw> to em1
pass in on $ext_if proto gre from any to em1
pass in on $ext_if inet proto tcp from any to em1 port pptp modulate state
pass in on $ext_if inet proto udp from any to em1 port 1194
#pass in on $ext_if inet proto tcp from any to em1 port { www, https }
synproxy state
#pass in on $ext_if inet proto tcp from any to <tlk_redmine> port { www,
https }
pass in on $ext_if inet proto tcp from any to <tlk_scm> port www
synproxy state
pass in on $ext_if inet proto tcp from any to <admin> port { 16881, 65001 }
pass in on $ext_if inet proto tcp from any to <presentation> port 65002
pass in on $ext_if inet proto tcp from any to <ali> port 65003
pass in on $ext_if inet proto tcp from any to <abu> port 65004
pass in on $ext_if inet proto udp from any to 192.168.5.244 port 27015
pass in on $ext_if inet proto tcp from any to 192.168.5.244 port 80
synproxy state
#pass in on $ext_if inet proto tcp from any to em1 port ftp
pass in quick on { tun1, tun2, tun3 } all
pass in quick on $int_if inet from <admin> to any queue (manager, ack)
pass in quick on $int_if inet proto udp from any to any port tftp
pass in quick on $int_if inet proto udp from any to <firewall> port bootps
#pass in quick on $int_if inet proto tcp from <tlk_lan> to x.x.x.x port www
#relayd tweak
#pass in quick on $int_if inet proto tcp from <tlk_wifi> to { <twiki>,
<bugzilla>, <tlv_scm>, <tlv_redmine> } port www
#pass in quick on $int_if inet proto tcp from <tlk_lan> to { <twiki>,
<bugzilla>, <tlv_mysql>, <tlv_scm>, <tlv_redmine> } port www
#pass in quick on $int_if inet proto tcp from 192.168.5.1 to any port
www divert-to 127.0.0.1 port 8080
#pass in quick on $wifi_if inet proto tcp from <tlk_wifi> to any port
www divert-to 127.0.0.1 port 8080
#pass in quick on $int_if inet proto tcp from <tlk_lan> to any port www
divert-to 127.0.0.1 port 8080
pass in on $int_if inet proto icmp icmp-type echoreq
pass in on $int_if inet proto tcp from <tlk_lan> to any port ssh
pass in on $int_if inet proto tcp from <tlk_lan> to $int_if port 22555
pass in on $int_if inet proto { udp, tcp } from <tlk_lan> to $int_if
port { ntp, domain }
pass in on $int_if inet proto tcp from { <tlk_qnap>, <tlk_proxmox> } to
em0 port 3551
pass in on $int_if inet proto tcp from <tlk_lan> to port ftp divert-to
127.0.0.1 port 8021
pass in on $int_if inet proto tcp from { <tlk_scm> } to any port { smtp,
submission, www, https }
pass in on $int_if inet proto tcp from <tlk_qnap> to <tlv_qnap>
pass in on $int_if inet proto tcp from <tlk_lan> to <tlv_qnap> port 445
pass in on $int_if inet proto { udp, tcp } from <tlk_lan> to <tlv_mentor>
pass in on $int_if inet proto { tcp, udp } from <tlk_lan> to <tlv_lic>
pass in on $int_if inet proto tcp from <tlk_lan> to <tlv_mysql> port mysql
pass in on $int_if inet proto tcp from <tlk_lan> to <tlv_lan> port { 23,
8001 }
pass in on $int_if inet proto tcp from { <ali>, <abu> } to any port
$portstuff queue (manager, ack)
pass in on $int_if inet proto tcp from <tlk_lan> to any port $portstuff
queue (bulk, ack)
pass in on $int_if inet proto gre from <sma> to any
pass in on $int_if inet proto tcp from <sma> to any port pptp
pass in on $int_if inet proto tcp from <psu> to any port 1194
pass in on $int_if inet proto tcp from { <msn>, 192.168.5.59 } to
<tlv_gw_int> port smtp
pass in on $int_if inet proto { tcp, udp } from { <msn>, 192.168.5.78 }
to { 192.168.2.195, 192.168.2.52 }
pass in on $int_if inet proto udp from <tlk_lan> to <agco>
pass in on tun0 inet proto { tcp, udp } from <tlk_vpn> to { <tlv_lan>,
<tlv_wifi>, <tlk_lan> }
#out
pass out on $ext_if proto esp from em1 to <tlv_gw>
pass out on $ext_if proto gre from em1 to any
pass out on $ext_if inet proto udp from any to any port domain queue dns
pass out on $ext_if inet proto udp from em1 to any port ntp
pass out on $ext_if inet proto tcp from em1 to any port { ftp, ssh,
smtp, 1194, pptp, 444, 22555, >49151, 2710 }
pass out on $ext_if inet proto tcp from em1 to any port $portstuff
pass out on $ext_if inet proto udp from any to <agco>
pass out quick on $int_if inet proto udp from <firewall> to any port {
bootpc, ntp }
pass out on $int_if inet proto icmp icmp-type echoreq
#pass out on $wifi_if inet proto tcp from { <tlk_lan>, <tlk_vpn>,
<tlv_lan>, <tlv_wifi> } to <tlk_wifi> port { 3389, 5900 }
pass out on $int_if inet proto tcp from any to <tlk_scm> port { www, 3690 }
pass out on $int_if inet proto tcp from any to <tlk_lic>
pass out on $int_if inet proto tcp from any to <admin> port { 16881, 65001 }
pass out on $int_if inet proto tcp from any to <presentation> port 65002
pass out on $int_if inet proto tcp from any to <ali> port 65003
pass out on $int_if inet proto tcp from any to <abu> port 65004
pass out on $int_if inet proto udp from any to 192.168.5.244 port 27015
pass out on $int_if inet proto tcp from any to 192.168.5.244 port www
pass out on $int_if inet proto tcp from 192.168.2.195 to <tlk_lan>
pass out on $int_if inet proto tcp from any to <tlk_mysql> port mysql
pass out on $int_if inet proto tcp from <tlk_vpn> to <macintosh> port 5900
pass out on $int_if inet proto tcp from { <tlv_qnap>, 192.168.66.102 }
to <tlk_qnap>
pass out on $int_if inet proto { tcp, udp } from { 192.168.2.195,
192.168.2.52 } to { <msn>, 192.168.5.78 }
pass out on $int_if inet proto tcp from { <tlv_lan>, <tlv_wifi>,
<tlk_vpn>, <tlv_vpn>, <pptp_vpn> } to { <tlk_scm> } port www
pass out on $int_if inet proto tcp from { <tlk_vpn>, <tlv_lan>,
<tlv_wifi>, <tlv_vpn> } to <tlk_qnap> port 445
pass out on $int_if inet proto tcp from { <tlk_vpn>, <tlv_lan>,
<tlv_wifi> } to <tlk_lan> port { 3389, 8554, 9100, www, https }
#pass out on $int_if inet proto tcp from any to <tlk_qnap> port ftp user
proxy
pass out on $int_if inet proto tcp from any to <tlk_lan> port ssh
pass out on $int_if inet proto tcp from <firewall> to any port ssh
