Peter is quite right, to add some examples to his suggestion;
tcpdump -nettti pflog0 <- Shows only dropped packets
tcpdump -nettti em0 <- Shows all packets on the interface, including ToS
values and VLAN ID etc.
tcpdump -nettti vlanX <- Shows only packets on the VLAN without the extra
info.
Sure you can figure out the rest..
There are also a few caveats to writing good PF QoS rules that some are not
aware off. For example the PRIO value is copied into the VLAN header as the
CoS value, but if it is an untagged VLAN the frame wont have a value as
their is no VLAN header to store it in. I.e. PRIO is only transitive for
connected VLAN subnets, beyond the nexthop you cannot control layer 2 CoS,
only layer 3 QoS is transitive.
Also PRIO is strictly speaking internal to the firewall, and it works for
both ingress and egress, whereas queuing/shaping is egress only. Best to
think of it as a priority picker or scheduler. I.e. packets get selected
from the buffers for processing based on their priority whether they are
input or output buffers (I am only 90% sure of this, so please correct me
if I am wrong).
Also common good practice assumes that you would normally want to use two
prio values; E.g.
pass quick on { $if_ext, $if_DMZ } proto { tcp, udp } from any to {
$int_ip_dns0 } port { 53 } queue (wan_web,wan_pri) set prio (2,4)
The first prio (2) is used for the payload packets in the session (ToS not
set), and the second prio (4) is used for the control packets (ACKs etc
because they have the ToS set). This again also sets the VLAN CoS correctly
for each packet type in the same session.
Another thing to be careful of is setting ToS yourself and using PRIO (and
if using queues too). For example;
match in proto tcp all scrub (no-df max-mss 1460)
match in proto { udp, icmp } all scrub (no-df max-mss 1472)
match out on { $if_ext } proto { tcp, udp } from any to { <ext_voipgateways>
} scrub (no-df max-mss 1420) set (tos ef, prio 7)
The first two lines are just housekeeping. But the third line will set the
ToS value EF on every single packet in the session (payload and ACKs) for
the VoIP traffic. This means that the later pass rules will place all voip
traffic into 'second' "queue" and second "priority".
And if you didn't spot the clue in the first example, yes, I believe state
does match returning traffic and does apply the prio to return traffic
also. But you wont see it with tcpdump unless you are using VLANs to
inspect the CoS field.
In my first example you will also notice I have only one rule that matches
traffic on both the inside and outside interfaces, so you need to make sure
you are using the same queue names on both the inside and outside
interfaces. This is done by adding the "on $if_ext" directive to your
queues. E.g.
queue ext_root on $if_ext bandwidth 800M
queue qlocal on $if_ext parent ext_root bandwidth 600M
queue local_kern on $if_ext parent qlocal bandwidth 6M min 6M burst
10M for 1000ms
queue local_pri on $if_ext parent qlocal bandwidth 60M min 60M
queue local_data on $if_ext parent qlocal bandwidth 510M min 100M
queue qwan on $if_ext parent ext_root bandwidth 190M
queue wan_rt on $if_ext parent qwan bandwidth 38M min 19M burst
38M for 5000ms
queue wan_int on $if_ext parent qwan bandwidth 19M min 9M
queue wan_pri on $if_ext parent qwan bandwidth 19M min 10M burst
25M for 2000ms
queue wan_vpn on $if_ext parent qwan bandwidth 50M min 25M
queue wan_web on $if_ext parent qwan bandwidth 19M min 10M burst
19M for 3000ms
queue wan_dflt on $if_ext parent qwan bandwidth 19M min 10M burst
19M for 5000ms
queue wan_bulk on $if_ext parent qwan bandwidth 20M max 50M default
queue trunk_root on $if_trunk bandwidth 4294M
queue qlocal on $if_trunk parent trunk_root bandwidth 4.1G
queue local_kern on $if_trunk parent qlocal bandwidth 8M min 8M
burst 8M for 1000ms
queue local_pri on $if_trunk parent qlocal bandwidth 150M min 150M
burst 200M for 2500ms
queue local_data on $if_trunk parent qlocal bandwidth 4G min 1G
queue qwan on $if_trunk parent trunk_root bandwidth 190M
queue wan_rt on $if_trunk parent qwan bandwidth 38M min 19M burst
38M for 5000ms
queue wan_int on $if_trunk parent qwan bandwidth 19M min 9M
queue wan_pri on $if_trunk parent qwan bandwidth 19M min 10M burst
25M for 2000ms
queue wan_vpn on $if_trunk parent qwan bandwidth 50M min 25M
queue wan_web on $if_trunk parent qwan bandwidth 19M min 10M burst
19M for 3000ms
queue wan_dflt on $if_trunk parent qwan bandwidth 19M min 10M burst
19M for 5000ms
queue wan_bulk on $if_trunk parent qwan bandwidth 20M max 50M
default
Another hint, if you are using VLANs to gain the benefit of using the
priority field in the VLAN header, to maintain your CoS on your layer 2 all
the way to the client, you should apply your queues to the trunk interface,
and not each VLAN.
This way you can share your total download transit bandwidth across all
internal VLANs instead of having to divide it out across the VLANs. Hope
that makes sense.
Just for some extra reading I have attached a paper that explains HFSC (the
underlying queuing algorithm).
PF Queue "bandwidth" = Link Sharing Service Curve
PF Queue "min" = Realtime Service Curve
PF Queue "burst" = Short-term realtime service curve for delay target
PF queue "for" = time 't' for burst (delay target to be achieved within
this initial period).
And I have also attached a PF Packet Flow diagram I made (it's not 100%
accurate, I need to update it but it should help).
And lastly just for fun if you are using CARP with busy layer 2 networks
where you are having problems with heartbeat delay, this solves that, again
not perfect, just ideas for you;
pass out quick proto carp keep state (no-sync) queue local_kern set (prio 7,
tos ef)
pass in quick proto carp keep state (no-sync) set (prio 7, tos ef)
pass out quick inet6 proto carp from { fe80::/10 } to { ff00::/8 } keep
state (no-sync) queue local_kern set (prio 7, tos ef)
pass in quick inet6 proto carp from { fe80::/10 } to { ff00::/8 } keep
state (no-sync) set (prio 7, tos ef)
pass quick on { $if_pfsync_dev } proto pfsync keep state (no-sync) set
(prio 7, tos ef)
Kind regards, Andy.
On Wed, Jun 15, 2016 at 11:02 AM, Peter N. M. Hansteen <pe...@bsdly.net>
wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> This is one of the cases where the best possible answer is, "tcpdump
> is your friend".
>
> You have outlined a number of test cases. It would be really useful if
> you try each one of them, and use tcpdump to record and identify the
> effects of each one. It's worth noting that tcpdump with the right
> options is able to display information such as the packets's ToS and
> which rule in the loaded PF rule set the packet matched.
>
> If you run those tests properly and report your findings, I'm sure it
> will be appreciated.
>
> - --
> Peter N. M. Hansteen, member of the first RFC 1149 implementation team
> http://bsdly.blogspot.com/ http://www.bsdly.net/ http://www.nuug.no/
> "Remember to set the evil bit on all malicious network traffic"
> delilah spamd[29949]: 85.152.224.147: disconnected after 42673 seconds.
> iQIcBAEBCAAGBQJXYSecAAoJELJiGF9h4DyeghcP/RZQeJ/4P8cj6DUoBhSw7HuZ
> q0t8fgnnyfw7ItkWGP6WayW9aT7oMfR9XdgX3jn/jFBLj8aW55K1i/v4PbXFJTkB
> yjnJ1WJN7fohVSYOYyfnjxCxw2RdGbcVUZpYkFCfIzsKPTxsuJynJyR7i6Ke8dYE
> 5FiF68oqhKq0yAiHcE91UlMVFH/v8NAy3crzkeK1yjgYK3sU5dVs0H7D/qR8Zlfe
> fmOO9SqDDcvMMn/7c6bQ9sHKBXSsHizZcf//yuQseSXv9ttsl/3XZyUEhS3fyqNt
> WKw80vNwQ7MJShOFqjn12G+j72s0kaSkiDEi93rXUZJxsoD28Vn6dyBJhcrWFtfr
> eEOwuyp82FiNabAvn3StzkKE+cAQ01Kag0hFhgwx/u1sD/9K31B9J8IiMpSIplFV
> tx4jfWBh1MjadAu3DIvHINYzEPoaju4zUY1mh840l5Wz7SpaBUyeJce0eNtA3n6Z
> pbpZQsi9mHCP7MOR2b+RvzcjFc4m5XoiLz29aMQDzeLj4GzroY9H0ramWchqbj1y
> BXKtFNgOglKIkjickdlSnzahFAf53r5T6vv1KY7Ea4Z5PP88e8OiXdcJqiuJlo0T
> c9VXE5cCy37i21ZPV4YK3LsuiCxMVuGtQ63B/OnP1kX34NVoatpZz6gcx5Y62MWA
> rsxLSEMFHSJuoJzgGF7j
> =mgmr
> -----END PGP SIGNATURE-----
>
>
--
*Download our latest free guide here
<https://www.brandwatch.com/competitive-intelligence-guide/>*
[demime 1.01d removed an attachment of type application/pdf which had a name of
A Hierarchical Fair Service Curve Algorithm.pdf]
[demime 1.01d removed an attachment of type application/pdf which had a name of
OpenBSDPFPacketFlow-Jan2016.pdf]
