The TOS class isn't (and can't be) used to match packets to the state. Once you have created state from a packet with one TOS class, other packets with the same src/dest ip/port match this state even if the class is different. (It has to be this way - say you are natting - you wouldn't want a different source port if some packets happen to use different TOS).
PF states include two queue names - one for normal traffic, one for "high prioroty" traffic: empty TCP ACKs and packets with TOS 'lowdelay' (0x10). So perhaps you would get somewhere by using 'match in from $work871 tos 0xB8 set tos lowdelay' to re-mark the incoming packets as 'lowdelay' and using e.g. 'set queue (vpn, vpn-fast)'. Otherwise, if you don't need NAT, disabling state tracking for these packets might be an option. Adam Gensler <open...@kristenandadam.net> wrote: >Hi all, > >I've been playing with pf for a number of months now and I've come >across a situation that I'm having trouble finding a solution for. >Specifically I'm working with the following topology: > >Internet --- OpenBSD box --- Cisco router --- other devices > >The Cisco router (a small 800 series router) is sitting behind an >OpenBSD box (version 5.1). This Cisco router has an IPsec tunnel >connected to another router out on the Internet. So, all the OpenBSD >box sees is a bunch of encrypted frames (udp port 4500). That's all >fine and dandy, that works well, no problems there. > >However, some of the devices behind the Cisco router are setting the >tos/dscp bits on their packets. I would like to be able to prioritize >those packets in pf for altq handling. So, I've created the following >rules: > >local_nets = "{ 172.28.1.0/24, 172.28.10.0/24, 172.28.11.0/24 }" >work871 = "172.28.1.3" >pass in quick inet proto udp from $work871 tos 0xB8 tag VOIP-RTP >pass in quick inet proto udp from $work871 tos 0x60 tag VOIP-SIG >pass in quick inet proto { tcp, udp } from $local_nets > >The idea here being that ingress traffic from 172.28.1.3, with the >various tos values will be tagged with a specific tag. On the egress >side I match on that tag and then apply it to a queue. That isn't >working though. It seems that PF creates a state entry first for the >overall ipsec tunnel using the third rule. After that state gets >established all subsequent packets do not evaluate the rules, even if >those packets have different tos values. This leaves me to believe that >pf isn't creating a state entry tuple that contains the tos value. > >I've confirmed the TOS bits are being carried through to the IP header >of the IPSEC packets. Here's a tcpdump of the incoming packets from my >LAN interface (vr0): > >tcpdump: listening on vr0, link-type EN10MB (Ethernet), capture size >65535 bytes >21:37:39.464357 IP (tos 0xc0, ttl 255, id 848, offset 0, flags [none], >proto UDP (17), length 144) >172.28.1.3.4500 > 1.1.1.1.4500: UDP-encap: >ESP(spi=0x32284280,seq=0x12e), length 116 >21:37:39.483350 IP (tos 0xc0, ttl 255, id 1707, offset 0, flags [none], >proto UDP (17), length 29) > 172.28.1.3.4500 > 64.102.7.50.4500: isakmp-nat-keep-alive >21:37:42.365389 IP (tos 0x0, ttl 255, id 849, offset 0, flags [none], >proto UDP (17), length 152) >172.28.1.3.4500 > 1.1.1.1.4500: UDP-encap: >ESP(spi=0x32284280,seq=0x12f), length 124 >21:37:45.465724 IP (tos 0xc0, ttl 255, id 850, offset 0, flags [none], >proto UDP (17), length 144) >172.28.1.3.4500 > 1.1.1.1.4500: UDP-encap: >ESP(spi=0x32284280,seq=0x130), length 116 >21:37:47.370081 IP (tos 0x0, ttl 255, id 851, offset 0, flags [none], >proto UDP (17), length 152) >172.28.1.3.4500 > 1.1.1.1.4500: UDP-encap: >ESP(spi=0x32284280,seq=0x131), length 124 >21:37:49.256302 IP (tos 0x60, ttl 255, id 852, offset 0, flags [none], >proto UDP (17), length 120) >172.28.1.3.4500 > 1.1.1.1.4500: UDP-encap: >ESP(spi=0x32284280,seq=0x132), length 92 > >From this it's clear that the last packet has "tos 0x60" set. However, >if I look at this particular rule, it doesn't have any matches or state >entries: > >pass in quick inet proto udp from 172.28.1.3 to any tos 0x60 keep state >tag VOIP-SIG >[ Evaluations: 34 Packets: 0 Bytes: 0 States: >0 ] > [ Inserted: uid 0 pid 13666 State Creations: 0 ] > >Instead, there's a state entry logged for this traffic under the third >rule: > >all udp 1.1.1.1:4500 <- 172.28.1.3:4500 MULTIPLE:MULTIPLE >age 00:15:50, expires in 00:00:57, 394:196 pkts, 52356:39176 bytes, >rule 37 > >Based on the above, it seems like the state entries don't include the >tos information making it impossible to properly classify traffic that >is encrypted with ipsec. The only way to differentiate various traffic >streams contained within the tunnel is via tos/dscp. This is fairly >common practice in the Enterprise routing space. I'd love to be able to >do the same thing here. > >Am I missing something obvious? I've tested OpenBSD 5.1 and (for what >its worth) FreeBSD 9.0 and also pfsense 2.1beta1, I see the same >behavior on all of them. > >Thanks in advance, >Adam