Hi Experts!
After a long self imposed absence I’m back…and as confused as ever. I’m trying
to use cake’s diffserv/dscp interpretation abilities with my Internet facing
wan link on openwrt.
The chances of getting DSCP across the Internet unmolested are nill, but I can
use dscp codes within my own LAN/Router to select relevant tins and hence
bandwidth/latency limits within cake. e.g. to de-prioritise certain host/port
combinations (bittorrent into bulk)
egress traffic is ‘easy’, simply use an iptables rule e.g. as produced by fw3
on openwrt:
/etc/config/firewall:
config rule
option name 'Bittorrent server DSCP BULK'
option target 'dscp'
option src 'lan'
option family 'ipv4'
option src_ip '192.168.218.12'
option dest '*'
option set_dscp 'CS1'
option src_port '6989'
option proto 'tcp udp'
Produces:
iptables -t mangle -A PREROUTING -p tcp -s 192.168.218.12/255.255.255.255 -m
tcp --sport 6989 -m comment --comment "!fw3: Bittorrent server DSCP BULK" -j
DSCP --set-dscp 0x08
iptables -t mangle -A PREROUTING -p udp -s 192.168.218.12/255.255.255.255 -m
udp --sport 6989 -m comment --comment "!fw3: Bittorrent server DSCP BULK" -j
DSCP --set-dscp 0x08
root@Router:~# iptables -t mangle -L
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
DSCP tcp -- Waldorf.lan.darbyshire-bryant.me.uk anywhere
tcp spt:6989 /* !fw3: Bittorrent server DSCP BULK */ DSCP set 0x08
DSCP udp -- Waldorf.lan.darbyshire-bryant.me.uk anywhere
udp spt:6989 /* !fw3: Bittorrent server DSCP BULK */ DSCP set 0x08
So that’s the egress (upload) sorted out ‘easy’. Of course the tcp ack packets
on the return (ingress) path aren’t classified as BULK nor is any bittorrent
download traffic.
Ingress is harder and my knowledge goes flaky at this point too. The qdisc is
running at a point pre de-NAT, this was the ‘flaw’ with host fairness in the
NAT world before the ’nat’ keyword. Similarly I also think that any rules to
change DSCP at this point have also happened too late for cake to notice. Then
Toke shone a light and gave Cake tc filter knowledge and so for a while I had
the following in a tweaked ‘layer_cake.qos’ script:
MAJOR=$( tc qdisc show dev $DEV | head -1 | awk '{print $3}' )
# catch bittorrent incoming
$TC filter add dev $DEV parent $MAJOR protocol ip u32 \
match ip dport 6989 0xffff action skbedit priority ${MAJOR}1
This catches any traffic destined to port 6989 and tells cake to put it into
the Bulk Tin. So I have a solution that is putting traffic coming from my
bittorrent host on port 6989 or going to port 6989 into the bulk tin. It’s not
exactly scaleable though.
Then I saw someone using connmark [1] and I’ve subsequently tweaked that into
my own version and it appears to work but… I don’t really know/understand why -
hence this email!
One of the things that openwrt’s sqm-scripts does to enable ingress QoS is to
redirect ingress traffic on the WAN interface to an ifb interface…and you put
the qdisc on the egress of the ifb to effectively create an ingress qdisc. The
mirror/redirect function is implemented by ‘action mirred redirect dev
target_dev’. Linux/TC also allows you to restore connection tracking marks at
this stage as well, hence the following line:
# redirect all IP packets arriving in $IFACE to ifb0
# and restore connmark
$TC filter add dev $IFACE parent ffff: protocol all prio 10 u32 \
match u32 0 0 flowid 1:1 action connmark action mirred egress redirect dev
$DEV
With the connmarks restored TC filters can be told to ‘do stuff’ based on those
marks, in this case edit the skb’s priority field so now Cake stuffs the packet
into the selected tin(1/3):
MAJOR=$( tc qdisc show dev $DEV | head -1 | awk '{print $3}' )
$TC filter add dev $DEV parent $MAJOR protocol ip \
handle 0x01 fw action skbedit priority ${MAJOR}1 #handle mark 0x01
$TC filter add dev $DEV parent $MAJOR protocol ip \
handle 0x03 fw action skbedit priority ${MAJOR}3 #handle mark 0x03
As I understand it, this isn’t changing the DSCP field on the packet itself but
just an skb priority that Cake understands (something to do with Major numbers
matching)
So now all we have to do is mark our connections as required. And here’s where
my knowledge trouble really starts:
# Configure iptables chain to mark packets
ipt -t mangle -N QOS_MARK_${IFACE}
# If packets have DSCP 8 then mark that connection so the response packets are
also sent to bulk
ipt -t mangle -A QOS_MARK_${IFACE} -p tcp -m dscp --dscp 8 -j MARK
--set-mark 0x01/0xff
ipt -t mangle -A QOS_MARK_${IFACE} -p udp -m dscp --dscp 8 -j MARK
--set-mark 0x01/0xff
#kdb high priority marking
ipt -t mangle -A QOS_MARK_${IFACE} -p tcp -m dscp --dscp 0x30 -j MARK
--set-mark 0x03/0xff
ipt -t mangle -A QOS_MARK_${IFACE} -p udp -m dscp --dscp 0x30 -j MARK
--set-mark 0x03/0xff
This makes sense so far, basically anything that has a matching dscp already,
set the corresponding connmark.
Then the original script does this:
# You can add more rules here. For example, to mark incomming connections on
port 9999 tcp/udp
# ipt -t mangle -A QOS_MARK_${IFACE} -i $IFACE -p tcp --dport 9999 -j MARK
--set-mark 0x01/0xff
# ipt -t mangle -A QOS_MARK_${IFACE} -i $IFACE -p udp --dport 9999 -j MARK
--set-mark 0x01/0xff
I’m confused at this point - see later.
# save the connmark - will be 0x00 unless any of the above changed it
ipt -t mangle -A QOS_MARK_${IFACE} -j CONNMARK --save-mark
That makes sense, I get that.
And finally after all that effort of making the chain, we’d better actually use
it:
# Send unmarked connections to the marking chain
ipt -t mangle -A PREROUTING -i $IFACE -m mark --mark 0x00/0xff -g
QOS_MARK_${IFACE}
ipt -t mangle -A POSTROUTING -o $IFACE -m mark --mark 0x00/0xff -g
QOS_MARK_${IFACE}
And so, finally, we come to my point of confusion and my question: How/Does
setting a firewall mark on ingress actually work because I didn’t think the
prerouting mangle table would have run before tc’s action connmark action
mirred.
Somehow it does appear to be working but I genuinely don’t understand how say
an unsolicited ingress UDP packet (from a bittorrent) would have been marked.
Sorry for the ‘War & Peace’ length. I’ve tried reading iptables tutorials and
a tc filters tutorial [2] but I can’t quite put it all together.
[1] https://github.com/dtaht/sch_cake/issues/97#issuecomment-412248518
[2] http://linux-ip.net/gl/tc-filters/tc-filters.html
Cheers,
Kevin D-B
012C ACB2 28C6 C53E 9775 9123 B3A2 389B 9DE2 334A
_______________________________________________
Cake mailing list
[email protected]
https://lists.bufferbloat.net/listinfo/cake
