> On 9 Mar 2019, at 14:08, Toke Høiland-Jørgensen <[email protected]> wrote: > > Kevin Darbyshire-Bryant <[email protected]> writes: > >> OK, what I am trying to do is classify incoming connections into >> relevant cake tins to impose some bandwidth fairness. e.g. classify >> bittorrent & things that are downloads into the Bulk tin, and >> prioritise stuff that is streaming video into the Video tin. Incoming >> DSCP has a) been washed and b) is unreliable anyway so is unhelpful in >> this case. iptables runs too late, so having rules to classify >> incoming stuff is pointless. > > Right, I see. > > [... snip .. ] > >> Then I recently discovered act_connmark >> (http://linux-ip.net/gl/tc-filters/tc-filters-node2.html) - the >> thinking being I could use iptables on egress to set fwmarks to >> classify a connection and have the ingress packets magically follow. >> This worked but still required 3 tc filter actions to cope with 4 >> tins: >> >> $TC filter add dev $IFACE parent $MAJOR protocol ip handle 0x01 fw action >> skbedit priority ${MAJOR}1 >> $TC filter add dev $IFACE parent $MAJOR protocol ip handle 0x03 fw action >> skbedit priority ${MAJOR}3 >> $TC filter add dev $IFACE parent $MAJOR protocol ip handle 0x04 fw action >> skbedit priority ${MAJOR}4 > > Right, so this can be replaced with the fwmark action we already added > (and that I just pushed an update to so it supports masking the value > before selecting a tin).
Yes. I’d point out the (hopefully obvious) that the flag mask needs to be one bit bigger than you might immediately think. e.g. diffserv4 needs to store 5 values (0-4), 3 bits. 0 is being used as an implied ’tin is not set, fall back to DSCP’. One could store DSCP+1 of course and use the same logic. > >> The overriding (if required) of DSCP could be done in iptables and to >> avoid going through the iptables DSCP decision/mangling for every >> packet I could use a flag within the fwmark to indicate the decision >> had previously been made and stored for this connection. > > [ ... ] > >> I’m doing 2 things. >> >> 1) Classifying traffic into tins on ingress based on the egress DSCP >> values contained in fwmarks. >> >> 2) Basing the fwmark contained DSCP on the initial packet of the >> connection, possibly after being modified once by iptables rules. > > So I tried prototyping what it would actually look like to do all this > in iptables. The result is below (in iptables-restore format). I haven't > tested it, but I believe something along the lines of this will work, > when used along with the CAKE fwmark support (setting a mask of 0xFF > when configuring CAKE). > > Now, the obvious eyesore on this is the need to replicate CAKEs diffserv > mappings in iptables rules (21 rules in this case, for the diffserv4 > mapping). As long as this only runs once per connection I don't actually > think it's much of a performance issue for normal use, but obviously > there could be pathological cases, and it's also annoying to have to do > that. > > So, first question becomes: Do you agree that the firewall rules below > would solve your use case (ignoring the ugliness of having to replicate > the diffserv parsing in iptables)? Or am I missing something? I’ve had a quick look over it and think it would work. The ugliness of doing the diffserv parsing is what prompted the idea of storing the DSCP directly and I felt the stored tin selection was effectively abstracting the diffserv field anyway. Storing the DSCP is more compatible with differing egress v ingress mappings (eg. egress diffserv4, ingress diffserv3 though I can’t really think of a use case for that) Of course using fwmark as tin number selector in cake doesn’t preclude some other mechanism of storing & recovering DSCP to/from firewall mark e.g. the previously discussed act-connmark+dscp which would help anyone who wanted to do such ‘link traversing’ DSCP shenanigans. That of course makes you happier since cake doesn’t embed itself further into conntrack. > > -Toke > > > > *mangle > :PREROUTING ACCEPT [0:0] > :APPLY-MARKS - [0:0] > :MARK-DSCP - [0:0] > :MARK-POLICY - [0:0] > > # Run on inside iface - eth0 in this case > -A PREROUTING -i eth0 -J APPLY-MARKS > > # Make sure we have the marks from conntrack > -A APPLY-MARKS -J CONNMARK --restore-mark --nfmask 0xFF --ctmask 0xFF > > # Abort if our "already set" bit is set > -A APPLY-MARKS -m mark --mark 0x80/0x80 -j RETURN > > # If a DSCP value is set, use DSCP-based marking > -A APPLY-MARKS -m dscp ! --dscp 0 -J MARK-DSCP > > # Otherwise, set our own policy > -A APPLY-MARKS -m dscp --dscp 0 -J MARK-POLICY > > # Set our "already set" bit, and store things back into conntrack > -A APPLY-MARKS -J MARK --set-mark 0x80/0x80 > -A APPLY-MARKS -J CONNMARK --save-mark --nfmask 0xFF --ctmask 0xFF > > # DSCP-based fwmark setting > # add whatever DSCP coverage you want here; the below is CAKE's diffserv4 > # fwmark offsets into CAKE's tin_order, and are 1-indexed, so tin selection > # is offset by 1, and bulk and besteffort are swapped compared to the array > # in sch_cake.c > # We assume CAKE is configured with an fwmark mask of 0x7F > > -A MARK-DSCP -m dscp --dscp 1 -J MARK --set-mark 0x3/0x7F > -A MARK-DSCP -m dscp --dscp 4 -J MARK --set-mark 0x3/0x7F > -A MARK-DSCP -m dscp --dscp 8 -J MARK --set-mark 0x1/0x7F > -A MARK-DSCP -m dscp --dscp 16 -J MARK --set-mark 0x3/0x7F > -A MARK-DSCP -m dscp --dscp 18 -J MARK --set-mark 0x3/0x7F > -A MARK-DSCP -m dscp --dscp 20 -J MARK --set-mark 0x3/0x7F > -A MARK-DSCP -m dscp --dscp 22 -J MARK --set-mark 0x3/0x7F > -A MARK-DSCP -m dscp --dscp 24 -J MARK --set-mark 0x3/0x7F > -A MARK-DSCP -m dscp --dscp 26 -J MARK --set-mark 0x3/0x7F > -A MARK-DSCP -m dscp --dscp 28 -J MARK --set-mark 0x3/0x7F > -A MARK-DSCP -m dscp --dscp 30 -J MARK --set-mark 0x3/0x7F > -A MARK-DSCP -m dscp --dscp 34 -J MARK --set-mark 0x4/0x7F > -A MARK-DSCP -m dscp --dscp 36 -J MARK --set-mark 0x4/0x7F > -A MARK-DSCP -m dscp --dscp 38 -J MARK --set-mark 0x4/0x7F > -A MARK-DSCP -m dscp --dscp 32 -J MARK --set-mark 0x4/0x7F > -A MARK-DSCP -m dscp --dscp 40 -J MARK --set-mark 0x4/0x7F > -A MARK-DSCP -m dscp --dscp 44 -J MARK --set-mark 0x4/0x7F > -A MARK-DSCP -m dscp --dscp 46 -J MARK --set-mark 0x4/0x7F > -A MARK-DSCP -m dscp --dscp 48 -J MARK --set-mark 0x4/0x7F > -A MARK-DSCP -m dscp --dscp 56 -J MARK --set-mark 0x4/0x7F > # default best effort > -A MARK-DSCP -m mark --mark 0x0/0x7F -J --set-mark 0x2/0x7F > > > # Policy-based fwmark setting > -A MARK-POLICY -p tcp -s 192.168.219.5 -m comment --comment "Skybox Bulk" -j > MARK --set-mark 0x1/0x7F > # add more policy rules here; anything not marked will use the DSCP bits > # of each packet; optional catch-all to avoid that and make everything BE: > #-A MARK-DSCP -m mark --mark 0x0/0x7F -J --set-mark 0x2/0x7F > _______________________________________________ Cake mailing list [email protected] https://lists.bufferbloat.net/listinfo/cake
