In my one-armed router setup I’m seeing host fairness work perfectly with
srchost or dsthost, but with dual-srchost or dual-dsthost, host fairness
deviates from the ideal, _only_ when there's bi-directional traffic. The
deviation is then dependent on the number of flows. Is this expected?
I had thought that dual-src/dsthost worked the same as src/dsthost (fairness
between hosts) with the exception that there is also fairness of flows within
each host.
Here are some results (all rates aggregate throughput in Mbit):
IP1=8 up / 1 down IP2=1 up / 8 down (post-test tc stats attached):
srchost/dsthost, upload only: IP1=48.1, IP2=47.9 (OK)
srchost/dsthost, download only: IP1=47.8, IP2=47.8 (OK)
srchost/dsthost, bi-directional: IP1=47.5 up / 43.9 down, IP2=44.7 up /
46.7 down (OK)
dual-srchost/dual-dsthost, upload only: IP1=48.1, IP2=48.0 (OK)
dual-srchost/dual-dsthost, download only: IP1=47.9, IP2=47.9 (OK)
dual-srchost/dual-dsthost, bi-directional: IP1=83.0 up / 10.7 down,
IP2=10.6 up / 83.0 down (*** asymmetric ***)
Dual-srchost/dual-dsthost, bi-directional tests with different flow counts:
IP1=4 up / 1 down IP2=1 up / 4 down:
IP1=74.8 up / 18.8 down, IP2=18.8 up / 74.8 down
IP1=2 up / 1 down IP2=1 up / 2 down:
IP1=62.4 up / 31.3 down, IP2=31.3 up / 62.4 down
IP1=4 up / 1 down IP2=1 up / 8 down:
IP1=81.8 up / 11.5 down, IP2=17.4 up / 76.3 down
IP1=2 up / 1 down IP2=1 up / 8 down:
IP1=79.9 up / 13.5 down, IP2=25.7 up / 68.1 down
The setup:
apu2a (kernel 4.9) <— default VLAN —> apu1a (kernel 3.16.7) <— VLAN
3300 —> apu2b (kernel 4.9)
- apu1a is the router, and has cake only on egress of both eth0 and eth0.3300,
rate limited to 100mbit for both
- it has no trouble shaping at 100mbit up and down simultaneously, so that
should not be a problem
- the same problem occurs at 25mbit or 50mbit)
- since apu2a is the client [dual-]dsthost is used on eth0 and [dual-]srchost
is used on eth0.3300
- the fairness test setup seems correct, based on the results of most of the
tests, at least.
- note in the qdisc stats attached there is a prio qdisc on eth0 for filtering
out VLAN traffic so it isn’t shaped twice
- I also get the exact same results with an htb or hfsc hierarchy on eth0
instead of adding a qdisc to eth0.3300
- printk’s in sch_cake.c shows values of flow_mode, srchost_hash and
dsthost_hash as expected
- I also see it going into allocate_src and allocate_dst as expected, and later
ending up in found_src and found_dst
I’m stumped. I know I’ve tested fairness of dual-src/dsthost before, but that
was from the egress of client and server, and it was on a recent kernel. Time
to sleep on it...
qdisc prio 1: dev eth0 root refcnt 2 bands 2 priomap 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1
Sent 1502062182 bytes 1525226 pkt (dropped 204373, overlimits 0 requeues 0)
backlog 0b 4294550547p requeues 0
qdisc cake 10: dev eth0 parent 1:1 bandwidth 100Mbit besteffort dual-dsthost
nonat nowash no-ack-filter split-gso rtt 100.0ms raw overhead 0
Sent 751856301 bytes 762993 pkt (dropped 8512, overlimits 2414269 requeues 0)
backlog 0b 0p requeues 0
memory used: 361600b of 5000000b
capacity estimate: 100Mbit
min/max network layer size: 42 / 1514
min/max overhead-adjusted size: 42 / 1514
average network hdr offset: 14
Tin 0
thresh 100Mbit
target 5.0ms
interval 100.0ms
pk_delay 3.9ms
av_delay 2.4ms
sp_delay 60us
backlog 0b
pkts 771505
bytes 764743469
way_inds 6
way_miss 26
way_cols 0
drops 8512
marks 0
ack_drop 0
sp_flows 9
bk_flows 4
un_flows 0
max_len 18168
quantum 1514
qdisc cake 8060: dev eth0.3300 root refcnt 2 bandwidth 100Mbit besteffort
dual-srchost nonat nowash no-ack-filter split-gso rtt 100.0ms raw overhead 0
Sent 750205881 bytes 762233 pkt (dropped 8542, overlimits 904221 requeues 0)
backlog 0b 0p requeues 0
memory used: 279744b of 5000000b
capacity estimate: 100Mbit
min/max network layer size: 42 / 1514
min/max overhead-adjusted size: 42 / 1514
average network hdr offset: 14
Tin 0
thresh 100Mbit
target 5.0ms
interval 100.0ms
pk_delay 157us
av_delay 83us
sp_delay 1us
backlog 0b
pkts 770775
bytes 763138469
way_inds 31166
way_miss 23
way_cols 0
drops 8542
marks 0
ack_drop 0
sp_flows 17
bk_flows 1
un_flows 0
max_len 30280
quantum 1514
_______________________________________________
Cake mailing list
[email protected]
https://lists.bufferbloat.net/listinfo/cake
