Hi All,
some more testing:
On Oct 12, 2014, at 01:12 , Sebastian Moeller <[email protected]> wrote:
> Hi,
>
> just to document my current understanding of using SQM on a router that also
> terminates a pppoe wan connection. We basically have two options either set
> up SQM on the real interface (let’s call it ge00 like cerowrt does) or on the
> associated pop device, pppoe-ge00. In theory both should produce the same
> results; in praxis current SQM has significant different results. Let me
> enumerate the main differences that show up when testing with
> netperf-wrapper’s RRUL test:
>
> 1) SQM on ge00 does not show a working egress classification in the RRUL test
> (no visible “banding”/stratification of the 4 different priority TCP flows),
> while SQM on pppoe-ge00 does show this stratification.
>
> Now the reason for this is quite obvious once we take into account that
> on ge00 the kernel sees a packet that already contains a PPP header between
> ethernet and IP header and has a different ether_type field, and our diffserv
> filters currently ignore everything except straight ipv4 and ipv6 packets, so
> due to the unexpected/un-handled PPP header everything lands in the default
> priority class and hence no stratification. If we shape on pppoe-ge00 the
> kernel seems to do all processing before encapsulating the data with PP so
> all filters just work. In theory that should be relatively easy to fix (at
> least for the specific PPPoE case, I am unsure about a generic solution) by
> using offsets to try to access the TOS bits in PPP-packets. Also most likely
> we face the same issue in other encapsulations that pass through cerowrt to
> some degree (except most of those will use an outer IP header from where we
> can scratch DSCPs…, but I digress)
Usind tc filters u32 filter makes it possible to actually dive into
PPPoE encapsulated ipv4 and ipv6 packets and perform classification on
“pass-through” PPPoE packets (as encountered when starting SQM on ge00 instead
of pppoe-ge00, if the latter actually handles the wan connection), so that one
is solved (but see below).
>
> 2) SQM on ge00 shows better latency under load (LUL), the LUL increases for
> ~2*fq_codels target so 10ms, while SQM on pppeo-ge00 shows a LUL-increase
> (LULI) roughly twice as large or around 20ms.
>
> I have no idea why that is, if anybody has an idea please chime in.
Once SQM on ge00 actually dives into the PPPoE packets and
applies/tests u32 filters the LUL increases to be almost identical to
pppoe-ge00’s if both ingress and egress classification are active and do work.
So it looks like the u32 filters I naively set up are quite costly. Maybe there
is a better way to set these up...
>
> 3) SQM on pppoe-ge00 has a rough 20% higher egress rate than SQM on ge00
> (with ingress more or less identical between the two). Also 2) and 3) do not
> seem to be coupled, artificially reducing the egress rate on pppoe-ge00 to
> yield the same egress rate as seen on ge00 does not reduce the LULI to the
> ge00 typical 10ms, but it stays at 20ms.
>
> For this I also have no good hypothesis, any ideas?
With classification fixed the difference in egress rate shrinks to ~10%
instead of 20, so this partly seems related to the classification issue as well.
>
>
> So the current choice is either to accept a noticeable increase in LULI (but
> note some years ago even an average of 20ms most likely was rare in the real
> life) or a equally noticeable decrease in egress bandwidth…
I guess it is back to the drawing board to figure out how to speed up
the classification… and then revisit the PPPoE question again…
Regards
Sebastian
>
> Best Regards
> Sebastian
>
> P.S.: It turns out, at least on my link, that for shaping on pppoe-ge00 the
> kernel does not account for any header automatically, so I need to specify a
> per-packet-overhead (PPOH) of 40 bytes (an an ADSL2+ link with ATM
> linklayer); when shaping on ge00 however (with the kernel still terminating
> the PPPoE link to my ISP) I only need to specify an PPOH of 26 as the kernel
> already adds the 14 bytes for the ethernet header…
>
_______________________________________________
Cerowrt-devel mailing list
[email protected]
https://lists.bufferbloat.net/listinfo/cerowrt-devel
