Re: [Cake] [PATCH iproute2-next v3] Add support for cake qdisc

2018-04-24 Thread Toke Høiland-Jørgensen 
Stephen Hemminger  writes:

> On Tue, 24 Apr 2018 16:52:57 +0200
> Toke Høiland-Jørgensen  wrote:
>
>> Well, this is leftover from keeping track of different versions of the
>> out-of-tree patch, and we already broke compatibility pretty thoroughly
>> as a preparation for upstreaming. So I'm fine with dropping the version
>> check; will resend.
>> 
>> That being said, the versioning comes from the XSTATS API, which does
>> not use netlink attributes for its members, but rather passes through as
>> struct. So what is one supposed to do in this case?
>
> If a structure is likely to change, then it should be decomposed into nested
> netlink attributes. Once you send a structure over userspace API in netlink
> it is fixed forever.

Right. Is decomposing stats into netlink attributes actually possible
within the qdisc dump_stats callback? Could we do something like:

nla_start_nest(d->skb, TCA_CAKE_STATS);

from within cake_dump_stats() and read it in cake_print_xstats in tc?

-Toke
___
Cake mailing list
Cake@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/cake

Re: [Cake] [PATCH iproute2-next v3] Add support for cake qdisc

2018-04-24 Thread Stephen Hemminger 
On Tue, 24 Apr 2018 16:52:57 +0200
Toke Høiland-Jørgensen  wrote:

> Well, this is leftover from keeping track of different versions of the
> out-of-tree patch, and we already broke compatibility pretty thoroughly
> as a preparation for upstreaming. So I'm fine with dropping the version
> check; will resend.
> 
> That being said, the versioning comes from the XSTATS API, which does
> not use netlink attributes for its members, but rather passes through as
> struct. So what is one supposed to do in this case?

If a structure is likely to change, then it should be decomposed into nested
netlink attributes. Once you send a structure over userspace API in netlink
it is fixed forever.
___
Cake mailing list
Cake@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/cake

Re: [Cake] [PATCH iproute2-next v3] Add support for cake qdisc

2018-04-24 Thread Toke Høiland-Jørgensen 
Stephen Hemminger  writes:

> On Tue, 24 Apr 2018 14:30:46 +0200
> Toke Høiland-Jørgensen  wrote:
>
>> +static void cake_print_json_tin(struct tc_cake_tin_stats *tst, uint version)
>> +{
>> +open_json_object(NULL);
>> +print_uint(PRINT_JSON, "threshold_rate", NULL, tst->threshold_rate);
>> +print_uint(PRINT_JSON, "target", NULL, tst->target_us);
>> +print_uint(PRINT_JSON, "interval", NULL, tst->interval_us);
>> +print_uint(PRINT_JSON, "peak_delay", NULL, tst->peak_delay_us);
>> +print_uint(PRINT_JSON, "average_delay", NULL, tst->avge_delay_us);
>> +print_uint(PRINT_JSON, "base_delay", NULL, tst->base_delay_us);
>> +print_uint(PRINT_JSON, "sent_packets", NULL, tst->sent.packets);
>> +print_uint(PRINT_JSON, "sent_bytes", NULL, tst->sent.bytes);
>> +print_uint(PRINT_JSON, "way_indirect_hits", NULL, 
>> tst->way_indirect_hits);
>> +print_uint(PRINT_JSON, "way_misses", NULL, tst->way_misses);
>> +print_uint(PRINT_JSON, "way_collisions", NULL, tst->way_collisions);
>> +print_uint(PRINT_JSON, "drops", NULL, tst->dropped.packets);
>> +print_uint(PRINT_JSON, "ecn_mark", NULL, tst->ecn_marked.packets);
>> +print_uint(PRINT_JSON, "ack_drops", NULL, tst->ack_drops.packets);
>> +print_uint(PRINT_JSON, "sparse_flows", NULL, tst->sparse_flows);
>> +print_uint(PRINT_JSON, "bulk_flows", NULL, tst->bulk_flows);
>> +print_uint(PRINT_JSON, "unresponsive_flows", NULL, 
>> tst->unresponse_flows);
>> +print_uint(PRINT_JSON, "max_pkt_len", NULL, tst->max_skblen);
>> +if (version >= 0x102)
>> +print_uint(PRINT_JSON, "flow_quantum", NULL, tst->flow_quantum);
>
> Please don't version objects in netlink. That is not how netlink is
> supposed to be used.

Well, this is leftover from keeping track of different versions of the
out-of-tree patch, and we already broke compatibility pretty thoroughly
as a preparation for upstreaming. So I'm fine with dropping the version
check; will resend.

That being said, the versioning comes from the XSTATS API, which does
not use netlink attributes for its members, but rather passes through as
struct. So what is one supposed to do in this case?

-Toke
___
Cake mailing list
Cake@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/cake

Re: [Cake] [PATCH iproute2-next v3] Add support for cake qdisc

2018-04-24 Thread Stephen Hemminger 
On Tue, 24 Apr 2018 14:30:46 +0200
Toke Høiland-Jørgensen  wrote:

> diff --git a/tc/q_cake.c b/tc/q_cake.c
> new file mode 100644
> index ..12263361
> --- /dev/null
> +++ b/tc/q_cake.c
> @@ -0,0 +1,778 @@
> +/* SPDX-License-Identifier: (GPL-2.0 OR BSD-3-Clause) */
> +/*
> + * Common Applications Kept Enhanced  --  CAKE
> + *
> + *  Copyright (C) 2014-2018 Jonathan Morton 
> + *  Copyright (C) 2017-2018 Toke Høiland-Jørgensen 
> + *
> + * Redistribution and use in source and binary forms, with or without
> + * modification, are permitted provided that the following conditions
> + * are met:
> + * 1. Redistributions of source code must retain the above copyright
> + *notice, this list of conditions, and the following disclaimer,
> + *without modification.
> + * 2. Redistributions in binary form must reproduce the above copyright
> + *notice, this list of conditions and the following disclaimer in the
> + *documentation and/or other materials provided with the distribution.
> + * 3. The names of the authors may not be used to endorse or promote products
> + *derived from this software without specific prior written permission.
> + *
> + * Alternatively, provided that this notice is retained in full, this
> + * software may be distributed under the terms of the GNU General
> + * Public License ("GPL") version 2, in which case the provisions of the
> + * GPL apply INSTEAD OF those given above.
> + *
> + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
> + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
> + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
> + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
> + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
> + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
> + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
> + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
> + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
> + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
> + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH
> + * DAMAGE.
> + *
> + */

If you  have SPDX tag you don't have to have all the GPL boilerplate anymore.
___
Cake mailing list
Cake@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/cake

[Cake] [PATCH iproute2-next v3] Add support for cake qdisc

2018-04-24 Thread Toke Høiland-Jørgensen 
sch_cake is intended to squeeze the most bandwidth and latency out of even
the slowest ISP links and routers, while presenting an API simple enough
that even an ISP can configure it.

Example of use on a cable ISP uplink:

tc qdisc add dev eth0 cake bandwidth 20Mbit nat docsis ack-filter

To shape a cable download link (ifb and tc-mirred setup elided)

tc qdisc add dev ifb0 cake bandwidth 200mbit nat docsis ingress wash besteffort

Cake is filled with:

* A hybrid Codel/Blue AQM algorithm, "Cobalt", tied to an FQ_Codel
  derived Flow Queuing system, which autoconfigures based on the bandwidth.
* A novel "triple-isolate" mode (the default) which balances per-host
  and per-flow FQ even through NAT.
* An deficit based shaper, that can also be used in an unlimited mode.
* 8 way set associative hashing to reduce flow collisions to a minimum.
* A reasonable interpretation of various diffserv latency/loss tradeoffs.
* Support for zeroing diffserv markings for entering and exiting traffic.
* Support for interacting well with Docsis 3.0 shaper framing.
* Support for DSL framing types and shapers.
* Support for ack filtering.
* Extensive statistics for measuring, loss, ecn markings, latency variation.

Various versions baking have been available as an out of tree build for
kernel versions going back to 3.10, as the embedded router world has been
running a few years behind mainline Linux. A stable version has been
generally available on lede-17.01 and later.

sch_cake replaces a combination of iptables, tc filter, htb and fq_codel
in the sqm-scripts, with sane defaults and vastly simpler configuration.

Cake's principal author is Jonathan Morton, with contributions from
Kevin Darbyshire-Bryant, Toke Høiland-Jørgensen, Sebastian Moeller,
Ryan Mounce, Guido Sarducci, Dean Scarff, Nils Andreas Svee, Dave Täht,
and Loganaden Velvindron.

Testing from Pete Heist, Georgios Amanakis, and the many other members of
the cake@lists.bufferbloat.net mailing list.

Signed-off-by: Dave Taht 
Signed-off-by: Toke Høiland-Jørgensen 
---
Changes since v2:
  - Accidentally sent a version of the tc patch from my testing branch
which includes a test flag that shouldn't be in there. Sorry for the
mistake.
  
 man/man8/tc-cake.8 | 632 +++
 man/man8/tc.8  |   1 +
 tc/Makefile|   1 +
 tc/q_cake.c| 778 +
 4 files changed, 1412 insertions(+)
 create mode 100644 man/man8/tc-cake.8
 create mode 100644 tc/q_cake.c

diff --git a/man/man8/tc-cake.8 b/man/man8/tc-cake.8
new file mode 100644
index ..30e41bc9
--- /dev/null
+++ b/man/man8/tc-cake.8
@@ -0,0 +1,632 @@
+.TH CAKE 8 "23 November 2017" "iproute2" "Linux"
+.SH NAME
+CAKE \- Common Applications Kept Enhanced (CAKE)
+.SH SYNOPSIS
+.B tc qdisc ... cake
+.br
+[
+.BR bandwidth
+RATE |
+.BR unlimited*
+|
+.BR autorate_ingress
+]
+.br
+[
+.BR rtt
+TIME |
+.BR datacentre
+|
+.BR lan
+|
+.BR metro
+|
+.BR regional
+|
+.BR internet*
+|
+.BR oceanic
+|
+.BR satellite
+|
+.BR interplanetary
+]
+.br
+[
+.BR besteffort
+|
+.BR diffserv8
+|
+.BR diffserv4
+|
+.BR diffserv3*
+]
+.br
+[
+.BR flowblind
+|
+.BR srchost
+|
+.BR dsthost
+|
+.BR hosts
+|
+.BR flows
+|
+.BR dual-srchost
+|
+.BR dual-dsthost
+|
+.BR triple-isolate*
+]
+.br
+[
+.BR nat
+|
+.BR nonat*
+]
+.br
+[
+.BR wash
+|
+.BR nowash*
+]
+.br
+[
+.BR ack-filter
+|
+.BR ack-filter-aggressive
+|
+.BR no-ack-filter*
+]
+.br
+[
+.BR memlimit
+LIMIT ]
+.br
+[
+.BR ptm
+|
+.BR atm
+|
+.BR noatm*
+]
+.br
+[
+.BR overhead
+N |
+.BR conservative
+|
+.BR raw*
+]
+.br
+[
+.BR mpu
+N ]
+.br
+[
+.BR ingress
+|
+.BR egress*
+]
+.br
+(* marks defaults)
+
+
+.SH DESCRIPTION
+CAKE (Common Applications Kept Enhanced) is a shaping-capable queue discipline
+which uses both AQM and FQ.  It combines COBALT, which is an AQM algorithm
+combining Codel and BLUE, a shaper which operates in deficit mode, and a 
variant
+of DRR++ for flow isolation.  8-way set-associative hashing is used to 
virtually
+eliminate hash collisions.  Priority queuing is available through a simplified
+diffserv implementation.  Overhead compensation for various encapsulation
+schemes is tightly integrated.
+
+All settings are optional; the default settings are chosen to be sensible in
+most common deployments.  Most people will only need to set the
+.B bandwidth
+parameter to get useful results, but reading the
+.B Overhead Compensation
+and
+.B Round Trip Time
+sections is strongly encouraged.
+
+.SH SHAPER PARAMETERS
+CAKE uses a deficit-mode shaper, which does not exhibit the initial burst
+typical of token-bucket shapers.  It will automatically burst precisely as much
+as required to maintain the configured throughput.  As such, it is very
+straightforward to configure.
+.PP
+.B unlimited
+(default)
+.br
+   No limit on the bandwidth.
+.PP
+.B bandwidth
+RATE
+.br
+   Set the shaper bandwidth.  See
+.BR tc(8)
+or examples