Re: [PATCH] pkt_sched: enable QFQ to support TSO/GSO
From: Paolo Valente Date: Tue, 6 Nov 2012 07:29:24 +0100 > If the max packet size for some class (configured through tc) is > violated by the actual size of the packets of that class, then QFQ > would not schedule classes correctly, and the data structures > implementing the bucket lists may get corrupted. This problem occurs > with TSO/GSO even if the max packet size is set to the MTU, and is, > e.g., the cause of the failure reported in [1]. Two patches have been > proposed to solve this problem in [2], one of them is a preliminary > version of this patch. > > This patch addresses the above issues by: 1) setting QFQ parameters to > proper values for supporting TSO/GSO (in particular, setting the > maximum possible packet size to 64KB), 2) automatically increasing the > max packet size for a class, lmax, when a packet with a larger size > than the current value of lmax arrives. > > The drawback of the first point is that the maximum weight for a class > is now limited to 4096, which is equal to 1/16 of the maximum weight > sum. > > Finally, this patch also forcibly caps the timestamps of a class if > they are too high to be stored in the bucket list. This capping, taken > from QFQ+ [3], handles the unfrequent case described in the comment to > the function slot_insert. > > [1] http://marc.info/?l=linux-netdev=134968777902077=2 > [2] http://marc.info/?l=linux-netdev=135096573507936=2 > [3] http://marc.info/?l=linux-netdev=134902691421670=2 > > Signed-off-by: Paolo Valente > Tested-by: Cong Wang > Acked-by: Stephen Hemminger Applied, thanks. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] pkt_sched: enable QFQ to support TSO/GSO
From: Paolo Valente paolo.vale...@unimore.it Date: Tue, 6 Nov 2012 07:29:24 +0100 If the max packet size for some class (configured through tc) is violated by the actual size of the packets of that class, then QFQ would not schedule classes correctly, and the data structures implementing the bucket lists may get corrupted. This problem occurs with TSO/GSO even if the max packet size is set to the MTU, and is, e.g., the cause of the failure reported in [1]. Two patches have been proposed to solve this problem in [2], one of them is a preliminary version of this patch. This patch addresses the above issues by: 1) setting QFQ parameters to proper values for supporting TSO/GSO (in particular, setting the maximum possible packet size to 64KB), 2) automatically increasing the max packet size for a class, lmax, when a packet with a larger size than the current value of lmax arrives. The drawback of the first point is that the maximum weight for a class is now limited to 4096, which is equal to 1/16 of the maximum weight sum. Finally, this patch also forcibly caps the timestamps of a class if they are too high to be stored in the bucket list. This capping, taken from QFQ+ [3], handles the unfrequent case described in the comment to the function slot_insert. [1] http://marc.info/?l=linux-netdevm=134968777902077w=2 [2] http://marc.info/?l=linux-netdevm=135096573507936w=2 [3] http://marc.info/?l=linux-netdevm=134902691421670w=2 Signed-off-by: Paolo Valente paolo.vale...@unimore.it Tested-by: Cong Wang amw...@redhat.com Acked-by: Stephen Hemminger shemmin...@vyatta.com Applied, thanks. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] pkt_sched: enable QFQ to support TSO/GSO
On Tue, 6 Nov 2012 07:29:24 +0100 Paolo Valente wrote: > If the max packet size for some class (configured through tc) is > violated by the actual size of the packets of that class, then QFQ > would not schedule classes correctly, and the data structures > implementing the bucket lists may get corrupted. This problem occurs > with TSO/GSO even if the max packet size is set to the MTU, and is, > e.g., the cause of the failure reported in [1]. Two patches have been > proposed to solve this problem in [2], one of them is a preliminary > version of this patch. > > This patch addresses the above issues by: 1) setting QFQ parameters to > proper values for supporting TSO/GSO (in particular, setting the > maximum possible packet size to 64KB), 2) automatically increasing the > max packet size for a class, lmax, when a packet with a larger size > than the current value of lmax arrives. > > The drawback of the first point is that the maximum weight for a class > is now limited to 4096, which is equal to 1/16 of the maximum weight > sum. > > Finally, this patch also forcibly caps the timestamps of a class if > they are too high to be stored in the bucket list. This capping, taken > from QFQ+ [3], handles the unfrequent case described in the comment to > the function slot_insert. > > [1] http://marc.info/?l=linux-netdev=134968777902077=2 > [2] http://marc.info/?l=linux-netdev=135096573507936=2 > [3] http://marc.info/?l=linux-netdev=134902691421670=2 > > Signed-off-by: Paolo Valente > Tested-by: Cong Wang > Acked-by: Stephen Hemminger Acked-by: Stephen Hemminger -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] pkt_sched: enable QFQ to support TSO/GSO
On Tue, 6 Nov 2012 07:29:24 +0100 Paolo Valente paolo.vale...@unimore.it wrote: If the max packet size for some class (configured through tc) is violated by the actual size of the packets of that class, then QFQ would not schedule classes correctly, and the data structures implementing the bucket lists may get corrupted. This problem occurs with TSO/GSO even if the max packet size is set to the MTU, and is, e.g., the cause of the failure reported in [1]. Two patches have been proposed to solve this problem in [2], one of them is a preliminary version of this patch. This patch addresses the above issues by: 1) setting QFQ parameters to proper values for supporting TSO/GSO (in particular, setting the maximum possible packet size to 64KB), 2) automatically increasing the max packet size for a class, lmax, when a packet with a larger size than the current value of lmax arrives. The drawback of the first point is that the maximum weight for a class is now limited to 4096, which is equal to 1/16 of the maximum weight sum. Finally, this patch also forcibly caps the timestamps of a class if they are too high to be stored in the bucket list. This capping, taken from QFQ+ [3], handles the unfrequent case described in the comment to the function slot_insert. [1] http://marc.info/?l=linux-netdevm=134968777902077w=2 [2] http://marc.info/?l=linux-netdevm=135096573507936w=2 [3] http://marc.info/?l=linux-netdevm=134902691421670w=2 Signed-off-by: Paolo Valente paolo.vale...@unimore.it Tested-by: Cong Wang amw...@redhat.com Acked-by: Stephen Hemminger shemmin...@vyatta.com Acked-by: Stephen Hemminger shemmin...@vyatta.com -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[PATCH] pkt_sched: enable QFQ to support TSO/GSO
If the max packet size for some class (configured through tc) is violated by the actual size of the packets of that class, then QFQ would not schedule classes correctly, and the data structures implementing the bucket lists may get corrupted. This problem occurs with TSO/GSO even if the max packet size is set to the MTU, and is, e.g., the cause of the failure reported in [1]. Two patches have been proposed to solve this problem in [2], one of them is a preliminary version of this patch. This patch addresses the above issues by: 1) setting QFQ parameters to proper values for supporting TSO/GSO (in particular, setting the maximum possible packet size to 64KB), 2) automatically increasing the max packet size for a class, lmax, when a packet with a larger size than the current value of lmax arrives. The drawback of the first point is that the maximum weight for a class is now limited to 4096, which is equal to 1/16 of the maximum weight sum. Finally, this patch also forcibly caps the timestamps of a class if they are too high to be stored in the bucket list. This capping, taken from QFQ+ [3], handles the unfrequent case described in the comment to the function slot_insert. [1] http://marc.info/?l=linux-netdev=134968777902077=2 [2] http://marc.info/?l=linux-netdev=135096573507936=2 [3] http://marc.info/?l=linux-netdev=134902691421670=2 Signed-off-by: Paolo Valente Tested-by: Cong Wang Acked-by: Stephen Hemminger --- net/sched/sch_qfq.c | 109 +-- 1 file changed, 79 insertions(+), 30 deletions(-) diff --git a/net/sched/sch_qfq.c b/net/sched/sch_qfq.c index f0dd83c..9687fa1 100644 --- a/net/sched/sch_qfq.c +++ b/net/sched/sch_qfq.c @@ -84,18 +84,19 @@ * grp->index is the index of the group; and grp->slot_shift * is the shift for the corresponding (scaled) sigma_i. */ -#define QFQ_MAX_INDEX 19 -#define QFQ_MAX_WSHIFT 16 +#define QFQ_MAX_INDEX 24 +#define QFQ_MAX_WSHIFT 12 #defineQFQ_MAX_WEIGHT (1groups[i] != cl->grp && cl->qdisc->q.qlen > 0) { + /* +* shift cl->F back, to not charge the +* class for the not-yet-served head +* packet +*/ + cl->F = cl->S; + /* remove class from its slot in the old group */ + qfq_deactivate_class(q, cl); + need_reactivation = true; + } + + qfq_update_class_params(q, cl, lmax, inv_w, delta_w); + + if (need_reactivation) /* activate in new group */ + qfq_activate_class(q, cl, qdisc_peek_len(cl->qdisc)); +} + + static int qfq_change_class(struct Qdisc *sch, u32 classid, u32 parentid, struct nlattr **tca, unsigned long *arg) { @@ -238,7 +265,7 @@ static int qfq_change_class(struct Qdisc *sch, u32 classid, u32 parentid, struct qfq_class *cl = (struct qfq_class *)*arg; struct nlattr *tb[TCA_QFQ_MAX + 1]; u32 weight, lmax, inv_w; - int i, err; + int err; int delta_w; if (tca[TCA_OPTIONS] == NULL) { @@ -270,16 +297,14 @@ static int qfq_change_class(struct Qdisc *sch, u32 classid, u32 parentid, if (tb[TCA_QFQ_LMAX]) { lmax = nla_get_u32(tb[TCA_QFQ_LMAX]); - if (!lmax || lmax > (1UL << QFQ_MTU_SHIFT)) { + if (lmax < QFQ_MIN_LMAX || lmax > (1UL << QFQ_MTU_SHIFT)) { pr_notice("qfq: invalid max length %u\n", lmax); return -EINVAL; } } else - lmax = 1UL << QFQ_MTU_SHIFT; + lmax = psched_mtu(qdisc_dev(sch)); if (cl != NULL) { - bool need_reactivation = false; - if (tca[TCA_RATE]) { err = gen_replace_estimator(>bstats, >rate_est, qdisc_root_sleeping_lock(sch), @@ -291,24 +316,8 @@ static int qfq_change_class(struct Qdisc *sch, u32 classid, u32 parentid, if (lmax == cl->lmax && inv_w == cl->inv_w) return 0; /* nothing to update */ - i = qfq_calc_index(inv_w, lmax); sch_tree_lock(sch); - if (>groups[i] != cl->grp && cl->qdisc->q.qlen > 0) { - /* -* shift cl->F back, to not charge the -* class for the not-yet-served head -* packet -*/ - cl->F = cl->S; - /* remove class from its slot in the old group */ -
[PATCH] pkt_sched: enable QFQ to support TSO/GSO
If the max packet size for some class (configured through tc) is
violated by the actual size of the packets of that class, then QFQ
would not schedule classes correctly, and the data structures
implementing the bucket lists may get corrupted. This problem occurs
with TSO/GSO even if the max packet size is set to the MTU, and is,
e.g., the cause of the failure reported in [1]. Two patches have been
proposed to solve this problem in [2], one of them is a preliminary
version of this patch.
This patch addresses the above issues by: 1) setting QFQ parameters to
proper values for supporting TSO/GSO (in particular, setting the
maximum possible packet size to 64KB), 2) automatically increasing the
max packet size for a class, lmax, when a packet with a larger size
than the current value of lmax arrives.
The drawback of the first point is that the maximum weight for a class
is now limited to 4096, which is equal to 1/16 of the maximum weight
sum.
Finally, this patch also forcibly caps the timestamps of a class if
they are too high to be stored in the bucket list. This capping, taken
from QFQ+ [3], handles the unfrequent case described in the comment to
the function slot_insert.
[1] http://marc.info/?l=linux-netdevm=134968777902077w=2
[2] http://marc.info/?l=linux-netdevm=135096573507936w=2
[3] http://marc.info/?l=linux-netdevm=134902691421670w=2
Signed-off-by: Paolo Valente paolo.vale...@unimore.it
Tested-by: Cong Wang amw...@redhat.com
Acked-by: Stephen Hemminger shemmin...@vyatta.com
---
net/sched/sch_qfq.c | 109 +--
1 file changed, 79 insertions(+), 30 deletions(-)
diff --git a/net/sched/sch_qfq.c b/net/sched/sch_qfq.c
index f0dd83c..9687fa1 100644
--- a/net/sched/sch_qfq.c
+++ b/net/sched/sch_qfq.c
@@ -84,18 +84,19 @@
* grp-index is the index of the group; and grp-slot_shift
* is the shift for the corresponding (scaled) sigma_i.
*/
-#define QFQ_MAX_INDEX 19
-#define QFQ_MAX_WSHIFT 16
+#define QFQ_MAX_INDEX 24
+#define QFQ_MAX_WSHIFT 12
#defineQFQ_MAX_WEIGHT (1QFQ_MAX_WSHIFT)
-#define QFQ_MAX_WSUM (2*QFQ_MAX_WEIGHT)
+#define QFQ_MAX_WSUM (16*QFQ_MAX_WEIGHT)
#define FRAC_BITS 30 /* fixed point arithmetic */
#define ONE_FP (1UL FRAC_BITS)
#define IWSUM (ONE_FP/QFQ_MAX_WSUM)
-#define QFQ_MTU_SHIFT 11
+#define QFQ_MTU_SHIFT 16 /* to support TSO/GSO */
#define QFQ_MIN_SLOT_SHIFT (FRAC_BITS + QFQ_MTU_SHIFT - QFQ_MAX_INDEX)
+#define QFQ_MIN_LMAX 256 /* min possible lmax for a class */
/*
* Possible group states. These values are used as indexes for the bitmaps
@@ -231,6 +232,32 @@ static void qfq_update_class_params(struct qfq_sched *q,
struct qfq_class *cl,
q-wsum += delta_w;
}
+static void qfq_update_reactivate_class(struct qfq_sched *q,
+ struct qfq_class *cl,
+ u32 inv_w, u32 lmax, int delta_w)
+{
+ bool need_reactivation = false;
+ int i = qfq_calc_index(inv_w, lmax);
+
+ if (q-groups[i] != cl-grp cl-qdisc-q.qlen 0) {
+ /*
+* shift cl-F back, to not charge the
+* class for the not-yet-served head
+* packet
+*/
+ cl-F = cl-S;
+ /* remove class from its slot in the old group */
+ qfq_deactivate_class(q, cl);
+ need_reactivation = true;
+ }
+
+ qfq_update_class_params(q, cl, lmax, inv_w, delta_w);
+
+ if (need_reactivation) /* activate in new group */
+ qfq_activate_class(q, cl, qdisc_peek_len(cl-qdisc));
+}
+
+
static int qfq_change_class(struct Qdisc *sch, u32 classid, u32 parentid,
struct nlattr **tca, unsigned long *arg)
{
@@ -238,7 +265,7 @@ static int qfq_change_class(struct Qdisc *sch, u32 classid,
u32 parentid,
struct qfq_class *cl = (struct qfq_class *)*arg;
struct nlattr *tb[TCA_QFQ_MAX + 1];
u32 weight, lmax, inv_w;
- int i, err;
+ int err;
int delta_w;
if (tca[TCA_OPTIONS] == NULL) {
@@ -270,16 +297,14 @@ static int qfq_change_class(struct Qdisc *sch, u32
classid, u32 parentid,
if (tb[TCA_QFQ_LMAX]) {
lmax = nla_get_u32(tb[TCA_QFQ_LMAX]);
- if (!lmax || lmax (1UL QFQ_MTU_SHIFT)) {
+ if (lmax QFQ_MIN_LMAX || lmax (1UL QFQ_MTU_SHIFT)) {
pr_notice(qfq: invalid max length %u\n, lmax);
return -EINVAL;
}
} else
- lmax = 1UL QFQ_MTU_SHIFT;
+ lmax = psched_mtu(qdisc_dev(sch));
if (cl != NULL) {
- bool need_reactivation = false;
-
if (tca[TCA_RATE]) {
err = gen_replace_estimator(cl-bstats,
