Re: qdisc spin lock

2016-04-01 Thread Michael Ma
2016-03-31 19:19 GMT-07:00 David Miller <da...@davemloft.net>: > From: Michael Ma <make0...@gmail.com> > Date: Thu, 31 Mar 2016 16:48:43 -0700 > >> I didn't really know that multiple qdiscs can be isolated using MQ so > ... > > Please stop top-posting. Sorry that I wasn't aware of this...

Re: qdisc spin lock

2016-03-31 Thread Michael Ma
Thanks for the suggestion - I'll try the MQ solution out. It seems to be able to solve the problem well with the assumption that bandwidth can be statically partitioned. 2016-03-31 12:18 GMT-07:00 Jesper Dangaard Brouer <bro...@redhat.com>: > > On Wed, 30 Mar 2016 00:20:03 -0700 Micha

Re: qdisc spin lock

2016-03-31 Thread Michael Ma
ngc...@gmail.com>: > On Wed, Mar 30, 2016 at 12:20 AM, Michael Ma <make0...@gmail.com> wrote: >> As far as I understand the design of TC is to simplify locking schema >> and minimize the work in __qdisc_run so that throughput won’t be >> affected, especially with large

qdisc spin lock

2016-03-30 Thread Michael Ma
Hi - I know this might be an old topic so bare with me – what we are facing is that applications are sending small packets using hundreds of threads so the contention on spin lock in __dev_xmit_skb increases the latency of dev_queue_xmit significantly. We’re building a network QoS solution to

Re: qdisc spin lock

2016-04-25 Thread Michael Ma
2016-04-21 15:12 GMT-07:00 Michael Ma <make0...@gmail.com>: > 2016-04-21 5:41 GMT-07:00 Eric Dumazet <eric.duma...@gmail.com>: >> On Wed, 2016-04-20 at 22:51 -0700, Michael Ma wrote: >>> 2016-04-20 15:34 GMT-07:00 Eric Dumazet <eric.duma...@gmail.com>: >

Re: qdisc spin lock

2016-04-20 Thread Michael Ma
2016-04-08 7:19 GMT-07:00 Eric Dumazet <eric.duma...@gmail.com>: > On Thu, 2016-03-31 at 16:48 -0700, Michael Ma wrote: >> I didn't really know that multiple qdiscs can be isolated using MQ so >> that each txq can be associated with a particular qdisc. Also we don't &g

Re: qdisc spin lock

2016-04-21 Thread Michael Ma
2016-04-21 5:41 GMT-07:00 Eric Dumazet <eric.duma...@gmail.com>: > On Wed, 2016-04-20 at 22:51 -0700, Michael Ma wrote: >> 2016-04-20 15:34 GMT-07:00 Eric Dumazet <eric.duma...@gmail.com>: >> > On Wed, 2016-04-20 at 14:24 -0700, Michael Ma wrote: >> >&

Re: qdisc spin lock

2016-04-15 Thread Michael Ma
2016-04-15 15:54 GMT-07:00 Eric Dumazet <eric.duma...@gmail.com>: > On Fri, 2016-04-15 at 15:46 -0700, Michael Ma wrote: >> 2016-04-08 7:19 GMT-07:00 Eric Dumazet <eric.duma...@gmail.com>: >> > On Thu, 2016-03-31 at 16:48 -0700, Michael Ma wrote: >> >> I d

Re: qdisc spin lock

2016-04-15 Thread Michael Ma
2016-04-08 7:19 GMT-07:00 Eric Dumazet <eric.duma...@gmail.com>: > On Thu, 2016-03-31 at 16:48 -0700, Michael Ma wrote: >> I didn't really know that multiple qdiscs can be isolated using MQ so >> that each txq can be associated with a particular qdisc. Also we don't &g

Re: qdisc spin lock

2016-04-20 Thread Michael Ma
2016-04-20 15:34 GMT-07:00 Eric Dumazet <eric.duma...@gmail.com>: > On Wed, 2016-04-20 at 14:24 -0700, Michael Ma wrote: >> 2016-04-08 7:19 GMT-07:00 Eric Dumazet <eric.duma...@gmail.com>: >> > On Thu, 2016-03-31 at 16:48 -0700, Michael Ma wrote: >> >> I d

Re: qdisc spin lock

2016-04-13 Thread Michael Ma
2016-03-31 20:44 GMT-07:00 John Fastabend <john.fastab...@gmail.com>: > On 16-03-31 04:48 PM, Michael Ma wrote: >> I didn't really know that multiple qdiscs can be isolated using MQ so >> that each txq can be associated with a particular qdisc. Also we don't >> real

Re: Modification to skb->queue_mapping affecting performance

2016-09-14 Thread Michael Ma
2016-09-13 22:22 GMT-07:00 Eric Dumazet <eric.duma...@gmail.com>: > On Tue, 2016-09-13 at 22:13 -0700, Michael Ma wrote: > >> I don't intend to install multiple qdisc - the only reason that I'm >> doing this now is to leverage MQ to workaround the lock contention, >

Modification to skb->queue_mapping affecting performance

2016-09-13 Thread Michael Ma
Hi - We currently use mqprio on ifb to work around the qdisc root lock contention on the receiver side. The problem that we found was that queue_mapping is already set when redirecting from ingress qdisc to ifb (based on RX selection, I guess?) so the TX queue selection is not based on priority.

Re: Modification to skb->queue_mapping affecting performance

2016-09-13 Thread Michael Ma
2016-09-13 17:09 GMT-07:00 Eric Dumazet <eric.duma...@gmail.com>: > On Tue, 2016-09-13 at 16:30 -0700, Michael Ma wrote: > >> The RX queue number I found from "ls /sys/class/net/eth0/queues" is >> 64. (is this the correct way of identifying the queue number on N

Re: Modification to skb->queue_mapping affecting performance

2016-09-13 Thread Michael Ma
2016-09-13 18:18 GMT-07:00 Eric Dumazet <eric.duma...@gmail.com>: > On Tue, 2016-09-13 at 17:23 -0700, Michael Ma wrote: > >> If I understand correctly this is still to associate a qdisc with each >> ifb TXQ. How should I do this if I want to use HTB? I guess I'll need &g

Re: Modification to skb->queue_mapping affecting performance

2016-09-13 Thread Michael Ma
2016-09-13 22:13 GMT-07:00 Michael Ma <make0...@gmail.com>: > 2016-09-13 18:18 GMT-07:00 Eric Dumazet <eric.duma...@gmail.com>: >> On Tue, 2016-09-13 at 17:23 -0700, Michael Ma wrote: >> >>> If I understand correctly this is still to associate a qdisc wi

Re: Modification to skb->queue_mapping affecting performance

2016-09-15 Thread Michael Ma
2016-09-14 10:46 GMT-07:00 Michael Ma <make0...@gmail.com>: > 2016-09-13 22:22 GMT-07:00 Eric Dumazet <eric.duma...@gmail.com>: >> On Tue, 2016-09-13 at 22:13 -0700, Michael Ma wrote: >> >>> I don't intend to install multiple qdisc - the only reason that I'm

Re: Modification to skb->queue_mapping affecting performance

2016-09-23 Thread Michael Ma
2016-09-16 15:00 GMT-07:00 Michael Ma <make0...@gmail.com>: > 2016-09-16 12:53 GMT-07:00 Eric Dumazet <eric.duma...@gmail.com>: >> On Fri, 2016-09-16 at 10:57 -0700, Michael Ma wrote: >> >>> This is actually the problem - if flows from different RX queues a

Re: Modification to skb->queue_mapping affecting performance

2016-09-16 Thread Michael Ma
2016-09-16 12:53 GMT-07:00 Eric Dumazet <eric.duma...@gmail.com>: > On Fri, 2016-09-16 at 10:57 -0700, Michael Ma wrote: > >> This is actually the problem - if flows from different RX queues are >> switched to the same RX queue in IFB, they'll use different processor

Re: Modification to skb->queue_mapping affecting performance

2016-09-16 Thread Michael Ma
2016-09-15 17:51 GMT-07:00 Michael Ma <make0...@gmail.com>: > 2016-09-14 10:46 GMT-07:00 Michael Ma <make0...@gmail.com>: >> 2016-09-13 22:22 GMT-07:00 Eric Dumazet <eric.duma...@gmail.com>: >>> On Tue, 2016-09-13 at 22:13 -0700, Michael Ma wrote: >>>

Re: Why do we need tasklet in IFB?

2016-11-01 Thread Michael Ma
2016-11-01 4:38 GMT-07:00 Jamal Hadi Salim <j...@mojatatu.com>: > On 16-10-31 02:10 PM, David Miller wrote: >> >> From: Michael Ma <make0...@gmail.com> >> Date: Mon, 31 Oct 2016 11:02:28 -0700 >> >>> 2016-10-28 14:52 GMT-07:00 Michael Ma <make0.

Re: Why do we need tasklet in IFB?

2016-10-31 Thread Michael Ma
2016-10-31 11:02 GMT-07:00 Michael Ma <make0...@gmail.com>: > 2016-10-28 14:52 GMT-07:00 Michael Ma <make0...@gmail.com>: >> 2016-10-28 14:48 GMT-07:00 Stephen Hemminger <step...@networkplumber.org>: >>> On Fri, 28 Oct 2016 14:45:07 -0700 >>> Michael Ma

Re: Why do we need tasklet in IFB?

2016-10-31 Thread Michael Ma
2016-10-28 14:52 GMT-07:00 Michael Ma <make0...@gmail.com>: > 2016-10-28 14:48 GMT-07:00 Stephen Hemminger <step...@networkplumber.org>: >> On Fri, 28 Oct 2016 14:45:07 -0700 >> Michael Ma <make0...@gmail.com> wrote: >> >>> 2016-10-28 14:38 GMT-07:

Why do we need tasklet in IFB?

2016-10-28 Thread Michael Ma
Hi - Currently IFB uses tasklet to process tx/rx on the interface that forwarded the packet to IFB. My understanding on why we're doing this is that since dev_queue_xmit() can be invoked in interrupt, we want to defer the processing of original tx/rx in case ifb_xmit() is called from interrupt.

Corrupted SKB

2017-04-17 Thread Michael Ma
Hi - We've implemented a "glue" qdisc similar to mqprio which can associate one qdisc to multiple txqs as the root qdisc. Reference count of the child qdiscs have been adjusted properly in this case so that it represents the number of txqs it has been attached to. However when sending packets we

Re: Corrupted SKB

2017-04-18 Thread Michael Ma
2017-04-18 16:12 GMT-07:00 Cong Wang <xiyou.wangc...@gmail.com>: > On Mon, Apr 17, 2017 at 5:39 PM, Michael Ma <make0...@gmail.com> wrote: >> Hi - >> >> We've implemented a "glue" qdisc similar to mqprio which can associate >> one qdisc to

Multiple Txqs with One Qdisc

2017-04-25 Thread Michael Ma
Hi - As I mentioned in a previous thread, we're implementing a qdisc similar to mqprio which can associate multiple txqs to one qdisc, so that we can work around the root qdisc bottleneck. I think this should be in general fine since without MQ, root qdisc is associated with multiple txqs anyway.

Re: Corrupted SKB

2017-04-25 Thread Michael Ma
2017-04-18 21:46 GMT-07:00 Michael Ma <make0...@gmail.com>: > 2017-04-18 16:12 GMT-07:00 Cong Wang <xiyou.wangc...@gmail.com>: >> On Mon, Apr 17, 2017 at 5:39 PM, Michael Ma <make0...@gmail.com> wrote: >>> Hi - >>> >>> We've implemented a &quo

Re: Per-CPU Queueing for QoS

2017-11-14 Thread Michael Ma
2017-11-13 18:18 GMT-08:00 Michael Ma <make0...@gmail.com>: > 2017-11-13 16:13 GMT-08:00 Alexander Duyck <alexander.du...@gmail.com>: >> On Mon, Nov 13, 2017 at 3:08 PM, Eric Dumazet <eric.duma...@gmail.com> wrote: >>> On Mon, 2017-11-13 at 14:47 -0800, Alexan

Re: Per-CPU Queueing for QoS

2017-11-14 Thread Michael Ma
2017-11-13 20:53 GMT-08:00 Tom Herbert <t...@herbertland.com>: > On Mon, Nov 13, 2017 at 7:45 PM, John Fastabend > <john.fastab...@gmail.com> wrote: >> On 11/13/2017 06:18 PM, Michael Ma wrote: >>> 2017-11-13 16:13 GMT-08:00 Alexander Duyck <alexander.du...@g

Re: Per-CPU Queueing for QoS

2017-11-14 Thread Michael Ma
2017-11-13 19:45 GMT-08:00 John Fastabend <john.fastab...@gmail.com>: > On 11/13/2017 06:18 PM, Michael Ma wrote: >> 2017-11-13 16:13 GMT-08:00 Alexander Duyck <alexander.du...@gmail.com>: >>> On Mon, Nov 13, 2017 at 3:08 PM, Eric Dumazet <eric.duma...@gmail.com&

Re: Per-CPU Queueing for QoS

2017-11-13 Thread Michael Ma
2017-11-13 15:08 GMT-08:00 Eric Dumazet <eric.duma...@gmail.com>: > On Mon, 2017-11-13 at 14:47 -0800, Alexander Duyck wrote: >> On Mon, Nov 13, 2017 at 10:17 AM, Michael Ma <make0...@gmail.com> wrote: >> > 2017-11-12 16:14 GMT-08:00 Stephen Hemminger <step...@netw

Re: Per-CPU Queueing for QoS

2017-11-13 Thread Michael Ma
2017-11-13 14:47 GMT-08:00 Alexander Duyck <alexander.du...@gmail.com>: > On Mon, Nov 13, 2017 at 10:17 AM, Michael Ma <make0...@gmail.com> wrote: >> 2017-11-12 16:14 GMT-08:00 Stephen Hemminger <step...@networkplumber.org>: >>> On Sun, 12 Nov 2017 13:43:13 -080

Re: Per-CPU Queueing for QoS

2017-11-13 Thread Michael Ma
2017-11-13 18:05 GMT-08:00 Michael Ma <make0...@gmail.com>: > 2017-11-13 15:08 GMT-08:00 Eric Dumazet <eric.duma...@gmail.com>: >> On Mon, 2017-11-13 at 14:47 -0800, Alexander Duyck wrote: >>> On Mon, Nov 13, 2017 at 10:17 AM, Michael Ma <make0...@gmail.com> wro

Re: Per-CPU Queueing for QoS

2017-11-13 Thread Michael Ma
2017-11-13 16:13 GMT-08:00 Alexander Duyck <alexander.du...@gmail.com>: > On Mon, Nov 13, 2017 at 3:08 PM, Eric Dumazet <eric.duma...@gmail.com> wrote: >> On Mon, 2017-11-13 at 14:47 -0800, Alexander Duyck wrote: >>> On Mon, Nov 13, 2017 at 10:17 AM, Michael

Re: Per-CPU Queueing for QoS

2017-11-13 Thread Michael Ma
2017-11-12 16:14 GMT-08:00 Stephen Hemminger <step...@networkplumber.org>: > On Sun, 12 Nov 2017 13:43:13 -0800 > Michael Ma <make0...@gmail.com> wrote: > >> Any comments? We plan to implement this as a qdisc and appreciate any early >> feedback. >> >&

Re: Per-CPU Queueing for QoS

2017-11-12 Thread Michael Ma
Any comments? We plan to implement this as a qdisc and appreciate any early feedback. Thanks, Michael > On Nov 9, 2017, at 5:20 PM, Michael Ma <make0...@gmail.com> wrote: > > Currently txq/qdisc selection is based on flow hash so packets from > the same flow will fol

Per-CPU Queueing for QoS

2017-11-09 Thread Michael Ma
Currently txq/qdisc selection is based on flow hash so packets from the same flow will follow the order when they enter qdisc/txq, which avoids out-of-order problem. To improve the concurrency of QoS algorithm we plan to have multiple per-cpu queues for a single TC class and do busy polling from