2016-09-13 17:09 GMT-07:00 Eric Dumazet <eric.duma...@gmail.com>: > On Tue, 2016-09-13 at 16:30 -0700, Michael Ma wrote: > >> The RX queue number I found from "ls /sys/class/net/eth0/queues" is >> 64. (is this the correct way of identifying the queue number on NIC?) >> I setup ifb with 24 queues which is equal to the TX queue number of >> eth0 and also the number of CPU cores. > > Please do not drop netdev@ from this mail exchange.
Sorry that I accidentally dropped that. > > ethtool -l eth0 > >> >> > There is no qdisc lock contention anymore AFAIK, since each cpu will use >> > a dedicate IFB queue and tasklet. >> > >> How is this achieved? I thought qdisc on ifb will still be protected >> by the qdisc root lock in __dev_xmit_skb() so essentially all threads >> processing qdisc are still serialized without using MQ? > > You have to properly setup ifb/mq like in : > > # netem based setup, installed at receiver side only > ETH=eth0 > IFB=ifb10 > #DELAY="delay 100ms" > EST="est 1sec 4sec" > #REORDER=1000us > #LOSS="loss 2.0" > TXQ=24 # change this to number of TX queues on the physical NIC > > ip link add $IFB numtxqueues $TXQ type ifb > ip link set dev $IFB up > > tc qdisc del dev $ETH ingress 2>/dev/null > tc qdisc add dev $ETH ingress 2>/dev/null > > tc filter add dev $ETH parent ffff: \ > protocol ip u32 match u32 0 0 flowid 1:1 \ > action mirred egress redirect dev $IFB > > tc qdisc del dev $IFB root 2>/dev/null > > tc qdisc add dev $IFB root handle 1: mq > for i in `seq 1 $TXQ` > do > slot=$( printf %x $(( i )) ) > tc qd add dev $IFB parent 1:$slot $EST netem \ > limit 100000 $DELAY $REORDER $LOSS > done > > If I understand correctly this is still to associate a qdisc with each ifb TXQ. How should I do this if I want to use HTB? I guess I'll need to divide the bandwidth of each class in HTB by the number of TX queues for each individual HTB qdisc associated? My original idea was to attach a HTB qdisc for each ifb queue representing a set of flows not sharing bandwidth with others so that root lock contention still happens but only affects flows in the same HTB. Did I understand the root lock contention issue incorrectly for ifb? I do see some comments in __dev_queue_xmit() about using a different code path for software devices which bypasses __dev_xmit_skb(). Does this mean ifb won't go through __dev_xmit_skb()?
