"dev" <dev-boun...@openvswitch.org> wrote on 05/12/2016 08:43:13 AM:
> From: Ilya Maximets <i.maxim...@samsung.com> > To: dev@openvswitch.org, Daniele Di Proietto <diproiet...@vmware.com> > Cc: Dyasly Sergey <s.dya...@samsung.com>, Flavio Leitner > <f...@sysclose.org>, Ilya Maximets <i.maxim...@samsung.com>, Kevin > Traynor <kevin.tray...@intel.com> > Date: 05/12/2016 08:44 AM > Subject: [ovs-dev] [PATCH RFC 4/6] dpif-netdev: XPS (Transmit Packet > Steering) implementation. > Sent by: "dev" <dev-boun...@openvswitch.org> > > If CPU number in pmd-cpu-mask is not divisible by the number of queues and > in a few more complex situations there may be unfair distribution of TX > queue-ids between PMD threads. > > For example, if we have 2 ports with 4 queues and 6 CPUs in pmd-cpu-mask > such distribution is possible: > <------------------------------------------------------------------------> > # ovs-appctl dpif-netdev/pmd-rxq-show > pmd thread numa_id 0 core_id 13: > port: vhost-user1 queue-id: 1 > port: dpdk0 queue-id: 3 > pmd thread numa_id 0 core_id 14: > port: vhost-user1 queue-id: 2 > pmd thread numa_id 0 core_id 16: > port: dpdk0 queue-id: 0 > pmd thread numa_id 0 core_id 17: > port: dpdk0 queue-id: 1 > pmd thread numa_id 0 core_id 12: > port: vhost-user1 queue-id: 0 > port: dpdk0 queue-id: 2 > pmd thread numa_id 0 core_id 15: > port: vhost-user1 queue-id: 3 > <------------------------------------------------------------------------> > > As we can see above dpdk0 port polled by threads on cores: > 12, 13, 16 and 17. > > By design of dpif-netdev, there is only one TX queue-id assigned to each > pmd thread. This queue-id's are sequential similar to core-id's. And > thread will send packets to queue with exact this queue-id regardless > of port. > > In previous example: > > pmd thread on core 12 will send packets to tx queue 0 > pmd thread on core 13 will send packets to tx queue 1 > ... > pmd thread on core 17 will send packets to tx queue 5 > > So, for dpdk0 port after truncating in netdev-dpdk: > > core 12 --> TX queue-id 0 % 4 == 0 > core 13 --> TX queue-id 1 % 4 == 1 > core 16 --> TX queue-id 4 % 4 == 0 > core 17 --> TX queue-id 5 % 4 == 1 > > As a result only 2 of 4 queues used. > > To fix this issue some kind of XPS implemented in following way: > > * TX queue-ids are allocated dynamically. > * When PMD thread first time tries to send packets to new port > it allocates less used TX queue for this port. > * PMD threads periodically performes revalidation of > allocated TX queue-ids. If queue wasn't used in last XPS_CYCLES > it will be freed while revalidation. > > Reported-by: Zhihong Wang <zhihong.w...@intel.com> > Signed-off-by: Ilya Maximets <i.maxim...@samsung.com> > --- As an FTI, the patch to lib/netdev-dummy.c didn't apply cleanly for me. Ryan _______________________________________________ dev mailing list dev@openvswitch.org http://openvswitch.org/mailman/listinfo/dev
