On Mon, Mar 15, 2021, at 09:38, David Marchand wrote: > On Wed, Feb 10, 2021 at 4:07 PM Gaetan Rivet <[email protected]> wrote: > > > > This patch series aims to improve the performance of the management > > of hw-offloads in dpif-netdev. In the current version, some setup > > will experience high memory usage and poor latency between a flow > > decision and its execution regarding hardware offloading. > > > > This series starts by measuring key metrics regarding both issues > > Those patches are introduced first to compare the current status > > with each improvements introduced. > > Offloads enqueued and inserted, as well as the latency > > from queue insertion to hardware insertion is measured. A new > > command 'ovs-appctl dpctl/offload-stats-show' is introduced > > to show the current measure. > > > > In my current performance test setup I am measuring an > > average latency hovering between 1~2 seconds. > > After the optimizations, it is reduced to 500~900 ms. > > Finally when using multiple threads and with proper driver > > support[1], it is measured in the order of 1 ms. > > > > A few modules are introduced: > > > > * An ID pool with reduced capabilities, simplifying its > > operations and allowing better performances in both > > single and multi-thread setup. > > > > * A lockless queue between PMDs / revalidators and > > offload thread(s). As the number of PMDs increases, > > contention can be high on the shared queue. > > This queue is designed to serve as message queue > > between threads. > > > > * A bounded lockless MPMC ring and some helpers for > > calculating moving averages. > > > > * A moving average module for Cumulative and Exponential > > moving averages. > > > > The netdev-offload-dpdk module is made thread-safe. > > Internal maps are made per-netdev instead, and locks are > > taken for shorter critical sections within the module. > > Hey Gaetan, > > This looks like an interesting series, I'll be looking too at it in > the coming weeks. > Just a first and easy update, I noticed an assert is triggered when > stopping OVS (at least). > 2021-03-11T20:32:01.928Z|00350|util|EMER|lib/netdev-offload.c:479: > assertion thread_is_hw_offload || thread_is_rcu failed in > netdev_offload_thread_init() > > I did not check yet where the issue is, but I noticed it too half way > of the series, and it was not on stop in this case. > You can probably catch it easily. > > I have hw-offloads enabled, 2 pf ports, 2 representors ports in a > single bridge, 2 pmds, no additional configuration. > I usually have bi directional traffic running through OVS while I restart. > > > -- > David Marchand
Hey David, Thanks for the report! This abort is added in patch 12: netdev-offload: Add multi-thread API It is relying on the offload thread being named 'hw_offload'. This name is changed by patch 22: dpif-netdev: Use one or more offload threads In the RFC series, a separate patch did a fix on the thread name [1]. I was relying on this fix happening first. I only did compilation checks in-between after squashing this patch. I can either re-introduce the fix patch separately, or rewrite the check in patch 12, then update it in patch 22. In my opinion having the fix separate was better but I can go with either solutions. Do you have a preference? (I did not have a setup this morning to do runtime checks, I will run them later. Reading the code this is my current understanding and it make sense for now.) [1]: https://mail.openvswitch.org/pipermail/ovs-dev/2020-December/378337.html -- Gaetan Rivet _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
