This series brings in the new asynchronous vHost API's in DPDK to OVS. With the asynchronous framework, vHost-user can offload the memory copy operation to DPDK DMA-dev based enabled devices like Intel® QuickData Technology without blocking the CPU.
Usage: To enable DMA offload for vhost path, one can issue the following command: ovs-vsctl --no-wait set Open_vSwitch . other_config:vhost-async-support=true Issuing this requires restarting the vswitchd daemon. Please note that, its required to have DMA devices visible to DPDK before OVS launch either by binding them to userspace driver vfio-pci or through other facilities like assigning specific DSA worker queues to DPDK as mentioned in the driver documentation. The current assignment model is 1 DMA device per data plane thread. If no DMA devices are found for that thread, it will fall back to perform SW copy. To check if async support is enabled, one can check for the following message in the ovs-vswitchd.log dpdk|INFO|Async support enabled for vhost-user-client. To verify if DMA devices are assigned to the data plane thread, one can check the ovs-vswitchd.log for messages like: netdev_dpdk(pmd-c01/id:7)|INFO|DMA device 0000:00:04.0 with dev id: 0 assigned to pmd for vhost async copy offload. This patch was tested on : OVS : branch : dpdk-latest + v2 of defer work : http://patchwork.ozlabs.org/project/openvswitch/list/?series=261267 with DPDK : 21.08 : bfd3c352eb (main) build: fix install from any directory with Meson 0.55 + v1 of removing async threshold : https://patches.dpdk.org/project/dpdk/list/?series=18367 + v1 of async southbound path : https://patches.dpdk.org/project/dpdk/list/?series=18701 + v1 of DMA IOMMU support : https://patches.dpdk.org/project/dpdk/list/?series=18590 + v19 of the dmadev library : https://patches.dpdk.org/project/dpdk/list/?series=18629 + v2 of the dmadev test suite : https://patches.dpdk.org/project/dpdk/list/?series=18607 + v2 of the IDXD driver : https://patches.dpdk.org/project/dpdk/list/?series=18658 + v2 of the IOAT driver : https://patches.dpdk.org/project/dpdk/list/?series=18661 Please note: All the DPDK API's (both vhost async and DMADEV) utilized in the patch are experimental. Also, only IOVA as VA mode is supported currently. Future work: - vhost hot plugging is not supported yet as there is currently no means to perform this operation gracefully without leading to deadlock. Ongoing discussions with DPDK vhost team to resolve this. - Compatibility with PMD auto load balancing. --- v1 -> v2 - DMA assignment model changed to per thread from per virtqueue. - Support for async southbound path. - Automatic assignment of DMA devices. - Use generic DPDK DMADEV API. - Use per thread defer work ring to query DMA completions. - Remove async threshold. Sunil Pai G (1): netdev-dpdk: Enable vhost async API's in OvS. lib/dpdk-stub.c | 6 + lib/dpdk.c | 39 +++ lib/dpdk.h | 1 + lib/netdev-dpdk.c | 823 ++++++++++++++++++++++++++++++++++++++++++++-- lib/netdev-dpdk.h | 20 ++ 5 files changed, 865 insertions(+), 24 deletions(-) -- 2.25.1 _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
