On 17.10.2019 13:16, Noa Ezra wrote:
There are two approaches to communicate with a guest, using virtIO or SR-IOV. SR-IOV allows working with port representor which is attached to the OVS and a matching VF is given with pass-through to the VM. HW rules can process packets from up-link and direct them to the VF without going through SW (OVS) and therefore SR-IOV gives the best performance. However, SR-IOV architecture requires that the guest will use a driver which is specific to the underlying HW. Specific HW driver has two main drawbacks: 1. Breaks virtualization in some sense (VM aware of the HW), can also limit the type of images supported. 2. Less natural support for live migration.Using virtIO interface solves both problems, but reduces performance and causes losing of some functionality, for example, for some HW offload, working directly with virtIO cannot be supported. In order to solve this conflict, we created a new netdev type-dpdkvdpa. The new netdev is basically similar to a regular dpdk netdev, but it has some additional functionality for transferring packets from virtIO guest (VM) to a VF and vice versa. With this solution we can benefit both SR-IOV and virtIO. vDPA netdev is designed to support both SW and HW use-cases. HW mode will be used to configure vDPA capable devices. The support for this mode is on progress in the dpdk community. SW acceleration is used to leverage SR-IOV offloads to virtIO guests by relaying packets between VF and virtio devices and as a pre-step for supporting vDPA in HW mode. Running example: 1. Configure OVS bridge and ports: ovs-vsctl add-br br0-ovs -- set bridge br0-ovs datapath_type=netdev ovs-vsctl add-port br0-ovs pf -- set Interface pf type=dpdk options: \ dpdk-devargs=<pf pci id> ovs-vsctl add-port br0 vdpa0 -- set Interface vdpa0 type=dpdkvdpa \ options:vdpa-socket-path=<sock path> \ options:vdpa-accelerator-devargs=<vf pci id> \ options:dpdk-devargs=<pf pci id>,representor=[id] 2. Run a virtIO guest (VM) in server mode that creates the socket of the vDPA port. 3. Send traffic. Noa Ezra (2): netdev-dpdk-vdpa: Introduce dpdkvdpa netdev netdev-dpdk: Add dpdkvdpa port Documentation/automake.mk | 1 + Documentation/topics/dpdk/index.rst | 1 + Documentation/topics/dpdk/vdpa.rst | 90 +++++ NEWS | 1 + lib/automake.mk | 4 +- lib/netdev-dpdk-vdpa.c | 750 ++++++++++++++++++++++++++++++++++++ lib/netdev-dpdk-vdpa.h | 54 +++ lib/netdev-dpdk.c | 162 ++++++++ vswitchd/vswitch.xml | 25 ++ 9 files changed, 1087 insertions(+), 1 deletion(-) create mode 100644 Documentation/topics/dpdk/vdpa.rst create mode 100755 lib/netdev-dpdk-vdpa.c create mode 100644 lib/netdev-dpdk-vdpa.h
Hi, everyone. So, I have a few questions (mostly to Roni?): 1. What happened to idea of implementing this as a DPDK vdev? 2. What was the results of "direct output optimization" patch [1] testing? At this point I also have to mention that OVS is going to have TSO support in one of the next releases (at least we have some progress on that, thanks to idea of using extbuf). This way in combine with patch [1] there should be no any benefits from having separate netdev for forwarding purposes. Even without this patch, havin TSO alone should provide good performance benefits making it considerable to not have any extra netdev. [1] https://patchwork.ozlabs.org/patch/1105880/ 3. Regarding implementation, I expected less code duplication. It's unclear why you're not reusing hotplug capabilities of exiting dpdk netdev. I expected that this new netdev will call netdev_open() like a 3 times with a few tweaks for TSO enabling or stripping/adding some options. 4. This code seem doesn't make sense in sw mode without full HW offloading. i.e. it doesn't make sense to use this netdev while VF representor is attached to userspace datapath. This is just because, according to the solution architecture, all the traffic will be copied firstly from the VM to VF (parsed on the fly to enable TSO), and will immediately appear on VF representor to be handled by OVS (same parsing with partial offload + actions execution). I'm completely unsure if it any faster than just sending packet from VM to VF withoout bypassing OVS processing. Maybe even slower and like 3 times heavier for PCI bandwidth. 5. "HW mode will be used to configure vDPA capable devices. The support for this mode is on progress in the dpdk community." AFAIU, host side of vDPA support is done already for some time in DPDK. API is prepared and there is a vhost-vdpa example that allows you to check the functionality. As I understand, the missing part in dpdk right now is a DPDK virtio driver for guest, but this is not a limitation for not implementing vDPA support from the host side. Am I missing something? Best regards, Ilya Maximets. _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
