On 08/09/2020 16:26, Ian Stokes wrote: > Support for vhost-user dequeue zero-copy was deprecated in OVS 2.14 with > the aim of removing it for OVS 2.15. Support for zero-copy will also be > removed from DPDK 20.11. As such remove support from OVS. >
Maybe you can add some links to the DPDK patches and OVS discussions seen as they are being requested on the ML. > Signed-off-by: Ian Stokes <[email protected]> > --- > Documentation/topics/dpdk/vhost-user.rst | 72 > -------------------------------- > NEWS | 2 + > lib/netdev-dpdk.c | 25 ----------- > vswitchd/vswitch.xml | 11 ----- > 4 files changed, 2 insertions(+), 108 deletions(-) > > diff --git a/Documentation/topics/dpdk/vhost-user.rst > b/Documentation/topics/dpdk/vhost-user.rst > index 4af738d11..595e40cde 100644 > --- a/Documentation/topics/dpdk/vhost-user.rst > +++ b/Documentation/topics/dpdk/vhost-user.rst > @@ -553,78 +553,6 @@ shown with:: > > $ ovs-vsctl get Interface dpdkvhostclient0 statistics:ovs_tx_retries > > -vhost-user Dequeue Zero Copy (experimental) > -------------------------------------------- > - > -.. warning:: > - > - vhost-user Dequeue Zero Copy is deprecated in OVS and will be removed in > - the next release. > - > -Normally when dequeuing a packet from a vHost User device, a memcpy operation > -must be used to copy that packet from guest address space to host address > -space. This memcpy can be removed by enabling dequeue zero-copy like so:: > - > - $ ovs-vsctl add-port br0 dpdkvhostuserclient0 -- set Interface \ > - dpdkvhostuserclient0 type=dpdkvhostuserclient \ > - options:vhost-server-path=/tmp/dpdkvhostclient0 \ > - options:dq-zero-copy=true > - > -With this feature enabled, a reference (pointer) to the packet is passed to > -the host, instead of a copy of the packet. Removing this memcpy can give a > -performance improvement for some use cases, for example switching large > packets > -between different VMs. However additional packet loss may be observed. > - > -Note that the feature is disabled by default and must be explicitly enabled > -by setting the ``dq-zero-copy`` option to ``true`` while specifying the > -``vhost-server-path`` option as above. If you wish to split out the command > -into multiple commands as below, ensure ``dq-zero-copy`` is set before > -``vhost-server-path``:: > - > - $ ovs-vsctl set Interface dpdkvhostuserclient0 options:dq-zero-copy=true > - $ ovs-vsctl set Interface dpdkvhostuserclient0 \ > - options:vhost-server-path=/tmp/dpdkvhostclient0 > - > -The feature is only available to ``dpdkvhostuserclient`` port types. > - > -A limitation exists whereby if packets from a vHost port with > -``dq-zero-copy=true`` are destined for a ``dpdk`` type port, the number of tx > -descriptors (``n_txq_desc``) for that port must be reduced to a smaller > number, > -128 being the recommended value. This can be achieved by issuing the > following > -command:: > - > - $ ovs-vsctl set Interface dpdkport options:n_txq_desc=128 > - > -Note: The sum of the tx descriptors of all ``dpdk`` ports the VM will send to > -should not exceed 128. For example, in case of a bond over two physical ports > -in balance-tcp mode, one must divide 128 by the number of links in the bond. > - > -Refer to :ref:`dpdk-queues-sizes` for more information. > - > -The reason for this limitation is due to how the zero copy functionality is > -implemented. The vHost device's 'tx used vring', a virtio structure used for > -tracking used ie. sent descriptors, will only be updated when the NIC frees > -the corresponding mbuf. If we don't free the mbufs frequently enough, that > -vring will be starved and packets will no longer be processed. One way to > -ensure we don't encounter this scenario, is to configure ``n_txq_desc`` to a > -small enough number such that the 'mbuf free threshold' for the NIC will be > hit > -more often and thus free mbufs more frequently. The value of 128 is > suggested, > -but values of 64 and 256 have been tested and verified to work too, with > -differing performance characteristics. A value of 512 can be used too, if the > -virtio queue size in the guest is increased to 1024 (available to configure > in > -QEMU versions v2.10 and greater). This value can be set like so:: > - > - $ qemu-system-x86_64 ... -chardev socket,id=char1,path=<sockpath>,server > - -netdev type=vhost-user,id=mynet1,chardev=char1,vhostforce > - -device virtio-net-pci,mac=00:00:00:00:00:01,netdev=mynet1, > - tx_queue_size=1024 > - > -Because of this limitation, this feature is considered 'experimental'. > - > -.. note:: > - > - Post-copy Live Migration is not compatible with dequeue zero copy. There's a reference to this in the post-copy section too that can be removed. > - > Further information can be found in the > `DPDK documentation > <https://doc.dpdk.org/guides-19.11/prog_guide/vhost_lib.html>`__ > diff --git a/NEWS b/NEWS > index 2f67d5047..2db246d1a 100644 > --- a/NEWS > +++ b/NEWS > @@ -1,5 +1,7 @@ > Post-v2.14.0 > --------------------- > + - DPDK: > + * Deprecated vhost-user dequeue zero-copy support removed. > Nit, not sure there's a need to say it was deprecated, and it reads a little funny as it is not being deprecated it in this release. > > v2.14.0 - 17 Aug 2020 > diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c > index 18c4adcc7..515ad612e 100644 > --- a/lib/netdev-dpdk.c > +++ b/lib/netdev-dpdk.c > @@ -2032,12 +2032,6 @@ netdev_dpdk_vhost_client_set_config(struct netdev > *netdev, > if (!nullable_string_is_equal(path, dev->vhost_id)) { > free(dev->vhost_id); > dev->vhost_id = nullable_xstrdup(path); > - /* check zero copy configuration */ > - if (smap_get_bool(args, "dq-zero-copy", false)) { > - dev->vhost_driver_flags |= RTE_VHOST_USER_DEQUEUE_ZERO_COPY; > - } else { > - dev->vhost_driver_flags &= ~RTE_VHOST_USER_DEQUEUE_ZERO_COPY; > - } > netdev_request_reconfigure(netdev); > } > } > @@ -5028,7 +5022,6 @@ netdev_dpdk_vhost_client_reconfigure(struct netdev > *netdev) > int err; > uint64_t vhost_flags = 0; > uint64_t vhost_unsup_flags; > - bool zc_enabled; > > ovs_mutex_lock(&dev->mutex); > > @@ -5054,19 +5047,6 @@ netdev_dpdk_vhost_client_reconfigure(struct netdev > *netdev) > vhost_flags |= RTE_VHOST_USER_POSTCOPY_SUPPORT; > } > > - zc_enabled = dev->vhost_driver_flags > - & RTE_VHOST_USER_DEQUEUE_ZERO_COPY; > - /* Enable zero copy flag, if requested */ > - if (zc_enabled) { > - vhost_flags |= RTE_VHOST_USER_DEQUEUE_ZERO_COPY; > - /* DPDK vHost library doesn't allow zero-copy with linear > buffers. > - * Hence disable Linear buffer. > - */ > - vhost_flags &= ~RTE_VHOST_USER_LINEARBUF_SUPPORT; > - VLOG_WARN("Zero copy enabled, disabling linear buffer" > - " check for vHost port %s", dev->up.name); > - } > - > /* Enable External Buffers if TCP Segmentation Offload is enabled. */ > if (userspace_tso_enabled()) { > vhost_flags |= RTE_VHOST_USER_EXTBUF_SUPPORT; > @@ -5083,11 +5063,6 @@ netdev_dpdk_vhost_client_reconfigure(struct netdev > *netdev) > VLOG_INFO("vHost User device '%s' created in 'client' mode, " > "using client socket '%s'", > dev->up.name, dev->vhost_id); > - if (zc_enabled) { > - VLOG_INFO("Zero copy enabled for vHost port %s", > dev->up.name); > - VLOG_WARN("Zero copy support is deprecated and will be " > - "removed in the next OVS release."); > - } > } > > err = rte_vhost_driver_callback_register(dev->vhost_id, > diff --git a/vswitchd/vswitch.xml b/vswitchd/vswitch.xml > index 81c84927f..07da2ee8c 100644 > --- a/vswitchd/vswitch.xml > +++ b/vswitchd/vswitch.xml > @@ -3236,17 +3236,6 @@ ovs-vsctl add-port br0 p0 -- set Interface p0 > type=patch options:peer=p1 \ > </p> > </column> > > - <column name="options" key="dq-zero-copy" > - type='{"type": "boolean"}'> > - <p> > - The value specifies whether or not to enable dequeue zero copy on > - the given interface. > - Must be set before vhost-server-path is specified. > - Only supported by dpdkvhostuserclient interfaces. > - The feature is considered experimental. > - </p> > - </column> > - > <column name="options" key="tx-retries-max" > type='{"type": "integer", "minInteger": 0, "maxInteger": 32}'> > <p> > Other than the minor comments above, LGTM, thanks. _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
