On 7/19/17, 9:40 AM, "[email protected] on behalf of Kevin
Traynor" <[email protected] on behalf of [email protected]>
wrote:
On 07/19/2017 10:30 AM, Michal Weglicki wrote:
> Upgrading to DPDK 17.05.1 stable release adds new
> significant features relevant to OVS, including,
> but not limited to:
> - tun/tap PMD,
> - VFIO hotplug support,
> - Generic flow API.
>
> Following changes are applied:
> - netdev-dpdk: Changes required by DPDK API modifications.
> - doc: Because of DPDK API changes, backward compatibility
> with previous DPDK releases will be broken, thus all
> relevant documentation entries are updated.
> - .travis: DPDK version change from 16.11.1 to 17.05.1.
> - rhel/openvswitch-fedora.spec.in: DPDK version change
> from 16.11 to 17.05.1
>
Hi Michal, were you able to check vhost features like multi-queue and
vhostclient reconnect are still working ok with this patch?
Few comments on the docs below.
> v1->v2: Patch rebase.
> v2->v3: Fixed wrong formating after v2 patch rebase.
>
> Signed-off-by: Michal Weglicki <[email protected]>
> Reviewed-by: Aaron Conole <[email protected]>
> ---
> .travis/linux-build.sh | 2 +-
> Documentation/intro/install/dpdk.rst | 8 +-
> Documentation/topics/dpdk/vhost-user.rst | 8 +-
> NEWS | 1 +
> lib/netdev-dpdk.c | 144
+++++++++++++++++++------------
> rhel/openvswitch-fedora.spec.in | 2 +-
> tests/dpdk/ring_client.c | 6 +-
> 7 files changed, 105 insertions(+), 66 deletions(-)
>
> diff --git a/.travis/linux-build.sh b/.travis/linux-build.sh
> index f66b534..efccdf1 100755
> --- a/.travis/linux-build.sh
> +++ b/.travis/linux-build.sh
> @@ -80,7 +80,7 @@ fi
>
> if [ "$DPDK" ]; then
> if [ -z "$DPDK_VER" ]; then
> - DPDK_VER="16.11.2"
> + DPDK_VER="17.05.1"
> fi
> install_dpdk $DPDK_VER
> if [ "$CC" = "clang" ]; then
> diff --git a/Documentation/intro/install/dpdk.rst
b/Documentation/intro/install/dpdk.rst
> index a05aa1a..4a178f3 100644
> --- a/Documentation/intro/install/dpdk.rst
> +++ b/Documentation/intro/install/dpdk.rst
> @@ -40,7 +40,7 @@ Build requirements
> In addition to the requirements described in :doc:`general`, building
Open
> vSwitch with DPDK will require the following:
>
> -- DPDK 16.11
> +- DPDK 17.05.1
>
> - A `DPDK supported NIC`_
>
> @@ -69,9 +69,9 @@ Install DPDK
> #. Download the `DPDK sources`_, extract the file and set ``DPDK_DIR``::
>
> $ cd /usr/src/
> - $ wget
https://urldefense.proofpoint.com/v2/url?u=http-3A__fast.dpdk.org_rel_dpdk-2D16.11.2.tar.xz&d=DwICAg&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-uZnsw&m=k97jBbskm5qtu47B5YEDKLa6LCgJq7IpV9fRXIASszk&s=VXewRbmVuHTeOAdaJZA9ivHiJCZFh617b1ay74Fjhqs&e=
> - $ tar xf dpdk-16.11.2.tar.xz
> - $ export DPDK_DIR=/usr/src/dpdk-stable-16.11.2
> + $ wget
https://urldefense.proofpoint.com/v2/url?u=http-3A__fast.dpdk.org_rel_dpdk-2D17.05.1.tar.xz&d=DwICAg&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-uZnsw&m=k97jBbskm5qtu47B5YEDKLa6LCgJq7IpV9fRXIASszk&s=2ExT20R-aUXe1Fm7vJWI1NTBBseyM6Tyz3youB2btL8&e=
> + $ tar xf dpdk-17.05.1.tar.xz
> + $ export DPDK_DIR=/usr/src/dpdk-stable-17.05.1
> $ cd $DPDK_DIR
>
> #. (Optional) Configure DPDK as a shared library
There is a reference to DPDK16.11 release notes at the end of this file
> diff --git a/Documentation/topics/dpdk/vhost-user.rst
b/Documentation/topics/dpdk/vhost-user.rst
> index e76da5f..9f11ea1 100644
> --- a/Documentation/topics/dpdk/vhost-user.rst
> +++ b/Documentation/topics/dpdk/vhost-user.rst
> @@ -292,9 +292,9 @@ To begin, instantiate a guest as described in
:ref:`dpdk-vhost-user` or
> DPDK sources to VM and build DPDK::
>
> $ cd /root/dpdk/
> - $ wget
https://urldefense.proofpoint.com/v2/url?u=http-3A__fast.dpdk.org_rel_dpdk-2D16.11.2.tar.xz&d=DwICAg&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-uZnsw&m=k97jBbskm5qtu47B5YEDKLa6LCgJq7IpV9fRXIASszk&s=VXewRbmVuHTeOAdaJZA9ivHiJCZFh617b1ay74Fjhqs&e=
> - $ tar xf dpdk-16.11.2.tar.xz
> - $ export DPDK_DIR=/root/dpdk/dpdk-stable-16.11.2
> + $ wget
https://urldefense.proofpoint.com/v2/url?u=http-3A__fast.dpdk.org_rel_dpdk-2D17.05.1.tar.xz&d=DwICAg&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-uZnsw&m=k97jBbskm5qtu47B5YEDKLa6LCgJq7IpV9fRXIASszk&s=2ExT20R-aUXe1Fm7vJWI1NTBBseyM6Tyz3youB2btL8&e=
> + $ tar xf dpdk-17.05.1.tar.xz
> + $ export DPDK_DIR=/root/dpdk/dpdk-stable-17.05.1
> $ export DPDK_TARGET=x86_64-native-linuxapp-gcc
> $ export DPDK_BUILD=$DPDK_DIR/$DPDK_TARGET
> $ cd $DPDK_DIR
> @@ -378,7 +378,7 @@ Sample XML
> </disk>
> <disk type='dir' device='disk'>
> <driver name='qemu' type='fat'/>
> - <source dir='/usr/src/dpdk-stable-16.11.2'/>
> + <source dir='/usr/src/dpdk-stable-17.05.1'/>
> <target dev='vdb' bus='virtio'/>
> <readonly/>
> </disk>
There are references to /tools/dpdk-devbind.py in a few docs, these
should change to /usertools/dpdk-devbind.py
> diff --git a/NEWS b/NEWS
> index d61fc5f..3f58e8b 100644
> --- a/NEWS
> +++ b/NEWS
> @@ -21,6 +21,7 @@ Post-v2.7.0
> still can be configured via extra arguments for DPDK EAL.
> * dpdkvhostuser ports are marked as deprecated. They will be
removed
> in an upcoming release.
> + * Support for DPDK v17.05.1.
I think you can add into the faq OVS/DPDK table "2.8.x 17.05.1" as part
of this or a follow up patch, if you want to be sure this makes OVS 2.8
first.
The change to releases.rst should be made as part of this patch.
It will make into OVS 2.8.
> - IPFIX now provides additional counters:
> * Total counters since metering process startup.
> * Per-flow TCP flag counters.
> diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c
> index ea17b97..5e767e1 100644
> --- a/lib/netdev-dpdk.c
> +++ b/lib/netdev-dpdk.c
> @@ -22,6 +22,9 @@
> #include <stdlib.h>
> #include <errno.h>
> #include <unistd.h>
> +#include <linux/virtio_net.h>
> +#include <sys/socket.h>
> +#include <linux/if.h>
>
> #include <rte_config.h>
> #include <rte_cycles.h>
> @@ -31,7 +34,7 @@
> #include <rte_malloc.h>
> #include <rte_mbuf.h>
> #include <rte_meter.h>
> -#include <rte_virtio_net.h>
> +#include <rte_vhost.h>
>
> #include "dirs.h"
> #include "dp-packet.h"
> @@ -56,6 +59,8 @@
> #include "timeval.h"
> #include "unixctl.h"
>
> +enum {VIRTIO_RXQ, VIRTIO_TXQ, VIRTIO_QNUM};
> +
> VLOG_DEFINE_THIS_MODULE(netdev_dpdk);
> static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 20);
>
> @@ -171,6 +176,21 @@ static const struct rte_eth_conf port_conf = {
> },
> };
>
> +/*
> + * These callbacks allow virtio-net devices to be added to vhost ports
when
> + * configuration has been fully completed.
> + */
> +static int new_device(int vid);
> +static void destroy_device(int vid);
> +static int vring_state_changed(int vid, uint16_t queue_id, int enable);
> +static const struct vhost_device_ops virtio_net_device_ops =
> +{
> + .new_device = new_device,
> + .destroy_device = destroy_device,
> + .vring_state_changed = vring_state_changed,
> + .features_changed = NULL
> +};
> +
> enum { DPDK_RING_SIZE = 256 };
> BUILD_ASSERT_DECL(IS_POW2(DPDK_RING_SIZE));
> enum { DRAIN_TSC = 200000ULL };
> @@ -412,8 +432,8 @@ struct netdev_rxq_dpdk {
> dpdk_port_t port_id;
> };
>
> -static int netdev_dpdk_class_init(void);
> -static int netdev_dpdk_vhost_class_init(void);
> +static void netdev_dpdk_destruct(struct netdev *netdev);
> +static void netdev_dpdk_vhost_destruct(struct netdev *netdev);
>
> int netdev_dpdk_get_vid(const struct netdev_dpdk *dev);
>
> @@ -423,8 +443,8 @@ netdev_dpdk_get_ingress_policer(const struct
netdev_dpdk *dev);
> static bool
> is_dpdk_class(const struct netdev_class *class)
> {
> - return class->init == netdev_dpdk_class_init
> - || class->init == netdev_dpdk_vhost_class_init;
> + return class->destruct == netdev_dpdk_destruct
> + || class->destruct == netdev_dpdk_vhost_destruct;
> }
>
> /* DPDK NIC drivers allocate RX buffers at a particular granularity,
typically
> @@ -948,13 +968,45 @@ netdev_dpdk_vhost_construct(struct netdev *netdev)
> if (err) {
> VLOG_ERR("vhost-user socket device setup failure for socket
%s\n",
> dev->vhost_id);
> + goto out;
> } else {
> fatal_signal_add_file_to_unlink(dev->vhost_id);
> VLOG_INFO("Socket %s created for vhost-user port %s\n",
> dev->vhost_id, name);
> }
> +
> + err = rte_vhost_driver_callback_register(dev->vhost_id,
> + &virtio_net_device_ops);
> + if (err) {
> + VLOG_ERR("rte_vhost_driver_callback_register failed for vhost
user "
> + "port: %s\n", name);
> + goto out;
> + }
> +
> + err = rte_vhost_driver_disable_features(dev->vhost_id,
> + 1ULL << VIRTIO_NET_F_HOST_TSO4
> + | 1ULL << VIRTIO_NET_F_HOST_TSO6
> + | 1ULL << VIRTIO_NET_F_CSUM);
> + if (err) {
> + VLOG_ERR("rte_vhost_driver_disable_features failed for vhost
user "
> + "port: %s\n", name);
> + goto out;
> + }
> +
> + err = rte_vhost_driver_start(dev->vhost_id);
> + if (err) {
> + VLOG_ERR("rte_vhost_driver_start failed for vhost user "
> + "port: %s\n", name);
> + goto out;
> + }
> +
> err = vhost_common_construct(netdev);
> + if (err) {
> + VLOG_ERR("vhost_common_construct failed for vhost user "
> + "port: %s\n", name);
> + }
>
> +out:
> ovs_mutex_unlock(&dpdk_mutex);
> VLOG_WARN_ONCE("dpdkvhostuser ports are considered deprecated; "
> "please migrate to dpdkvhostuserclient ports.");
> @@ -968,6 +1020,10 @@ netdev_dpdk_vhost_client_construct(struct netdev
*netdev)
>
> ovs_mutex_lock(&dpdk_mutex);
> err = vhost_common_construct(netdev);
> + if (err) {
> + VLOG_ERR("vhost_common_construct failed for vhost user client"
> + "port: %s\n", netdev->name);
> + }
> ovs_mutex_unlock(&dpdk_mutex);
> return err;
> }
> @@ -2456,12 +2512,9 @@ static void
> set_irq_status(int vid)
> {
> uint32_t i;
> - uint64_t idx;
>
> - for (i = 0; i < rte_vhost_get_queue_num(vid); i++) {
> - idx = i * VIRTIO_QNUM;
> - rte_vhost_enable_guest_notification(vid, idx + VIRTIO_RXQ, 0);
> - rte_vhost_enable_guest_notification(vid, idx + VIRTIO_TXQ, 0);
> + for (i = 0; i < rte_vhost_get_vring_num(vid); i++) {
> + rte_vhost_enable_guest_notification(vid, i, 0);
> }
> }
>
> @@ -2524,7 +2577,7 @@ new_device(int vid)
> LIST_FOR_EACH(dev, list_node, &dpdk_list) {
> ovs_mutex_lock(&dev->mutex);
> if (strncmp(ifname, dev->vhost_id, IF_NAME_SZ) == 0) {
> - uint32_t qp_num = rte_vhost_get_queue_num(vid);
> + uint32_t qp_num = rte_vhost_get_vring_num(vid)/VIRTIO_QNUM;
>
> /* Get NUMA information */
> newnode = rte_vhost_get_numa_node(vid);
> @@ -2691,27 +2744,6 @@ netdev_dpdk_get_ingress_policer(const struct
netdev_dpdk *dev)
> return ovsrcu_get(struct ingress_policer *, &dev->ingress_policer);
> }
>
> -/*
> - * These callbacks allow virtio-net devices to be added to vhost ports
when
> - * configuration has been fully complete.
> - */
> -static const struct virtio_net_device_ops virtio_net_device_ops =
> -{
> - .new_device = new_device,
> - .destroy_device = destroy_device,
> - .vring_state_changed = vring_state_changed
> -};
> -
> -static void *
> -start_vhost_loop(void *dummy OVS_UNUSED)
> -{
> - pthread_detach(pthread_self());
> - /* Put the vhost thread into quiescent state. */
> - ovsrcu_quiesce_start();
> - rte_vhost_driver_session_start();
> - return NULL;
> -}
> -
> static int
> netdev_dpdk_class_init(void)
> {
> @@ -2731,25 +2763,6 @@ netdev_dpdk_class_init(void)
> return 0;
> }
>
> -static int
> -netdev_dpdk_vhost_class_init(void)
> -{
> - static struct ovsthread_once once = OVSTHREAD_ONCE_INITIALIZER;
> -
> - /* This function can be called for different classes. The
initialization
> - * needs to be done only once */
> - if (ovsthread_once_start(&once)) {
> - rte_vhost_driver_callback_register(&virtio_net_device_ops);
> - rte_vhost_feature_disable(1ULL << VIRTIO_NET_F_HOST_TSO4
> - | 1ULL << VIRTIO_NET_F_HOST_TSO6
> - | 1ULL << VIRTIO_NET_F_CSUM);
> - ovs_thread_create("vhost_thread", start_vhost_loop, NULL);
> -
> - ovsthread_once_done(&once);
> - }
> -
> - return 0;
> -}
>
> /* Client Rings */
>
> @@ -3204,6 +3217,31 @@ netdev_dpdk_vhost_client_reconfigure(struct netdev
*netdev)
> "using client socket '%s'",
> dev->up.name, dev->vhost_id);
> }
> +
> + err = rte_vhost_driver_callback_register(dev->vhost_id,
> + &virtio_net_device_ops);
> + if (err) {
> + VLOG_ERR("rte_vhost_driver_callback_register failed for "
> + "vhost user client port: %s\n", dev->up.name);
> + goto unlock;
> + }
> +
> + err = rte_vhost_driver_disable_features(dev->vhost_id,
> + 1ULL << VIRTIO_NET_F_HOST_TSO4
> + | 1ULL << VIRTIO_NET_F_HOST_TSO6
> + | 1ULL << VIRTIO_NET_F_CSUM);
> + if (err) {
> + VLOG_ERR("rte_vhost_driver_disable_features failed for vhost
user "
> + "client port: %s\n", dev->up.name);
> + goto unlock;
> + }
> +
> + err = rte_vhost_driver_start(dev->vhost_id);
> + if (err) {
> + VLOG_ERR("rte_vhost_driver_start failed for vhost user "
> + "client port: %s\n", dev->up.name);
> + goto unlock;
> + }
> }
>
> err = dpdk_vhost_reconfigure_helper(dev);
> @@ -3324,7 +3362,7 @@ static const struct netdev_class dpdk_ring_class =
> static const struct netdev_class dpdk_vhost_class =
> NETDEV_DPDK_CLASS(
> "dpdkvhostuser",
> - netdev_dpdk_vhost_class_init,
> + NULL,
> netdev_dpdk_vhost_construct,
> netdev_dpdk_vhost_destruct,
> NULL,
> @@ -3339,7 +3377,7 @@ static const struct netdev_class dpdk_vhost_class =
> static const struct netdev_class dpdk_vhost_client_class =
> NETDEV_DPDK_CLASS(
> "dpdkvhostuserclient",
> - netdev_dpdk_vhost_class_init,
> + NULL,
> netdev_dpdk_vhost_client_construct,
> netdev_dpdk_vhost_destruct,
> netdev_dpdk_vhost_client_set_config,
> diff --git a/rhel/openvswitch-fedora.spec.in
b/rhel/openvswitch-fedora.spec.in
> index 3a045d3..2bb7102 100644
> --- a/rhel/openvswitch-fedora.spec.in
> +++ b/rhel/openvswitch-fedora.spec.in
> @@ -84,7 +84,7 @@ BuildRequires: libcap-ng libcap-ng-devel
> %endif
> %if %{with dpdk}
> BuildRequires: libpcap-devel numactl-devel
> -BuildRequires: dpdk-devel >= 16.11
> +BuildRequires: dpdk-devel >= 17.05.1
> Provides: %{name}-dpdk = %{version}-%{release}
> %endif
>
> diff --git a/tests/dpdk/ring_client.c b/tests/dpdk/ring_client.c
> index b153713..8cc3fb5 100644
> --- a/tests/dpdk/ring_client.c
> +++ b/tests/dpdk/ring_client.c
> @@ -185,15 +185,15 @@ main(int argc, char *argv[])
> /* Try dequeuing max possible packets first, if that fails, get
the
> * most we can. Loop body should only execute once, maximum.
> */
> - while (unlikely(rte_ring_dequeue_bulk(rx_ring, pkts, rx_pkts) !=
0) &&
> - rx_pkts > 0) {
> + while (unlikely(rte_ring_dequeue_bulk(rx_ring, pkts,
> + rx_pkts, NULL) != 0) && rx_pkts > 0) {
> rx_pkts = (uint16_t)RTE_MIN(rte_ring_count(rx_ring),
PKT_READ_SIZE);
> }
>
> if (rx_pkts > 0) {
> /* blocking enqueue */
> do {
> - rslt = rte_ring_enqueue_bulk(tx_ring, pkts, rx_pkts);
> + rslt = rte_ring_enqueue_bulk(tx_ring, pkts, rx_pkts,
NULL);
> } while (rslt == -ENOBUFS);
> }
> }
>
_______________________________________________
dev mailing list
[email protected]
https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.openvswitch.org_mailman_listinfo_ovs-2Ddev&d=DwICAg&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-uZnsw&m=k97jBbskm5qtu47B5YEDKLa6LCgJq7IpV9fRXIASszk&s=8mA-wZogFjmIwTKBY9pGKuhiFDblnvK0PWhPsJsvl9M&e=
_______________________________________________
dev mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
