Hi All, VHOST Client reconnection also works without any problems. Suggested documentation changes will be part of v4 patch which will be available shortly.
Br, Michal. > -----Original Message----- > From: Stokes, Ian > Sent: Monday, July 24, 2017 7:15 PM > To: Darrell Ball <[email protected]>; Kevin Traynor <[email protected]>; > Weglicki, MichalX <[email protected]>; > [email protected] > Subject: RE: [ovs-dev] [PATCH v3] Update relevant artifacts to add support > for DPDK 17.05.1. > > > On 7/19/17, 9:40 AM, "[email protected] on behalf of Kevin > > Traynor" <[email protected] on behalf of > > [email protected]> wrote: > > > > On 07/19/2017 10:30 AM, Michal Weglicki wrote: > > > Upgrading to DPDK 17.05.1 stable release adds new > > > significant features relevant to OVS, including, > > > but not limited to: > > > - tun/tap PMD, > > > - VFIO hotplug support, > > > - Generic flow API. > > > > > > Following changes are applied: > > > - netdev-dpdk: Changes required by DPDK API modifications. > > > - doc: Because of DPDK API changes, backward compatibility > > > with previous DPDK releases will be broken, thus all > > > relevant documentation entries are updated. > > > - .travis: DPDK version change from 16.11.1 to 17.05.1. > > > - rhel/openvswitch-fedora.spec.in: DPDK version change > > > from 16.11 to 17.05.1 > > > > > > > Hi Michal, were you able to check vhost features like multi-queue and > > vhostclient reconnect are still working ok with this patch? > Hi All, > > I've tested the vhost multi queue aspect of this patch today and I found no > issues with it. > > Thanks > Ian > > > > Few comments on the docs below. > > > > > v1->v2: Patch rebase. > > > v2->v3: Fixed wrong formating after v2 patch rebase. > > > > > > Signed-off-by: Michal Weglicki <[email protected]> > > > Reviewed-by: Aaron Conole <[email protected]> > > > --- > > > .travis/linux-build.sh | 2 +- > > > Documentation/intro/install/dpdk.rst | 8 +- > > > Documentation/topics/dpdk/vhost-user.rst | 8 +- > > > NEWS | 1 + > > > lib/netdev-dpdk.c | 144 +++++++++++++++++++- > > ----------- > > > rhel/openvswitch-fedora.spec.in | 2 +- > > > tests/dpdk/ring_client.c | 6 +- > > > 7 files changed, 105 insertions(+), 66 deletions(-) > > > > > > diff --git a/.travis/linux-build.sh b/.travis/linux-build.sh > > > index f66b534..efccdf1 100755 > > > --- a/.travis/linux-build.sh > > > +++ b/.travis/linux-build.sh > > > @@ -80,7 +80,7 @@ fi > > > > > > if [ "$DPDK" ]; then > > > if [ -z "$DPDK_VER" ]; then > > > - DPDK_VER="16.11.2" > > > + DPDK_VER="17.05.1" > > > fi > > > install_dpdk $DPDK_VER > > > if [ "$CC" = "clang" ]; then > > > diff --git a/Documentation/intro/install/dpdk.rst > > b/Documentation/intro/install/dpdk.rst > > > index a05aa1a..4a178f3 100644 > > > --- a/Documentation/intro/install/dpdk.rst > > > +++ b/Documentation/intro/install/dpdk.rst > > > @@ -40,7 +40,7 @@ Build requirements > > > In addition to the requirements described in :doc:`general`, > > building Open > > > vSwitch with DPDK will require the following: > > > > > > -- DPDK 16.11 > > > +- DPDK 17.05.1 > > > > > > - A `DPDK supported NIC`_ > > > > > > @@ -69,9 +69,9 @@ Install DPDK > > > #. Download the `DPDK sources`_, extract the file and set > > ``DPDK_DIR``:: > > > > > > $ cd /usr/src/ > > > - $ wget https://urldefense.proofpoint.com/v2/url?u=http- > > 3A__fast.dpdk.org_rel_dpdk- > > 2D16.11.2.tar.xz&d=DwICAg&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih- > > uZnsw&m=k97jBbskm5qtu47B5YEDKLa6LCgJq7IpV9fRXIASszk&s=VXewRbmVuHTeOAdaJZA9 > > ivHiJCZFh617b1ay74Fjhqs&e= > > > - $ tar xf dpdk-16.11.2.tar.xz > > > - $ export DPDK_DIR=/usr/src/dpdk-stable-16.11.2 > > > + $ wget https://urldefense.proofpoint.com/v2/url?u=http- > > 3A__fast.dpdk.org_rel_dpdk- > > 2D17.05.1.tar.xz&d=DwICAg&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih- > > uZnsw&m=k97jBbskm5qtu47B5YEDKLa6LCgJq7IpV9fRXIASszk&s=2ExT20R- > > aUXe1Fm7vJWI1NTBBseyM6Tyz3youB2btL8&e= > > > + $ tar xf dpdk-17.05.1.tar.xz > > > + $ export DPDK_DIR=/usr/src/dpdk-stable-17.05.1 > > > $ cd $DPDK_DIR > > > > > > #. (Optional) Configure DPDK as a shared library > > > > There is a reference to DPDK16.11 release notes at the end of this > > file > > > > > diff --git a/Documentation/topics/dpdk/vhost-user.rst > > b/Documentation/topics/dpdk/vhost-user.rst > > > index e76da5f..9f11ea1 100644 > > > --- a/Documentation/topics/dpdk/vhost-user.rst > > > +++ b/Documentation/topics/dpdk/vhost-user.rst > > > @@ -292,9 +292,9 @@ To begin, instantiate a guest as described in > > :ref:`dpdk-vhost-user` or > > > DPDK sources to VM and build DPDK:: > > > > > > $ cd /root/dpdk/ > > > - $ wget https://urldefense.proofpoint.com/v2/url?u=http- > > 3A__fast.dpdk.org_rel_dpdk- > > 2D16.11.2.tar.xz&d=DwICAg&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih- > > uZnsw&m=k97jBbskm5qtu47B5YEDKLa6LCgJq7IpV9fRXIASszk&s=VXewRbmVuHTeOAdaJZA9 > > ivHiJCZFh617b1ay74Fjhqs&e= > > > - $ tar xf dpdk-16.11.2.tar.xz > > > - $ export DPDK_DIR=/root/dpdk/dpdk-stable-16.11.2 > > > + $ wget https://urldefense.proofpoint.com/v2/url?u=http- > > 3A__fast.dpdk.org_rel_dpdk- > > 2D17.05.1.tar.xz&d=DwICAg&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih- > > uZnsw&m=k97jBbskm5qtu47B5YEDKLa6LCgJq7IpV9fRXIASszk&s=2ExT20R- > > aUXe1Fm7vJWI1NTBBseyM6Tyz3youB2btL8&e= > > > + $ tar xf dpdk-17.05.1.tar.xz > > > + $ export DPDK_DIR=/root/dpdk/dpdk-stable-17.05.1 > > > $ export DPDK_TARGET=x86_64-native-linuxapp-gcc > > > $ export DPDK_BUILD=$DPDK_DIR/$DPDK_TARGET > > > $ cd $DPDK_DIR > > > @@ -378,7 +378,7 @@ Sample XML > > > </disk> > > > <disk type='dir' device='disk'> > > > <driver name='qemu' type='fat'/> > > > - <source dir='/usr/src/dpdk-stable-16.11.2'/> > > > + <source dir='/usr/src/dpdk-stable-17.05.1'/> > > > <target dev='vdb' bus='virtio'/> > > > <readonly/> > > > </disk> > > > > There are references to /tools/dpdk-devbind.py in a few docs, these > > should change to /usertools/dpdk-devbind.py > > > > > diff --git a/NEWS b/NEWS > > > index d61fc5f..3f58e8b 100644 > > > --- a/NEWS > > > +++ b/NEWS > > > @@ -21,6 +21,7 @@ Post-v2.7.0 > > > still can be configured via extra arguments for DPDK EAL. > > > * dpdkvhostuser ports are marked as deprecated. They will be > > removed > > > in an upcoming release. > > > + * Support for DPDK v17.05.1. > > > > I think you can add into the faq OVS/DPDK table "2.8.x 17.05.1" as > > part > > of this or a follow up patch, if you want to be sure this makes OVS > > 2.8 > > first. > > > > The change to releases.rst should be made as part of this patch. > > It will make into OVS 2.8. > > > > > > > - IPFIX now provides additional counters: > > > * Total counters since metering process startup. > > > * Per-flow TCP flag counters. > > > diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c > > > index ea17b97..5e767e1 100644 > > > --- a/lib/netdev-dpdk.c > > > +++ b/lib/netdev-dpdk.c > > > @@ -22,6 +22,9 @@ > > > #include <stdlib.h> > > > #include <errno.h> > > > #include <unistd.h> > > > +#include <linux/virtio_net.h> > > > +#include <sys/socket.h> > > > +#include <linux/if.h> > > > > > > #include <rte_config.h> > > > #include <rte_cycles.h> > > > @@ -31,7 +34,7 @@ > > > #include <rte_malloc.h> > > > #include <rte_mbuf.h> > > > #include <rte_meter.h> > > > -#include <rte_virtio_net.h> > > > +#include <rte_vhost.h> > > > > > > #include "dirs.h" > > > #include "dp-packet.h" > > > @@ -56,6 +59,8 @@ > > > #include "timeval.h" > > > #include "unixctl.h" > > > > > > +enum {VIRTIO_RXQ, VIRTIO_TXQ, VIRTIO_QNUM}; > > > + > > > VLOG_DEFINE_THIS_MODULE(netdev_dpdk); > > > static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 20); > > > > > > @@ -171,6 +176,21 @@ static const struct rte_eth_conf port_conf = { > > > }, > > > }; > > > > > > +/* > > > + * These callbacks allow virtio-net devices to be added to vhost > > ports when > > > + * configuration has been fully completed. > > > + */ > > > +static int new_device(int vid); > > > +static void destroy_device(int vid); > > > +static int vring_state_changed(int vid, uint16_t queue_id, int > > enable); > > > +static const struct vhost_device_ops virtio_net_device_ops = > > > +{ > > > + .new_device = new_device, > > > + .destroy_device = destroy_device, > > > + .vring_state_changed = vring_state_changed, > > > + .features_changed = NULL > > > +}; > > > + > > > enum { DPDK_RING_SIZE = 256 }; > > > BUILD_ASSERT_DECL(IS_POW2(DPDK_RING_SIZE)); > > > enum { DRAIN_TSC = 200000ULL }; > > > @@ -412,8 +432,8 @@ struct netdev_rxq_dpdk { > > > dpdk_port_t port_id; > > > }; > > > > > > -static int netdev_dpdk_class_init(void); > > > -static int netdev_dpdk_vhost_class_init(void); > > > +static void netdev_dpdk_destruct(struct netdev *netdev); > > > +static void netdev_dpdk_vhost_destruct(struct netdev *netdev); > > > > > > int netdev_dpdk_get_vid(const struct netdev_dpdk *dev); > > > > > > @@ -423,8 +443,8 @@ netdev_dpdk_get_ingress_policer(const struct > > netdev_dpdk *dev); > > > static bool > > > is_dpdk_class(const struct netdev_class *class) > > > { > > > - return class->init == netdev_dpdk_class_init > > > - || class->init == netdev_dpdk_vhost_class_init; > > > + return class->destruct == netdev_dpdk_destruct > > > + || class->destruct == netdev_dpdk_vhost_destruct; > > > } > > > > > > /* DPDK NIC drivers allocate RX buffers at a particular > > granularity, typically > > > @@ -948,13 +968,45 @@ netdev_dpdk_vhost_construct(struct netdev > > *netdev) > > > if (err) { > > > VLOG_ERR("vhost-user socket device setup failure for socket > > %s\n", > > > dev->vhost_id); > > > + goto out; > > > } else { > > > fatal_signal_add_file_to_unlink(dev->vhost_id); > > > VLOG_INFO("Socket %s created for vhost-user port %s\n", > > > dev->vhost_id, name); > > > } > > > + > > > + err = rte_vhost_driver_callback_register(dev->vhost_id, > > > + > > &virtio_net_device_ops); > > > + if (err) { > > > + VLOG_ERR("rte_vhost_driver_callback_register failed for > > vhost user " > > > + "port: %s\n", name); > > > + goto out; > > > + } > > > + > > > + err = rte_vhost_driver_disable_features(dev->vhost_id, > > > + 1ULL << VIRTIO_NET_F_HOST_TSO4 > > > + | 1ULL << VIRTIO_NET_F_HOST_TSO6 > > > + | 1ULL << VIRTIO_NET_F_CSUM); > > > + if (err) { > > > + VLOG_ERR("rte_vhost_driver_disable_features failed for > > vhost user " > > > + "port: %s\n", name); > > > + goto out; > > > + } > > > + > > > + err = rte_vhost_driver_start(dev->vhost_id); > > > + if (err) { > > > + VLOG_ERR("rte_vhost_driver_start failed for vhost user " > > > + "port: %s\n", name); > > > + goto out; > > > + } > > > + > > > err = vhost_common_construct(netdev); > > > + if (err) { > > > + VLOG_ERR("vhost_common_construct failed for vhost user " > > > + "port: %s\n", name); > > > + } > > > > > > +out: > > > ovs_mutex_unlock(&dpdk_mutex); > > > VLOG_WARN_ONCE("dpdkvhostuser ports are considered deprecated; > > " > > > "please migrate to dpdkvhostuserclient ports."); > > > @@ -968,6 +1020,10 @@ netdev_dpdk_vhost_client_construct(struct > > netdev *netdev) > > > > > > ovs_mutex_lock(&dpdk_mutex); > > > err = vhost_common_construct(netdev); > > > + if (err) { > > > + VLOG_ERR("vhost_common_construct failed for vhost user > > client" > > > + "port: %s\n", netdev->name); > > > + } > > > ovs_mutex_unlock(&dpdk_mutex); > > > return err; > > > } > > > @@ -2456,12 +2512,9 @@ static void > > > set_irq_status(int vid) > > > { > > > uint32_t i; > > > - uint64_t idx; > > > > > > - for (i = 0; i < rte_vhost_get_queue_num(vid); i++) { > > > - idx = i * VIRTIO_QNUM; > > > - rte_vhost_enable_guest_notification(vid, idx + VIRTIO_RXQ, > > 0); > > > - rte_vhost_enable_guest_notification(vid, idx + VIRTIO_TXQ, > > 0); > > > + for (i = 0; i < rte_vhost_get_vring_num(vid); i++) { > > > + rte_vhost_enable_guest_notification(vid, i, 0); > > > } > > > } > > > > > > @@ -2524,7 +2577,7 @@ new_device(int vid) > > > LIST_FOR_EACH(dev, list_node, &dpdk_list) { > > > ovs_mutex_lock(&dev->mutex); > > > if (strncmp(ifname, dev->vhost_id, IF_NAME_SZ) == 0) { > > > - uint32_t qp_num = rte_vhost_get_queue_num(vid); > > > + uint32_t qp_num = > > rte_vhost_get_vring_num(vid)/VIRTIO_QNUM; > > > > > > /* Get NUMA information */ > > > newnode = rte_vhost_get_numa_node(vid); > > > @@ -2691,27 +2744,6 @@ netdev_dpdk_get_ingress_policer(const struct > > netdev_dpdk *dev) > > > return ovsrcu_get(struct ingress_policer *, &dev- > > >ingress_policer); > > > } > > > > > > -/* > > > - * These callbacks allow virtio-net devices to be added to vhost > > ports when > > > - * configuration has been fully complete. > > > - */ > > > -static const struct virtio_net_device_ops virtio_net_device_ops = > > > -{ > > > - .new_device = new_device, > > > - .destroy_device = destroy_device, > > > - .vring_state_changed = vring_state_changed > > > -}; > > > - > > > -static void * > > > -start_vhost_loop(void *dummy OVS_UNUSED) > > > -{ > > > - pthread_detach(pthread_self()); > > > - /* Put the vhost thread into quiescent state. */ > > > - ovsrcu_quiesce_start(); > > > - rte_vhost_driver_session_start(); > > > - return NULL; > > > -} > > > - > > > static int > > > netdev_dpdk_class_init(void) > > > { > > > @@ -2731,25 +2763,6 @@ netdev_dpdk_class_init(void) > > > return 0; > > > } > > > > > > -static int > > > -netdev_dpdk_vhost_class_init(void) > > > -{ > > > - static struct ovsthread_once once = OVSTHREAD_ONCE_INITIALIZER; > > > - > > > - /* This function can be called for different classes. The > > initialization > > > - * needs to be done only once */ > > > - if (ovsthread_once_start(&once)) { > > > - rte_vhost_driver_callback_register(&virtio_net_device_ops); > > > - rte_vhost_feature_disable(1ULL << VIRTIO_NET_F_HOST_TSO4 > > > - | 1ULL << VIRTIO_NET_F_HOST_TSO6 > > > - | 1ULL << VIRTIO_NET_F_CSUM); > > > - ovs_thread_create("vhost_thread", start_vhost_loop, NULL); > > > - > > > - ovsthread_once_done(&once); > > > - } > > > - > > > - return 0; > > > -} > > > > > > /* Client Rings */ > > > > > > @@ -3204,6 +3217,31 @@ netdev_dpdk_vhost_client_reconfigure(struct > > netdev *netdev) > > > "using client socket '%s'", > > > dev->up.name, dev->vhost_id); > > > } > > > + > > > + err = rte_vhost_driver_callback_register(dev->vhost_id, > > > + > > &virtio_net_device_ops); > > > + if (err) { > > > + VLOG_ERR("rte_vhost_driver_callback_register failed for > > " > > > + "vhost user client port: %s\n", dev->up.name); > > > + goto unlock; > > > + } > > > + > > > + err = rte_vhost_driver_disable_features(dev->vhost_id, > > > + 1ULL << VIRTIO_NET_F_HOST_TSO4 > > > + | 1ULL << > > VIRTIO_NET_F_HOST_TSO6 > > > + | 1ULL << VIRTIO_NET_F_CSUM); > > > + if (err) { > > > + VLOG_ERR("rte_vhost_driver_disable_features failed for > > vhost user " > > > + "client port: %s\n", dev->up.name); > > > + goto unlock; > > > + } > > > + > > > + err = rte_vhost_driver_start(dev->vhost_id); > > > + if (err) { > > > + VLOG_ERR("rte_vhost_driver_start failed for vhost user > > " > > > + "client port: %s\n", dev->up.name); > > > + goto unlock; > > > + } > > > } > > > > > > err = dpdk_vhost_reconfigure_helper(dev); > > > @@ -3324,7 +3362,7 @@ static const struct netdev_class > > dpdk_ring_class = > > > static const struct netdev_class dpdk_vhost_class = > > > NETDEV_DPDK_CLASS( > > > "dpdkvhostuser", > > > - netdev_dpdk_vhost_class_init, > > > + NULL, > > > netdev_dpdk_vhost_construct, > > > netdev_dpdk_vhost_destruct, > > > NULL, > > > @@ -3339,7 +3377,7 @@ static const struct netdev_class > > dpdk_vhost_class = > > > static const struct netdev_class dpdk_vhost_client_class = > > > NETDEV_DPDK_CLASS( > > > "dpdkvhostuserclient", > > > - netdev_dpdk_vhost_class_init, > > > + NULL, > > > netdev_dpdk_vhost_client_construct, > > > netdev_dpdk_vhost_destruct, > > > netdev_dpdk_vhost_client_set_config, > > > diff --git a/rhel/openvswitch-fedora.spec.in b/rhel/openvswitch- > > fedora.spec.in > > > index 3a045d3..2bb7102 100644 > > > --- a/rhel/openvswitch-fedora.spec.in > > > +++ b/rhel/openvswitch-fedora.spec.in > > > @@ -84,7 +84,7 @@ BuildRequires: libcap-ng libcap-ng-devel > > > %endif > > > %if %{with dpdk} > > > BuildRequires: libpcap-devel numactl-devel > > > -BuildRequires: dpdk-devel >= 16.11 > > > +BuildRequires: dpdk-devel >= 17.05.1 > > > Provides: %{name}-dpdk = %{version}-%{release} > > > %endif > > > > > > diff --git a/tests/dpdk/ring_client.c b/tests/dpdk/ring_client.c > > > index b153713..8cc3fb5 100644 > > > --- a/tests/dpdk/ring_client.c > > > +++ b/tests/dpdk/ring_client.c > > > @@ -185,15 +185,15 @@ main(int argc, char *argv[]) > > > /* Try dequeuing max possible packets first, if that fails, > > get the > > > * most we can. Loop body should only execute once, > > maximum. > > > */ > > > - while (unlikely(rte_ring_dequeue_bulk(rx_ring, pkts, > > rx_pkts) != 0) && > > > - rx_pkts > 0) { > > > + while (unlikely(rte_ring_dequeue_bulk(rx_ring, pkts, > > > + rx_pkts, NULL) != 0) && rx_pkts > 0) { > > > rx_pkts = (uint16_t)RTE_MIN(rte_ring_count(rx_ring), > > PKT_READ_SIZE); > > > } > > > > > > if (rx_pkts > 0) { > > > /* blocking enqueue */ > > > do { > > > - rslt = rte_ring_enqueue_bulk(tx_ring, pkts, > > rx_pkts); > > > + rslt = rte_ring_enqueue_bulk(tx_ring, pkts, > > rx_pkts, NULL); > > > } while (rslt == -ENOBUFS); > > > } > > > } > > > > > > > _______________________________________________ > > dev mailing list > > [email protected] > > https://urldefense.proofpoint.com/v2/url?u=https- > > 3A__mail.openvswitch.org_mailman_listinfo_ovs- > > 2Ddev&d=DwICAg&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih- > > uZnsw&m=k97jBbskm5qtu47B5YEDKLa6LCgJq7IpV9fRXIASszk&s=8mA- > > wZogFjmIwTKBY9pGKuhiFDblnvK0PWhPsJsvl9M&e= > > > > > > _______________________________________________ > > dev mailing list > > [email protected] > > https://mail.openvswitch.org/mailman/listinfo/ovs-dev _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
