In order to use dpdk ports in ovs they have to be bound to a DPDK compatible driver before ovs is started.
This patch adds the possibility to hotplug (or hot-unplug) a device after ovs has been started. The implementation adds two appctl commands: netdev-dpdk/port-attach and netdev-dpdk/port-detach After the user attaches a new device, it has to be added to a bridge using the add-port command, similarly, before detaching a device, it has to be removed using the del-port command. Signed-off-by: Mauricio Vasquez B <mauricio.vasquezber...@studenti.polito.it> --- v7: - rebase to master v6: - add explicit comment about supporting VFIO - rebase to master v5: - use two appctl commands instead of a single one - rebase to master v4: - fix typo in commit message - remove unnecessary whitespace change in INSTALL.DPDK.md v3: - create dpdk_port_attach and dpdk_port_detach functions - modify mutex locking order v2: - use rte_eth_dev_is_valid_port() to check if a port is valid INSTALL.DPDK-ADVANCED.md | 25 ++++++++++++ NEWS | 1 + lib/netdev-dpdk.c | 101 +++++++++++++++++++++++++++++++++++++++++++---- 3 files changed, 120 insertions(+), 7 deletions(-) diff --git a/INSTALL.DPDK-ADVANCED.md b/INSTALL.DPDK-ADVANCED.md index 9ae536d..61b4e82 100644 --- a/INSTALL.DPDK-ADVANCED.md +++ b/INSTALL.DPDK-ADVANCED.md @@ -12,6 +12,7 @@ OVS DPDK ADVANCED INSTALL GUIDE 7. [QOS](#qos) 8. [Rate Limiting](#rl) 9. [Vsperf](#vsperf) +10. [Port Hotplug](#hotplug) ## <a name="overview"></a> 1. Overview @@ -835,6 +836,29 @@ environment. More information can be found in below link. https://wiki.opnfv.org/display/vsperf/VSperf+Home +## <a name="hotplug"></a> 10. Port Hotplug + +OvS supports port hotplugging, it allows to use ports that were not bound +to DPDK when vswitchd was started. +In order to attach a port, it has to be bound to DPDK using the +dpdk_nic_bind.py script: + +`$DPDK_DIR/tools/dpdk_nic_bind.py --bind=igb_uio 0000:01:00.0` + +Then it can be attached to OVS: + +`ovs-appctl netdev-dpdk/port-attach 0000:01:00.0` + +At this point, the user can create a ovs port using the add-port command. + +It is also possible to detach a port from ovs, the user has to remove the +port using the del-port command, then it can be detached using: + +`ovs-appctl netdev-dpdk/port-detach dpdk0` + +This feature is not supported with VFIO and could not work with some NICs, +please refer to the [DPDK Port Hotplug Framework] in order to get more +information. Bug Reporting: -------------- @@ -850,3 +874,4 @@ Please report problems to b...@openvswitch.org. [Guest VM using libvirt]: INSTALL.DPDK.md#ovstc [INSTALL DPDK]: INSTALL.DPDK.md#build [INSTALL OVS]: INSTALL.DPDK.md#build +[DPDK Port Hotplug Framework]: http://dpdk.org/doc/guides/prog_guide/port_hotplug_framework.html diff --git a/NEWS b/NEWS index 6496dc1..9064225 100644 --- a/NEWS +++ b/NEWS @@ -58,6 +58,7 @@ Post-v2.5.0 * PMD threads servicing vHost User ports can now come from the NUMA node that device memory is located on if CONFIG_RTE_LIBRTE_VHOST_NUMA is enabled in DPDK. + * Port Hotplug is now supported. - Increase number of registers to 16. - ovs-benchmark: This utility has been removed due to lack of use and bitrot. diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index 85b18fd..3fab52c 100644 --- a/lib/netdev-dpdk.c +++ b/lib/netdev-dpdk.c @@ -644,7 +644,7 @@ dpdk_eth_dev_init(struct netdev_dpdk *dev) OVS_REQUIRES(dpdk_mutex) int diag; int n_rxq, n_txq; - if (dev->port_id < 0 || dev->port_id >= rte_eth_dev_count()) { + if (!rte_eth_dev_is_valid_port(dev->port_id)) { return ENODEV; } @@ -2172,6 +2172,83 @@ netdev_dpdk_set_admin_state(struct unixctl_conn *conn, int argc, unixctl_command_reply(conn, "OK"); } +static void +netdev_dpdk_port_attach(struct unixctl_conn *conn, int argc OVS_UNUSED, + const char *argv[], void *aux OVS_UNUSED) +{ + int ret; + char response[128]; + uint8_t port_id; + + ovs_mutex_lock(&dpdk_mutex); + + ret = rte_eth_dev_attach(argv[1], &port_id); + if (ret < 0) { + snprintf(response, sizeof(response), + "Error attaching device '%s'", argv[1]); + ovs_mutex_unlock(&dpdk_mutex); + unixctl_command_reply_error(conn, response); + return; + } + + snprintf(response, sizeof(response), + "Device '%s' has been attached as 'dpdk%d'", argv[1], port_id); + + ovs_mutex_unlock(&dpdk_mutex); + unixctl_command_reply(conn, response); +} + +static void +netdev_dpdk_port_detach(struct unixctl_conn *conn, int argc OVS_UNUSED, + const char *argv[], void *aux OVS_UNUSED) +{ + int ret; + char response[128]; + unsigned int parsed_port; + uint8_t port_id; + char devname[RTE_ETH_NAME_MAX_LEN]; + + ovs_mutex_lock(&dpdk_mutex); + + ret = dpdk_dev_parse_name(argv[1], "dpdk", &parsed_port); + if (ret) { + snprintf(response, sizeof(response), + "'%s' is not a valid port", argv[1]); + goto error; + } + + port_id = parsed_port; + + struct netdev *netdev = netdev_from_name(argv[1]); + if (netdev) { + netdev_close(netdev); + snprintf(response, sizeof(response), + "Port '%s' is being used. Remove it before detaching", + argv[1]); + goto error; + } + + rte_eth_dev_close(port_id); + + ret = rte_eth_dev_detach(port_id, devname); + if (ret < 0) { + snprintf(response, sizeof(response), + "Port '%s' can not be detached", argv[1]); + goto error; + } + + snprintf(response, sizeof(response), + "Port '%s' has been detached", argv[1]); + + ovs_mutex_unlock(&dpdk_mutex); + unixctl_command_reply(conn, response); + return; + +error: + ovs_mutex_unlock(&dpdk_mutex); + unixctl_command_reply_error(conn, response); +} + /* * Set virtqueue flags so that we do not receive interrupts. */ @@ -2457,6 +2534,15 @@ dpdk_common_init(void) "[netdev] up|down", 1, 2, netdev_dpdk_set_admin_state, NULL); + unixctl_command_register("netdev-dpdk/port-attach", + "pci address of device", 1, 1, + netdev_dpdk_port_attach, NULL); + + unixctl_command_register("netdev-dpdk/port-detach", + "port", 1, 1, + netdev_dpdk_port_detach, NULL); + + ovs_thread_create("dpdk_watchdog", dpdk_watchdog, NULL); } /* Client Rings */ @@ -2467,7 +2553,7 @@ dpdk_ring_create(const char dev_name[], unsigned int port_no, { struct dpdk_ring *ivshmem; char ring_name[RTE_RING_NAMESIZE]; - int err; + int err, port_id; ivshmem = dpdk_rte_mzalloc(sizeof *ivshmem); if (ivshmem == NULL) { @@ -2501,19 +2587,20 @@ dpdk_ring_create(const char dev_name[], unsigned int port_no, return ENOMEM; } - err = rte_eth_from_rings(dev_name, &ivshmem->cring_rx, 1, - &ivshmem->cring_tx, 1, SOCKET0); + port_id = rte_eth_from_rings(dev_name, &ivshmem->cring_rx, 1, + &ivshmem->cring_tx, 1, SOCKET0); - if (err < 0) { + if (port_id < 0) { rte_free(ivshmem); return ENODEV; } ivshmem->user_port_id = port_no; - ivshmem->eth_port_id = rte_eth_dev_count() - 1; + ivshmem->eth_port_id = port_id; + *eth_port_id = port_id; + ovs_list_push_back(&dpdk_ring_list, &ivshmem->list_node); - *eth_port_id = ivshmem->eth_port_id; return 0; } -- 1.9.1 _______________________________________________ dev mailing list dev@openvswitch.org http://openvswitch.org/mailman/listinfo/dev
