Currently mempools for vhost are being assigned before the vhost device is added. In some cases this may be just reusing an existing mempool but in others it can require creation of a mempool.
For multi-NUMA, the NUMA info of the vhost port is not known until a device is added to the port, so on multi-NUMA systems the initial NUMA node for the mempool is a best guess based on vswitchd affinity. When a device is added to the vhost port, the NUMA info can be checked and if the guess was incorrect a mempool on the correct NUMA node created. For multi-NUMA, the current scheme can have the effect of creating a mempool on a NUMA node that will not be needed and at least for a certain time period requires more memory on a NUMA node. It is also difficult for a user trying to provision memory on different NUMA nodes, if they are not sure which NUMA node the initial mempool for a vhost port will be on. For single NUMA, even though the mempool will be on the correct NUMA, it is assigned ahead of time and if a vhost device was not added, it could also be using uneeded memory. This patch delays the creation of the mempool for a vhost port until the vhost device is added. Signed-off-by: Kevin Traynor <[email protected]> Reviewed-by: David Marchand <[email protected]> --- NEWS | 4 +++- lib/netdev-dpdk.c | 21 ++++++++++++--------- 2 files changed, 15 insertions(+), 10 deletions(-) diff --git a/NEWS b/NEWS index 9fe3f44f4..9d552797f 100644 --- a/NEWS +++ b/NEWS @@ -33,5 +33,7 @@ Post-v2.17.0 * OVS validated with DPDK 21.11.1. It is recommended to use this version until further releases. - + * Delay creating or reusing a mempool for vhost ports until the vhost + device has been added. A failure to create a mempool will now also be + logged when the vhost device is added. v2.17.0 - 17 Feb 2022 diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index f9535bfb4..081900576 100644 --- a/lib/netdev-dpdk.c +++ b/lib/netdev-dpdk.c @@ -3927,5 +3927,6 @@ new_device(int vid) if (dev->requested_n_txq < qp_num || dev->requested_n_rxq < qp_num - || dev->requested_socket_id != newnode) { + || dev->requested_socket_id != newnode + || dev->dpdk_mp == NULL) { dev->requested_socket_id = newnode; dev->requested_n_rxq = qp_num; @@ -4977,5 +4978,4 @@ dpdk_vhost_reconfigure_helper(struct netdev_dpdk *dev) dev->up.n_txq = dev->requested_n_txq; dev->up.n_rxq = dev->requested_n_rxq; - int err; /* Always keep RX queue 0 enabled for implementations that won't @@ -4995,12 +4995,15 @@ dpdk_vhost_reconfigure_helper(struct netdev_dpdk *dev) netdev_dpdk_remap_txqs(dev); - err = netdev_dpdk_mempool_configure(dev); - if (!err) { - /* A new mempool was created or re-used. */ - netdev_change_seq_changed(&dev->up); - } else if (err != EEXIST) { - return err; - } if (netdev_dpdk_get_vid(dev) >= 0) { + int err; + + err = netdev_dpdk_mempool_configure(dev); + if (!err) { + /* A new mempool was created or re-used. */ + netdev_change_seq_changed(&dev->up); + } else if (err != EEXIST) { + return err; + } + if (dev->vhost_reconfigured == false) { dev->vhost_reconfigured = true; -- 2.34.3 _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
