A mempool is currently created for a vhost port when it is added. The NUMA info of the vhost port is not known until a device is added to the port, so on multi-NUMA systems the initial NUMA node for the mempool is a best guess based on vswitchd affinity.
When a device is added to the vhost port, the NUMA info can be checked and if the guess was incorrect a mempool on the correct NUMA node created. The current scheme can have the effect of creating a mempool on a NUMA node that will not be needed and at least for a certain time period requires memory. It is also difficult for a user trying to provision memory on different NUMA nodes, if they are not sure which NUMA node the initial mempool for a vhost port will be on. This patch delays the creation of the mempool for a vhost port on multi-NUMA systems until the vhost NUMA info is known. Signed-off-by: Kevin Traynor <[email protected]> --- lib/netdev-dpdk.c | 30 +++++++++++++++++++++--------- 1 file changed, 21 insertions(+), 9 deletions(-) diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index f9535bfb4..95065060c 100644 --- a/lib/netdev-dpdk.c +++ b/lib/netdev-dpdk.c @@ -3927,5 +3927,6 @@ new_device(int vid) if (dev->requested_n_txq < qp_num || dev->requested_n_rxq < qp_num - || dev->requested_socket_id != newnode) { + || dev->requested_socket_id != newnode + || dev->dpdk_mp == NULL) { dev->requested_socket_id = newnode; dev->requested_n_rxq = qp_num; @@ -4975,7 +4976,8 @@ dpdk_vhost_reconfigure_helper(struct netdev_dpdk *dev) OVS_REQUIRES(dev->mutex) { + int vid, n_numas; + dev->up.n_txq = dev->requested_n_txq; dev->up.n_rxq = dev->requested_n_rxq; - int err; /* Always keep RX queue 0 enabled for implementations that won't @@ -4995,12 +4997,22 @@ dpdk_vhost_reconfigure_helper(struct netdev_dpdk *dev) netdev_dpdk_remap_txqs(dev); - err = netdev_dpdk_mempool_configure(dev); - if (!err) { - /* A new mempool was created or re-used. */ - netdev_change_seq_changed(&dev->up); - } else if (err != EEXIST) { - return err; + vid = netdev_dpdk_get_vid(dev); + n_numas = ovs_numa_get_n_numas(); + + if (n_numas == 1 + || (n_numas > 1 && vid >= 0)) { + + int err; + + err = netdev_dpdk_mempool_configure(dev); + if (!err) { + /* A new mempool was created or re-used. */ + netdev_change_seq_changed(&dev->up); + } else if (err != EEXIST) { + return err; + } } - if (netdev_dpdk_get_vid(dev) >= 0) { + + if (vid >= 0) { if (dev->vhost_reconfigured == false) { dev->vhost_reconfigured = true; -- 2.34.1 _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
