Hi, Kevin
Thanks for reviewing.
On Thu, Jun 5, 2025 at 8:44 PM Kevin Traynor <ktray...@redhat.com> wrote:
>
> On 08/05/2025 04:50, Changliang Wu wrote:
> > The ovs op hangs when performing:
> > 1. Create a DPDK bridge (datapath_type=netdev) with bind-ed DPDK port
> > 2. Restart OVS service (systemctl restart openvswitch)
> > 3. Delete the bridge (ovs-vsctl del-br)
> > 4. Recreate bridge and re-add DPDK port
> >
> > Root cause:
> > During OVS service restart, ovs-vswitchd reloads all ports from OVSDB via:
> > dpdk_init() -> dpdk_init__() -> rte_eal_init() -> ... ->
> > rte_eth_dev_probing_finish()
> > This leaves port in DPDK's global rte_eth_devices array with
> > dev->state=RTE_ETH_DEV_ATTACHED
> >
> > When recreating port:
> > netdev_dpdk_process_devargs() calls rte_eth_dev_is_valid_port(port_id)
> > Since port_id is valid, skips rte_dev_probe() and does not set
> > netdev->attached=true
> >
>
> Hi,
>
> Do you hit an issue with this when running, or is this from review ?
I hit this issue in the production environment.
I restarted the service because of upgrading ovs,
and then deleted the bridge,
and later found that these ports on the old bridge
could no longer be added to the new bridge.
>
> rte_eal_init() will probe all devices at init (except when dpdk-extra
> has -a/-b args to allow/block specific devices). Probed devices in DPDK
> have rte_eth_devices[].state set as RTE_ETH_DEV_ATTACHED. Other devices
> are set as RTE_ETH_DEV_UNUSED.
>
> If a device that is not already probed is added as a port in OVS then
> OVS will call probe for it. In DPDK, rte_eth_devices[].state is then set
> to RTE_ETH_DEV_ATTACHED.
>
> Note, OVS dev->attached=true is only used to track whether the device
> was explicitly probed from OVS, so that when the port is deleted we can
> remove the device and return it to it's previous RTE_ETH_DEV_UNUSED
> state in DPDK.
>
> OVS dev->attached is *not* a mirror of
> rte_eth_devices[].state=RTE_ETH_DEV_ATTACHED and may have a different value.
>
> The existing code looks ok to me. Your change would mean that devices
> that were not explicitly probed by OVS are being removed by OVS. That's
> not something we want to do, perhaps there could be some hotplug
> limitations and the device cannot be reused etc.
>
These above make sense to me. Modifying OVS dev->attached seems not a
good option.
> If you are having an issue with a device probed during rte_eal_init() it
> is probably something that should be reported in DPDK ml. One option is
> to set dpdk-extra like below, so any device will be explicitly probed on
> port add.
>
> ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-extra="-a
> 0000:00:00.0"
It works for my problem, and also found that I can explicitly detach
the device with
ovs-appctl netdev-dpdk/detach 0000:00:00.0
In fact, the problem here is that when deleting the bridge,
the status of all ports on the bridge is not reset.
In contrast, if delete the port directly, the status is reset normally,
regardless of whether the port is added manually or automatically
after restarting the service.
So maybe we need to discuss whether there is a problem with this scenario?
>
> thanks,
> Kevin.
>
> > During bridge/port deletion:
> > netdev_dpdk_destruct() checks if(dev->attached)
> > and fail to execute cleanup logic for attached devices
> > Leave a device in rte_eth_devices with RTE_ETH_DEV_ATTACHED
> >
> > Subsequent port addition:
> > rte_eth_dev_is_valid_port() still returns true,
> > critical PCI initialization (rte_dev_probe()) was skip,
> > the following configurations hang due to uninitialized device state
> >
> > Fix implementation:
> > Modify netdev_dpdk_process_devargs() to set netdev->attached=true
> > explicitly,
> > when port_id is not DPDK_ETH_PORT_ID_INVALID.
> >
> > Signed-off-by: Changliang Wu <changliang...@smartx.com>
> > ---
> > lib/netdev-dpdk.c | 3 ++-
> > 1 file changed, 2 insertions(+), 1 deletion(-)
> >
> > diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c
> > index e2708a8a5..a551b204e 100644
> > --- a/lib/netdev-dpdk.c
> > +++ b/lib/netdev-dpdk.c
> > @@ -2129,7 +2129,6 @@ netdev_dpdk_process_devargs(struct netdev_dpdk *dev,
> > new_port_id = netdev_dpdk_get_port_by_devargs(devargs);
> > if (rte_eth_dev_is_valid_port(new_port_id)) {
> > /* Attach successful */
> > - dev->attached = true;
> > VLOG_INFO("Device '%s' attached to DPDK", devargs);
> > } else {
> > /* Attach unsuccessful */
> > @@ -2141,6 +2140,8 @@ netdev_dpdk_process_devargs(struct netdev_dpdk *dev,
> >
> > if (new_port_id == DPDK_ETH_PORT_ID_INVALID) {
> > VLOG_WARN_BUF(errp, "Error attaching device '%s' to DPDK",
> > devargs);
> > + } else {
> > + dev->attached = true;
> > }
> >
> > return new_port_id;
>
thanks,
Changliang
_______________________________________________
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
