Re: [ovs-dev] [PATCH v5] system-dpdk: Test with mlx5 devices.
On 11/22/23 17:09, David Marchand wrote: > The DPDK unit test only runs if vfio or igb_uio kernel modules are loaded: > on systems with only mlx5, this test is always skipped. > > Besides, the test tries to grab the first device listed by dpdk-devbind.py, > regardless of the PCI device status regarding kmod binding. > > Remove dependency on this DPDK script and use a minimal script that > reads PCI sysfs. > > This script is not perfect, as one can imagine PCI devices bound to > vfio-pci for virtual machines. > Plus, this script only tries to take over vfio-pci devices. mlx5 devices > can't be taken over blindly as it could mean losing connectivity to the > machine if the netdev was in use for this system. > > For those two reasons, add a new environment variable DPDK_PCI_ADDR for > testers to select the PCI device of their liking. > For consistency and grep, the temporary file PCI_ADDR is renamed > to DPDK_PCI_ADDR. > > Reviewed-by: Maxime Coquelin > Acked-by: Eelco Chaudron > Signed-off-by: David Marchand > --- > Changes since v4: > - separated from the original series, > - rebased, > - dropped mlx5 devices from the discovery script, > - documented DPDK_PCI_ADDR env variable, > > Changes since v3: > - fixed nit from Maxime, > > Changes since v2: > - sorted logs alphabetically, > > --- > Documentation/topics/testing.rst | 11 ++--- > tests/automake.mk| 1 + > tests/system-dpdk-find-device.py | 39 > tests/system-dpdk-macros.at | 10 ++-- > tests/system-dpdk.at | 14 ++-- > 5 files changed, 57 insertions(+), 18 deletions(-) > create mode 100755 tests/system-dpdk-find-device.py > > diff --git a/Documentation/topics/testing.rst > b/Documentation/topics/testing.rst > index 5f6940b84d..fb9b3e77b1 100644 > --- a/Documentation/topics/testing.rst > +++ b/Documentation/topics/testing.rst > @@ -343,15 +343,20 @@ To see a list of all the available tests, run:: > > These tests support a `DPDK supported NIC`_. The tests operate on a wider > set of > environments, for instance, when a virtual port is used. > -They do require proper DPDK variables (``DPDK_DIR`` and ``DPDK_BUILD``). > Moreover you need to have root privileges to load the required modules and > to bind > -the NIC to the DPDK-compatible driver. > +a PCI device to the DPDK-compatible driver. > > .. _DPDK supported NIC: https://core.dpdk.org/supported/#nics > > +The phy test will skip if no suitable PCI device is found. > +It is possible to select which PCI device is used for this test by setting > the > +DPDK_PCI_ADDR environment variable, which is especially useful when testing > +with a mlx5 device:: > + > +# DPDK_PCI_ADDR=:82:00.0 make check-dpdk > + > All tests are skipped if no hugepages are configured. User must look into > the DPDK > manual to figure out how to `Configure hugepages`_. > -The phy test will skip if no compatible physical device is available. > > .. _Configure hugepages: > https://doc.dpdk.org/guides-22.11/linux_gsg/sys_reqs.html > > diff --git a/tests/automake.mk b/tests/automake.mk > index f8a925012d..69117514cc 100644 > --- a/tests/automake.mk > +++ b/tests/automake.mk > @@ -192,6 +192,7 @@ SYSTEM_OFFLOADS_TESTSUITE_AT = \ > > SYSTEM_DPDK_TESTSUITE_AT = \ > tests/system-common-macros.at \ > + tests/system-dpdk-find-device.py \ I didn't review this version in full, but I think, this should go to CHECK_PYFILES instead. The *_AT macros are for .at files. Maybe Kevin can make this change before applying. Assuming the flake8-check will pass. > tests/system-dpdk-macros.at \ > tests/system-dpdk-testsuite.at \ > tests/system-dpdk.at > diff --git a/tests/system-dpdk-find-device.py > b/tests/system-dpdk-find-device.py > new file mode 100755 > index 00..b0997380bc > --- /dev/null > +++ b/tests/system-dpdk-find-device.py > @@ -0,0 +1,39 @@ > +#!/usr/bin/env python3 > +# Copyright (c) 2021 Red Hat, Inc. 2024, I guess? :) Best regards, Ilya Maximets. ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH v5] system-dpdk: Test with mlx5 devices.
On 22/11/2023 16:09, David Marchand wrote: > The DPDK unit test only runs if vfio or igb_uio kernel modules are loaded: > on systems with only mlx5, this test is always skipped. > > Besides, the test tries to grab the first device listed by dpdk-devbind.py, > regardless of the PCI device status regarding kmod binding. > > Remove dependency on this DPDK script and use a minimal script that > reads PCI sysfs. > > This script is not perfect, as one can imagine PCI devices bound to > vfio-pci for virtual machines. > Plus, this script only tries to take over vfio-pci devices. mlx5 devices > can't be taken over blindly as it could mean losing connectivity to the > machine if the netdev was in use for this system. > > For those two reasons, add a new environment variable DPDK_PCI_ADDR for > testers to select the PCI device of their liking. > For consistency and grep, the temporary file PCI_ADDR is renamed > to DPDK_PCI_ADDR. > > Reviewed-by: Maxime Coquelin > Acked-by: Eelco Chaudron > Signed-off-by: David Marchand > --- > Changes since v4: > - separated from the original series, > - rebased, > - dropped mlx5 devices from the discovery script, > - documented DPDK_PCI_ADDR env variable, > > Changes since v3: > - fixed nit from Maxime, > > Changes since v2: > - sorted logs alphabetically, > > --- > Documentation/topics/testing.rst | 11 ++--- > tests/automake.mk| 1 + > tests/system-dpdk-find-device.py | 39 > tests/system-dpdk-macros.at | 10 ++-- > tests/system-dpdk.at | 14 ++-- > 5 files changed, 57 insertions(+), 18 deletions(-) > create mode 100755 tests/system-dpdk-find-device.py Tested with a combination of mlx5 and intel NICs with/without DPDK_PCI_ADDR and the correct NIC is selected. Acked-by: Kevin Traynor ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH v5] system-dpdk: Test with mlx5 devices.
On 1 Dec 2023, at 15:37, David Marchand wrote: > On Tue, Nov 28, 2023 at 9:40 AM David Marchand > wrote: >> >> On Wed, Nov 22, 2023 at 5:34 PM David Marchand >> wrote: >>> >>> The DPDK unit test only runs if vfio or igb_uio kernel modules are loaded: >>> on systems with only mlx5, this test is always skipped. >>> >>> Besides, the test tries to grab the first device listed by dpdk-devbind.py, >>> regardless of the PCI device status regarding kmod binding. >>> >>> Remove dependency on this DPDK script and use a minimal script that >>> reads PCI sysfs. >>> >>> This script is not perfect, as one can imagine PCI devices bound to >>> vfio-pci for virtual machines. >>> Plus, this script only tries to take over vfio-pci devices. mlx5 devices >>> can't be taken over blindly as it could mean losing connectivity to the >>> machine if the netdev was in use for this system. >>> >>> For those two reasons, add a new environment variable DPDK_PCI_ADDR for >>> testers to select the PCI device of their liking. >>> For consistency and grep, the temporary file PCI_ADDR is renamed >>> to DPDK_PCI_ADDR. >>> >>> Reviewed-by: Maxime Coquelin >>> Acked-by: Eelco Chaudron >>> Signed-off-by: David Marchand >> >> This patch can't be merged as is. >> I am preparing some fixes for the system-dpdk MTU tests that got >> merged since my v4. > > Coming back on this. > I sent a fix > https://patchwork.ozlabs.org/project/openvswitch/patch/20231201142931.1782046-1-david.march...@redhat.com/. > This current patch on making it possible to select a PCI device to > test is kind of orthogonal to this fix. > Yet, people will likely want to test with both applied. Tested with both patches applied and it still looks good to me. Re-Acking for the record ;) //Eelco Acked-by: Eelco Chaudron ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH v5] system-dpdk: Test with mlx5 devices.
On Tue, Nov 28, 2023 at 9:40 AM David Marchand wrote: > > On Wed, Nov 22, 2023 at 5:34 PM David Marchand > wrote: > > > > The DPDK unit test only runs if vfio or igb_uio kernel modules are loaded: > > on systems with only mlx5, this test is always skipped. > > > > Besides, the test tries to grab the first device listed by dpdk-devbind.py, > > regardless of the PCI device status regarding kmod binding. > > > > Remove dependency on this DPDK script and use a minimal script that > > reads PCI sysfs. > > > > This script is not perfect, as one can imagine PCI devices bound to > > vfio-pci for virtual machines. > > Plus, this script only tries to take over vfio-pci devices. mlx5 devices > > can't be taken over blindly as it could mean losing connectivity to the > > machine if the netdev was in use for this system. > > > > For those two reasons, add a new environment variable DPDK_PCI_ADDR for > > testers to select the PCI device of their liking. > > For consistency and grep, the temporary file PCI_ADDR is renamed > > to DPDK_PCI_ADDR. > > > > Reviewed-by: Maxime Coquelin > > Acked-by: Eelco Chaudron > > Signed-off-by: David Marchand > > This patch can't be merged as is. > I am preparing some fixes for the system-dpdk MTU tests that got > merged since my v4. Coming back on this. I sent a fix https://patchwork.ozlabs.org/project/openvswitch/patch/20231201142931.1782046-1-david.march...@redhat.com/. This current patch on making it possible to select a PCI device to test is kind of orthogonal to this fix. Yet, people will likely want to test with both applied. -- David Marchand ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH v5] system-dpdk: Test with mlx5 devices.
On Wed, Nov 22, 2023 at 5:34 PM David Marchand wrote: > > The DPDK unit test only runs if vfio or igb_uio kernel modules are loaded: > on systems with only mlx5, this test is always skipped. > > Besides, the test tries to grab the first device listed by dpdk-devbind.py, > regardless of the PCI device status regarding kmod binding. > > Remove dependency on this DPDK script and use a minimal script that > reads PCI sysfs. > > This script is not perfect, as one can imagine PCI devices bound to > vfio-pci for virtual machines. > Plus, this script only tries to take over vfio-pci devices. mlx5 devices > can't be taken over blindly as it could mean losing connectivity to the > machine if the netdev was in use for this system. > > For those two reasons, add a new environment variable DPDK_PCI_ADDR for > testers to select the PCI device of their liking. > For consistency and grep, the temporary file PCI_ADDR is renamed > to DPDK_PCI_ADDR. > > Reviewed-by: Maxime Coquelin > Acked-by: Eelco Chaudron > Signed-off-by: David Marchand This patch can't be merged as is. I am preparing some fixes for the system-dpdk MTU tests that got merged since my v4. -- David Marchand ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
[ovs-dev] [PATCH v5] system-dpdk: Test with mlx5 devices.
The DPDK unit test only runs if vfio or igb_uio kernel modules are loaded:
on systems with only mlx5, this test is always skipped.
Besides, the test tries to grab the first device listed by dpdk-devbind.py,
regardless of the PCI device status regarding kmod binding.
Remove dependency on this DPDK script and use a minimal script that
reads PCI sysfs.
This script is not perfect, as one can imagine PCI devices bound to
vfio-pci for virtual machines.
Plus, this script only tries to take over vfio-pci devices. mlx5 devices
can't be taken over blindly as it could mean losing connectivity to the
machine if the netdev was in use for this system.
For those two reasons, add a new environment variable DPDK_PCI_ADDR for
testers to select the PCI device of their liking.
For consistency and grep, the temporary file PCI_ADDR is renamed
to DPDK_PCI_ADDR.
Reviewed-by: Maxime Coquelin
Acked-by: Eelco Chaudron
Signed-off-by: David Marchand
---
Changes since v4:
- separated from the original series,
- rebased,
- dropped mlx5 devices from the discovery script,
- documented DPDK_PCI_ADDR env variable,
Changes since v3:
- fixed nit from Maxime,
Changes since v2:
- sorted logs alphabetically,
---
Documentation/topics/testing.rst | 11 ++---
tests/automake.mk| 1 +
tests/system-dpdk-find-device.py | 39
tests/system-dpdk-macros.at | 10 ++--
tests/system-dpdk.at | 14 ++--
5 files changed, 57 insertions(+), 18 deletions(-)
create mode 100755 tests/system-dpdk-find-device.py
diff --git a/Documentation/topics/testing.rst b/Documentation/topics/testing.rst
index 5f6940b84d..fb9b3e77b1 100644
--- a/Documentation/topics/testing.rst
+++ b/Documentation/topics/testing.rst
@@ -343,15 +343,20 @@ To see a list of all the available tests, run::
These tests support a `DPDK supported NIC`_. The tests operate on a wider set
of
environments, for instance, when a virtual port is used.
-They do require proper DPDK variables (``DPDK_DIR`` and ``DPDK_BUILD``).
Moreover you need to have root privileges to load the required modules and to
bind
-the NIC to the DPDK-compatible driver.
+a PCI device to the DPDK-compatible driver.
.. _DPDK supported NIC: https://core.dpdk.org/supported/#nics
+The phy test will skip if no suitable PCI device is found.
+It is possible to select which PCI device is used for this test by setting the
+DPDK_PCI_ADDR environment variable, which is especially useful when testing
+with a mlx5 device::
+
+# DPDK_PCI_ADDR=:82:00.0 make check-dpdk
+
All tests are skipped if no hugepages are configured. User must look into the
DPDK
manual to figure out how to `Configure hugepages`_.
-The phy test will skip if no compatible physical device is available.
.. _Configure hugepages:
https://doc.dpdk.org/guides-22.11/linux_gsg/sys_reqs.html
diff --git a/tests/automake.mk b/tests/automake.mk
index f8a925012d..69117514cc 100644
--- a/tests/automake.mk
+++ b/tests/automake.mk
@@ -192,6 +192,7 @@ SYSTEM_OFFLOADS_TESTSUITE_AT = \
SYSTEM_DPDK_TESTSUITE_AT = \
tests/system-common-macros.at \
+ tests/system-dpdk-find-device.py \
tests/system-dpdk-macros.at \
tests/system-dpdk-testsuite.at \
tests/system-dpdk.at
diff --git a/tests/system-dpdk-find-device.py b/tests/system-dpdk-find-device.py
new file mode 100755
index 00..b0997380bc
--- /dev/null
+++ b/tests/system-dpdk-find-device.py
@@ -0,0 +1,39 @@
+#!/usr/bin/env python3
+# Copyright (c) 2021 Red Hat, Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at:
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+from pathlib import Path
+import os
+import sys
+
+# The tester might want to select a PCI device, if so, trust it.
+if 'DPDK_PCI_ADDR' in os.environ:
+print(os.environ['DPDK_PCI_ADDR'])
+sys.exit(0)
+
+for device in sorted(Path('/sys/bus/pci/devices').iterdir()):
+class_path = device / 'class'
+# Only consider Network class devices
+if class_path.read_text().strip() != '0x02':
+continue
+kmod_path = device / 'driver' / 'module'
+kmod_name = kmod_path.resolve().name
+# Only care about devices bound to vfio_pci or igb_uio.
+if kmod_name not in ['vfio_pci', 'igb_uio']:
+continue
+print(device.resolve().name)
+sys.exit(0)
+
+sys.exit(1)
diff --git a/tests/system-dpdk-macros.at b/tests/system-dpdk-macros.at
index dcdfa55741..3b5a3512d4 100644
--- a/tests/system-dpdk-macros.at
+++ b/tests/system-dpdk-macros.at
