Goal of this patchset is to fix probing logic in mlx5 networking PMD
to allow support of BlueField DPUs with Socket Direct.
In such use case, BlueField DPU will be connected through PCI
to 2 different CPUs on the host.
Each host CPU sees 2 PFs.
Each PF is connected to one of the physical ports.
+--------+ +--------+
|CPU 0 | |CPU 1 |
| | | |
| pf0 | | pf0 |
| | | |
| pf1 | | pf1 |
| | | |
+---+----+ +-+------+
| |
| |
| |
+----+ +-----+
| |
| |
| |
+---+-----------+----+
|BF3 DPU |
| |
| pf0hpf pf1hpf |
| |
| pf2hpf pf3hpf |
| |
| p0 p1 |
+------+------+------+
| phy0 | | phy1 |
+------+ +------+
On BlueField DPU ARM Linux netdevs map to PFs/ports as follows:
- p0 and p1 to physical ports 0 and 1 respectively,
- pf0hpf and pf2hpf to CPU0 pf0 and CPU1 pf0 respectively,
- pf1hpf and pf3hpf to CPU0 pf1 and CPU1 pf1 respectively.
There are several possible ways to use such a setup:
- Single E-Switch (embedded switch) per each CPU PF to
physical port connection.
- Shared E-Switch for related CPU PFs:
- For example, both pf0hpf and pf2hpf are in the same E-Switch.
- Multiport E-Switch.
Existing probing logic in mlx5 PMD did not support case (2).
In this case there is one physical port (uplink in mlx5 naming)
and 2 host PFs.
On such a setup mlx5 generated port names with the following syntax:
03:00.0_representor_vfX
Port name syntax was selected based on specific setup type.
Since setup was not recognized as neither bond nor MPESW,
mlx5 selected the default name without PF index.
Since BlueField with Socket Direct would have 2 host PFs,
such probing logic caused DPDK port name collisions
on the attempt to probe 2 host PFs at the same time.
More over there were some false positives during probing on systems
with or without specific udev rules which change mlx5 IB device
name to include "bond" in the name.
This patchset addresses the above:
- Patch 1 and 2 - Fixes bond detection logic.
Previously mlx5 PMD relied on "bond" appearing in IB device name
which is not always the case. Moved to sysfs checks.
- Patch 3 - Fixed uplink and host PF probing logic.
Previously mlx5 PMD relied on specific setup type.
With this patch probing is more generic and based on
types and number of available ports on the E-Switch.
With this patchset, a DPDK application run on BlueField ARM is able to
probe all the relevant representors (corresponding to available netdevs).
Using testpmd syntax users will be able to do the following:
# Probe both physical ports
port attach 03:00.0,dv_flow_en=2,representor=pf0-1
# Probe both host PF 0 from CPU 0
# (VF representor index -1 is special encoding for host PF)
port attach 03:00.0,dv_flow_en=2,representor=pf0vf65535
# or with explicit controller index
port attach 03:00.0,dv_flow_en=2,representor=c1pf0vf65535
# Probe both host PF 0 from CPU 1
port attach 03:00.0,dv_flow_en=2,representor=pf2vf65535
# or with explicit controller index
port attach 03:00.0,dv_flow_en=2,representor=c2pf2vf65535
v1:
https://patches.dpdk.org/project/dpdk/cover/[email protected]/
v1->v2:
- Squash patches 3-5 and add Fixes tag,
since these patches fix existing probing logic.
Dariusz Sosnowski (3):
common/mlx5: fix bond check
net/mlx5: fix bond check
net/mlx5: fix probing to allow BlueField Socket Direct
drivers/common/mlx5/linux/mlx5_common_os.c | 86 ++++-
drivers/common/mlx5/linux/mlx5_common_os.h | 9 +
drivers/net/mlx5/linux/mlx5_os.c | 356 ++++++++++++++-------
drivers/net/mlx5/mlx5.h | 2 +
4 files changed, 338 insertions(+), 115 deletions(-)
--
2.47.3
