Hmm dmesg shows this:
[82646.225888] vfio-pci 0000:05:00.0: Masking broken INTx support
[82647.057220] vfio-pci 0000:05:00.1: Masking broken INTx support
In "vppctl sh hard" what caught my attention was that BondEthernet0
shows "Link speed: unknown" while ext0 and ext1 both detected 40Gbps.
"vppctl sh pci" shows only these "RV" lines for the XL710
0000:05:00.0 0 8086:1583 unknown vfio-pci XL710 40GbE Controller
RV: 0x 86
0000:05:00.1 0 8086:1583 unknown vfio-pci XL710 40GbE Controller
RV: 0x 86
The other interfaces have more lines of output, with "MN", "RV" and
"V[0-6]" prefixes, but I'm not sure what those mean.
On 8/27/25 4:50 AM, Benoit Ganne (bganne) via lists.fd.io wrote:
Hi Andre,
from what you are reporting this looks like a DPDK initialization issue indeed.
Intel XL710 should have very good support, they are in widespread use and we
have quite a few of them in CSIT, so it's probably some specific combination of
fw/hw/driver/config in your setup causing the issue.
Do you see anything suspicious in dmesg, vppctl sh log, vppctl sh hard, vppctl
sh pci?
Does the interface work flawlessly with DPDK testpmd?
Alternatively you could try the native AVF driver instead of DPDK and see if it
is better: https://git.fd.io/vpp/tree/src/plugins/avf/README.rst
best
ben
________________________________________
From: vpp-dev@lists.fd.io <vpp-dev@lists.fd.io> on behalf of Andre Nathan via
lists.fd.io <andre=digirati.com...@lists.fd.io>
Sent: Wednesday, August 27, 2025 5:48
To: vpp-dev@lists.fd.io
Subject: [vpp-dev] Initialization problem (Intel XL710 related?)
Hello
I have a router with three NICs in use by VPP with LCP, all Intel models
(E810, XL710 and X540).
I have a consistent initialization issue that seems to only happen with
the XL710 NIC. Sometimes I get one of the two errors below, either
lcp create: unknown input `ext1 host-if ext1'
or
dpdk: Interface ext0 error -95: Unknown error -95
interface: sw_set_flags_helper: dpdk_interface_admin_up_down: Interface
start failed
However most times there's no error and yet the interface has no
connectivity.
When pinging a machine in the same local network as the XL710 NIC's, its
MAC address shows up correctly with "vppctl show neighbors", but not on
the kernel arp table, where it appears as "incomplete". The ping target
machine has the router's MAC in its arp table.
Previously this router had two XL710 NICs and the issue happened on both
of them, and once I upgraded one of them to an E810, it stopped
happening with it, which makes me suspect it's some sort of driver issue (?)
The problem is "fixed" by restarting VPP a number of times until the
interface comes up properly, with no configuration changes.
Has anyone experienced anything like this? Can anyone recommend a 40Gbps
NIC that works reliably?
My configuration files follow:
# /etc/vpp/startup.conf
unix {
gid vpp
nodaemon
cli-listen /run/vpp/cli.sock
exec /etc/vpp/bootstrap.vpp
}
api-trace { on }
api-segment { gid vpp }
socksvr { default }
memory {
main-heap-size 2G
main-heap-page-size default-hugepage
}
cpu {
main-core 1
corelist-workers 3,5,7,9,11,13,15,17,19
}
buffers {
buffers-per-numa 128000
default data-size 16384
page-size default-hugepage
}
statseg {
size 1G
page-size default-hugepage
per-node-counters off
}
plugins {
plugin default { disable }
plugin dpdk_plugin.so { enable }
plugin linux_nl_plugin.so { enable }
plugin linux_cp_plugin.so { enable }
}
logging {
default-log-level warn
default-syslog-log-level warn
}
dpdk {
uio-driver vfio-pci
dev 0000:41:00.0 {
name uplink1
}
dev 0000:42:00.0 {
name uplink2
}
dev 0000:05:00.0 {
name ext0
}
dev 0000:05:00.1 {
name ext1
}
}
linux-cp {
default netns vpp
lcp-sync
lcp-auto-subint
del-static-on-link-down
del-dynamic-on-link-down
}
# /etc/vpp/bootstrap.vpp
lcp create uplink1 host-if uplink1
set interface state uplink1 up
set interface ip address uplink1 a.a.a.a.a/30
lcp create uplink2 host-if uplink2
set interface state uplink2 up
set interface ip address uplink2 b.b.b.b/30
create loopback interface instance 0
lcp create loop0 host-if management
set interface state loop0 up
set interface ip address loop0 c.c.c.c/32
lcp create ext0 host-if ext0
lcp create ext1 host-if ext1
create bond mode active-backup
bond add BondEthernet0 ext0
bond add BondEthernet0 ext1
set interface state ext0 up
set interface state ext1 up
lcp create BondEthernet0 host-if external
set interface state BondEthernet0 up
set interface ip address BondEthernet0 d.d.d.d/22
I have already tried to configure the external interface as just a
regular NIC (without bonding) but that didn't seem to help, so it
doesn't appear to be related to bonding. Also, the uplink1 interface
used to be an XL710 and had this problem even without any sort of
bonding configuration.
Thanks,
Andre
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#26298): https://lists.fd.io/g/vpp-dev/message/26298
Mute This Topic: https://lists.fd.io/mt/114912573/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/leave/14379924/21656/631435203/xyzzy
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
