Thank you for your quick response. We are very interested in solving this issue as this servers are part of our critical infrastructure - mongodb replica set, so I secured a permission to restart the servers.
Here are the attached udev.log files. One is the abnormal where we had the issue and the normal is from one of our other machines unaffected by the issue. I have zipped the files as they were quite large ~6MB ________________________________ From: [email protected] <[email protected]> on behalf of Nick Rosbrook <[email protected]> Sent: Tuesday, October 29, 2024 15:28 To: Stoyan Atanasov <[email protected]> Subject: [Bug 2085835] Re: "Predictable Network Interface Naming" suddenly changed the name from enp96s0f0 to enp96s0f0np0 causing outage By default (and as we can see in the ID_NET_LINK_FILE= property above), the name policy is controlled by NamePolicy= in /usr/lib/systemd/network/99-default.link. On Jammy, this looks like: root@jammy:~# cat /lib/systemd/network/99-default.link # SPDX-License-Identifier: LGPL-2.1-or-later # # This file is part of systemd. # # systemd is free software; you can redistribute it and/or modify it # under the terms of the GNU Lesser General Public License as published by # the Free Software Foundation; either version 2.1 of the License, or # (at your option) any later version. [Match] OriginalName=* [Link] NamePolicy=keep kernel database onboard slot path AlternativeNamesPolicy=database onboard slot path MACAddressPolicy=persistent Basically, udev will try those policies in order until one matches. Based on the udev properties above, it looks like the 'path' policy was selected. This *might* different than before, or maybe something else changed on the system that then changed resulting name constructed by udev. Can you please enable debug logs for udev (ideally on both a working and non-working system), and share them here? E.g. $ mkdir -p /etc/systemd/system/systemd-udevd.service.d $ cat > /etc/systemd/system/systemd-udevd.service.d/debug.conf << EOF [Service] Environment=SYSTEMD_LOG_LEVEL=debug EOF Then, reboot, and then run: $ journalctl -u systemd-udevd.service -b > udev.log and attach the result. -- You received this bug notification because you are subscribed to the bug report. https://bugs.launchpad.net/bugs/2085835 Title: "Predictable Network Interface Naming" suddenly changed the name from enp96s0f0 to enp96s0f0np0 causing outage Status in systemd package in Ubuntu: New Bug description: On three identical servers - sleds in a blade chassis. the NIC name of one of the servers suddenly changed, causing an outage. The netplan config could not assign IP to the interface due to the name change. enp96s0f0 to enp96s0f0np0 The two other servers still have the enp96s0f0 name. No BIOS or other configuration change is detectable between the three systems. Firmware, software - all the same I tried to debug this issue myself, but it turned out to be a very niche and complicated topic. I went to submit an issue on https://github.com/systemd/systemd/issues, but they only accept issues for newer versions. System info: dundts@mongodb-rs2:~$ lsb_release -rd Description: Ubuntu 22.04.3 LTS Release: 22.04 dundts@mongodb-rs2:~$ apt-cache policy systemd system: Installiert: 249.11-0ubuntu3.11 Installationskandidat: 249.11-0ubuntu3.12 Versionstabelle: 249.11-0ubuntu3.12 500 500 http://de.archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages *** 249.11-0ubuntu3.11 100 100 /var/lib/dpkg/status 249.11-0ubuntu3.7 500 500 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages 249.11-0ubuntu3 500 500 http://de.archive.ubuntu.com/ubuntu jammy/main amd64 Packages dundts@mongodb-rs2:~$ sudo dmesg | grep i40e [sudo] password for dundts: [ 3.657531] i40e: Intel(R) Ethernet Connection XL710 Network Driver [ 3.658208] i40e: Copyright (c) 2013 - 2019 Intel Corporation. [ 3.715866] i40e 0000:60:00.0: fw 3.1.54559 api 1.5 nvm 3.2d 0x80000b4b 1.1767.0 [8086:37d3] [152d:8a40] [ 3.720222] i40e 0000:60:00.0: MAC address: d8:c4:97:4c:66:ae [ 3.720558] i40e 0000:60:00.0: FW LLDP is enabled [ 3.732613] i40e 0000:60:00.0: Added LAN device PF0 bus=0x60 dev=0x00 func=0x00 [ 3.733075] i40e 0000:60:00.0: Features: PF-id[0] VFs: 32 VSIs: 66 QP: 32 RSS FD_ATR FD_SB NTUPLE DCB VxLAN Geneve PTP VEPA [ 3.788671] i40e 0000:60:00.1: fw 3.1.54559 api 1.5 nvm 3.2d 0x80000b4b 1.1767.0 [8086:37d3] [152d:8a40] [ 3.804478] i40e 0000:60:00.1: MAC address: d8:c4:97:4c:66:af [ 3.804964] i40e 0000:60:00.1: FW LLDP is enabled [ 3.839982] i40e 0000:60:00.1 eth1: NIC Link is Up, 10 Gbps Full Duplex, Flow Control: None [ 3.851939] i40e 0000:60:00.1: Added LAN device PF1 bus=0x60 dev=0x00 func=0x01 [ 3.877994] i40e 0000:60:00.1: Features: PF-id[1] VFs: 32 VSIs: 66 QP: 32 RSS FD_ATR FD_SB NTUPLE DCB VxLAN Geneve PTP VEPA [ 4.065330] i40e 0000:60:00.0 enp96s0f0np0: renamed from eth0 [ 4.096320] i40e 0000:60:00.1 enp96s0f1np1: renamed from eth1 dundts@mongodb-rs2:~$ sudo udevadm info /sys/class/net/enp96s0f0np0 P: /devices/pci0000:5d/0000:5d:00.0/0000:5e:00.0/0000:5f:03.0/0000:60:00.0/net/enp96s0f0np0 L: 0 E: DEVPATH=/devices/pci0000:5d/0000:5d:00.0/0000:5e:00.0/0000:5f:03.0/0000:60:00.0/net/enp96s0f0np0 E: INTERFACE=enp96s0f0np0 E: IFINDEX=2 E: SUBSYSTEM=net E: USEC_INITIALIZED=4079509 E: ID_MM_CANDIDATE=1 E: ID_NET_NAMING_SCHEME=v249 E: ID_NET_NAME_MAC=enxd8c4974c66ae E: ID_OUI_FROM_DATABASE=Quanta Computer Inc. E: ID_NET_NAME_PATH=enp96s0f0np0 E: ID_BUS=pci E: ID_VENDOR_ID=0x8086 E: ID_MODEL_ID=0x37d3 E: ID_PCI_CLASS_FROM_DATABASE=Network controller E: ID_PCI_SUBCLASS_FROM_DATABASE=Ethernet controller E: ID_VENDOR_FROM_DATABASE=Intel Corporation E: ID_MODEL_FROM_DATABASE=Ethernet Connection X722 for 10GbE SFP+ E: ID_PATH=pci-0000:60:00.0 E: ID_PATH_TAG=pci-0000_60_00_0 E: ID_NET_DRIVER=i40e E: ID_NET_LINK_FILE=/usr/lib/systemd/network/99-default.link E: ID_NET_NAME=enp96s0f0np0 E: NM_UNMANAGED=1 E: SYSTEMD_ALIAS=/sys/subsystem/net/devices/enp96s0f0np0 E: TAGS=:systemd: E: CURRENT_TAGS=:system: To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/2085835/+subscriptions ** Attachment added: "udev.logs.zip" https://bugs.launchpad.net/bugs/2085835/+attachment/5832635/+files/udev.logs.zip -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2085835 Title: "Predictable Network Interface Naming" suddenly changed the name from enp96s0f0 to enp96s0f0np0 causing outage To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/2085835/+subscriptions -- ubuntu-bugs mailing list [email protected] https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
