[Bug 1888726] Re: systemd-udevd regression: some renamed network interfaces stuck in "pending" state
Finally got to check this. On latest Hirsute with netplan 0.101-0ubuntu3, my initial sample configuration seems to work as expected. >From my understanding, this hasn't been backported to Focal yet. However specifying the "driver" for the "lan" interface helps, as a workaround. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1888726 Title: systemd-udevd regression: some renamed network interfaces stuck in "pending" state To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/netplan.io/+bug/1888726/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1888726] [NEW] systemd-udevd regression: some renamed network interfaces stuck in "pending" state
Public bug reported: Summary: In Ubuntu Server 20.04 LTS and newer, using netplan.io and systemd- networkd, in certain network configurations, renamed network interfaces get stuck in "pending" state and are not configured properly. On boot, system is stuck on "Wait for Network to be Configured" for 2 minutes. How to reproduce: 1. Configure a machine (for example, a virtual machine) with two Ethernet network cards. Make note of MAC addresses of these network cards. 2. Set up netplan with a single configuration file, contents are in the attached "00-static.yaml" file. Replace the MAC addresses to match your setup. IP address configuration is omitted and is not necessary to reproduce the bug. 3. Reboot the system. Expected outcome: 1. System boots in a reasonable time 2. First network interface (wan) is brought up, is not renamed, and is marked as configured by networkd 3. Second network interface (lan) is brought up, renamed, configured for MTU=9000, and is marked as configured by networkd 4. VLAN interface (vlan20) is brought up, renamed, configured for MTU=9000, and is marked as configured by networkd Actual outcome: 1. System boot is delayed by 2 minutes 2. First network interface (wan) is configured as expected 3. Second network interface (lan) is configured as expected 4. VLAN interface (vlan20) seems to be configured as expected, but is stuck in "pending" state according to networkctl list Test environment: Hardware: Virtual machine with the following configuration * 4 amd64 CPU cores (also tested with a single core) * 1 GB RAM * 8 GB disk * 2 network cards (vmxnet3 in VMware, virtio in Parallels) Working as expected: * [1] Ubuntu Server 19.10, kernel 5.3.0-62, netplan.io 0.99-0ubuntu3~19.10.2, systemd+udev 242-7ubuntu3.11 Broken: * [2] Ubuntu Server 19.10, kernel 5.3.0-62, netplan.io 0.99-0ubuntu3~19.10.2, systemd 242-7ubuntu3.11, udev 245.4-4ubuntu3.1 * [3] Ubuntu Server 20.04 LTS, kernel 5.4.0-42, netplan.io 0.99-0ubuntu3~20.04.2, systemd+udev 245.4-4ubuntu3.1 * [4] Ubuntu Server 20.04 LTS, kernel 5.4.0-42, netplan.io 0.99-0ubuntu3~20.04.2, systemd+udev vanilla upstream git e9769453 * [5] Ubuntu Server 20.04 LTS, kernel 5.4.0-42, netplan.io 0.99-0ubuntu3~20.04.2, systemd+udev vanilla upstream git v242 * [6] Ubuntu Server 20.10 daily-live, kernel 5.4.0-26, netplan.io 0.99-0ubuntu5, systemd+udev 245.6-3ubuntu3 [2] As noted above, issue was reproduced in 19.10 by upgrading ONLY udev and libudev1 to ones shipped in focal-updates. [4] It was also reproduced in vanilla upstream systemd, git master commit e9769453. Just installed on top of existing systemd using "sudo ninja -C build install". [5] Interestingly enough, issue also seems to exist in vanilla v242. Either that, or the installation didn't replace the packaged systemd properly. This may hint at some distribution-specific patch that got removed before 20.04. This issue was reproduced in VMware ESXi 6.7U3, VMware Fusion 11.5.5 and Parallels Desktop 15.1.4. This leads me to believe that network card drivers or virtualisation engines do not play part in the issue. Extra observations: To make the example configuration (00-static.yaml) not get stuck in "pending" state, any one of the following options helps: * Remove "set-name" parameter for "lan" interface * Remove "mtu" parameter for "lan" interface * Remove "wan" interface entirely I got some data/logs for each of these scenarios for eoan [1] and focal [3], as well as the original broken config, and put them together in the attached "parallels.tar.gz". Note about Apport: Attached apport report was generated for test environment [2] above. Attachments: * 00-static.yaml: minimalistic broken netplan configuration example * parallels.tar.gz: various logs for eoan [1] and focal [3], as described in "Extra observations" above ProblemType: Bug DistroRelease: Ubuntu 19.10 Package: udev 245.4-4ubuntu3.2 ProcVersionSignature: Ubuntu 5.3.0-62.56-generic 5.3.18 Uname: Linux 5.3.0-62-generic x86_64 ApportVersion: 2.20.11-0ubuntu8.9 Architecture: amd64 CustomUdevRuleFiles: 70-snap.core.rules Date: Thu Jul 23 18:54:58 2020 InstallationDate: Installed on 2020-07-22 (1 days ago) InstallationMedia: Ubuntu-Server 19.10 "Eoan Ermine" - Release amd64 (20191017) Lsusb: Error: command ['lsusb'] failed with exit code 1: MachineType: Parallels Software International Inc. Parallels Virtual Platform ProcEnviron: TERM=linux PATH=(custom, no user) LANG=C.UTF-8 SHELL=/bin/bash ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-5.3.0-62-generic root=UUID=c42113c4-0f7a-44bc-a6ae-b27e2b146723 ro SourcePackage: systemd UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 04/13/2020 dmi.bios.vendor: Parallels Software International Inc. dmi.bios.version: 15.1.4 (47270) dmi.board.name: Parallels Virtual Platform dmi.board.vendor: Parallels Software International Inc. dmi.board.version: None dmi.chassis.type: 10 dmi.chassis.vendor:
[Bug 1888726] Re: systemd-udevd regression: some renamed network interfaces stuck in "pending" state
** Attachment added: "Various logs for eoan [1] and focal [3], as described in "Extra observations" above" https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1888726/+attachment/5395343/+files/parallels.tar.gz -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1888726 Title: systemd-udevd regression: some renamed network interfaces stuck in "pending" state To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1888726/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1581777] Re: igb transmit queue times out
The patch is now upstream in 4.4.20 and 4.7.3, both released on 7th of September. >From what I know, it's not in the Ubuntu kernels yet. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1581777 Title: igb transmit queue times out To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1581777/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1581777] Re: igb transmit queue times out
This seems to be a problem in ESXi, it was promised to be fixed in 6.0 Update 3 which hasn't been released as of yet. However there is now a workaround patch that will make it to 4.4 and 4.7 stable trees: http://www.kernel.org/git/?p=linux/kernel/git/stable/stable- queue.git;a=summary queue-4.4/genirq-msi-make-sure-pci-msis-are-activated-early.patch queue-4.7/genirq-msi-make-sure-pci-msis-are-activated-early.patch -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1581777 Title: igb transmit queue times out To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1581777/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1581777] Re: igb transmit queue times out
Reported to Thomas Gleixner and linux-kernel: https://lkml.org/lkml/2016/7/23/93 Jiang Liu is no longer reachable under the address specified in MAINTAINERS: "Recipient address rejected: User unknown in virtual mailbox table". -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1581777 Title: igb transmit queue times out To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1581777/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1581777] Re: igb transmit queue times out
Also, the bug is present in 4.7-rc7. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1581777 Title: igb transmit queue times out To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1581777/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1581777] Re: igb transmit queue times out
** Attachment added: "Bisect for bug" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1581777/+attachment/4703544/+files/Bisect.txt ** Changed in: linux (Ubuntu) Status: Incomplete => Confirmed ** Tags removed: kernel-bug-exists-upstream-4.6-rc7 needs-bisect ** Tags added: kernel-bug-exists-upstream-4.7-rc7 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1581777 Title: igb transmit queue times out To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1581777/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1581777] Re: igb transmit queue times out
52f518a3a7c2f80551a38d38be28bc9f335e713c is the first bad commit Full bisect log attached. A couple of notes: 1. Often the bug is not reproduced the first time the system is booted with a given kernel version. Subsequent boots reproduce the bug. 2. In order to get bootable kernels from commit 9dda1658a9bd450d65da5153a2427955785d17c2 and onwards during bisecting I had to manually apply commit 425be5679fd292a3c36cb1fe423086708a99f11a. Otherwise the system gets stuck on "Loading initial ramdisk". -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1581777 Title: igb transmit queue times out To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1581777/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1581777] Re: igb transmit queue times out
Bug reported to Intel Ethernet Driver mailing list: http://lists.osuosl.org/pipermail/intel-wired-lan/Week-of- Mon-20160516/005444.html ** Tags added: kernel-bug-reported-upstream -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1581777 Title: igb transmit queue times out To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1581777/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1581777] Re: igb transmit queue times out
Tested with latest mainline kernel (linux- image-4.6.0-040600rc7-generic), problem persists, appropriate tag added. ** Tags added: kernel-bug-exists-upstream -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1581777 Title: igb transmit queue times out To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1581777/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1581777] [NEW] igb transmit queue times out
Public bug reported: Using an Intel I210AT built-in PCI-E NIC passed through from the ESXi 6 host to an Ubuntu 16.04 server virtual machine. Network is unusable due to constant "transmit queue 0 timed out" errors. Issue is NOT present on a 14.04.4 machine running "3.19.0-59-generic #65~14.04.1-Ubuntu SMP Tue Apr 19 18:57:09 UTC 2016" kernel. ProblemType: Bug DistroRelease: Ubuntu 16.04 Package: linux-image-4.4.0-22-generic 4.4.0-22.39 ProcVersionSignature: Ubuntu 4.4.0-22.39-generic 4.4.8 Uname: Linux 4.4.0-22-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 May 14 12:09 seq crw-rw 1 root audio 116, 33 May 14 12:09 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.20.1-0ubuntu2 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: [Errno 2] No such file or directory: 'fuser' Date: Sat May 14 12:22:44 2016 HibernationDevice: RESUME=UUID=c31589d1-6ec7-471b-9073-180b6f2886f3 InstallationDate: Installed on 2016-05-13 (0 days ago) InstallationMedia: Ubuntu-Server 16.04 LTS "Xenial Xerus" - Release amd64 (20160420.3) IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 002 Device 003: ID 0e0f:0002 VMware, Inc. Virtual USB Hub Bus 002 Device 002: ID 0e0f:0003 VMware, Inc. Virtual Mouse Bus 002 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub MachineType: VMware, Inc. VMware7,1 PciMultimedia: ProcFB: 0 svgadrmfb ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.4.0-22-generic root=UUID=1c9de470-43c0-4955-80ab-85497792601e ro noplymouth crashkernel=384M-:128M RelatedPackageVersions: linux-restricted-modules-4.4.0-22-generic N/A linux-backports-modules-4.4.0-22-generic N/A linux-firmware1.157 RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 06/25/2015 dmi.bios.vendor: VMware, Inc. dmi.bios.version: VMW71.00V.0.B64.1506250318 dmi.board.name: 440BX Desktop Reference Platform dmi.board.vendor: Intel Corporation dmi.board.version: None dmi.chassis.asset.tag: No Asset Tag dmi.chassis.type: 1 dmi.chassis.vendor: No Enclosure dmi.chassis.version: N/A dmi.modalias: dmi:bvnVMware,Inc.:bvrVMW71.00V.0.B64.1506250318:bd06/25/2015:svnVMware,Inc.:pnVMware7,1:pvrNone:rvnIntelCorporation:rn440BXDesktopReferencePlatform:rvrNone:cvnNoEnclosure:ct1:cvrN/A: dmi.product.name: VMware7,1 dmi.product.version: None dmi.sys.vendor: VMware, Inc. ** Affects: linux (Ubuntu) Importance: Undecided Status: New ** Tags: amd64 apport-bug xenial -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1581777 Title: igb transmit queue times out To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1581777/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
