[Kernel-packages] [Bug 1811963] Re: Sporadic problems with X710 (i40e) and bonding where one interface is shown as "state DOWN" and without LOWER_UP
This was fixed for me by patching the latest firmware on the GBICs. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1811963 Title: Sporadic problems with X710 (i40e) and bonding where one interface is shown as "state DOWN" and without LOWER_UP Status in linux package in Ubuntu: Confirmed Bug description: After rebooting the physical server there is a 50/50 chance of all connected interfaces coming up. This affects Dell EMC R740's and R440's equipped with the X710 network cards. As far as I noticed (~20 reboots on different machines), this happens only when using bonding (in this case active-backup or mode 1, did not test different modes yet). The networking-hardware on the other side shows the ports "connected". tcpdump shows frames being received, even if the interface is in "state DOWN". Tried with: Ubuntu 16.04, kernel 4.4.0-141, driver 2.7.26 (from the Intel-website), firmware 18.8.9 Ubuntu 16.04, kernel 4.4.0-141, driver 1.4.25-k, firmware 18.8.9 Ubuntu 16.04, kernel 4.15.0-43 (hwe), driver 2.1.14-k, firmware 18.8.9 The following excerpts are made using Intels driver in version 2.7.26, therefore tainting the kernel, but the same happens using the original kernel's version or the hardware enablement kernel's version. Sporadic failure case: [6.319226] i40e: loading out-of-tree module taints kernel. [6.319227] i40e: loading out-of-tree module taints kernel. [6.319422] i40e: module verification failed: signature and/or required key missing - tainting kernel [6.410837] i40e: Intel(R) 40-10 Gigabit Ethernet Connection Network Driver - version 2.7.26 [6.410838] i40e: Copyright(c) 2013 - 2018 Intel Corporation. [6.423542] i40e :3b:00.0: fw 6.81.49447 api 1.7 nvm 6.80 0x80003d72 18.8.9 [6.658526] i40e :3b:00.0: MAC address: ff:ff:ff:ff:ff:ff [6.710391] i40e :3b:00.0: PCI-Express: Speed 8.0GT/s Width x8 [6.725692] i40e :3b:00.0: Features: PF-id[0] VFs: 64 VSIs: 2 QP: 40 RSS FD_ATR FD_SB NTUPLE CloudF DCB VxLAN Geneve NVGRE PTP VEPA [6.750239] i40e :3b:00.1: fw 6.81.49447 api 1.7 nvm 6.80 0x80003d72 18.8.9 [6.987874] i40e :3b:00.1: MAC address: ff:ff:ff:ff:ff:f1 [7.005397] i40e :3b:00.1 eth0: NIC Link is Up, 10 Gbps Full Duplex, Flow Control: None [7.024993] i40e :3b:00.1: PCI-Express: Speed 8.0GT/s Width x8 [7.040298] i40e :3b:00.1: Features: PF-id[1] VFs: 64 VSIs: 2 QP: 40 RSS FD_ATR FD_SB NTUPLE CloudF DCB VxLAN Geneve NVGRE PTP VEPA [7.054384] i40e :3b:00.1 enp59s0f1: renamed from eth0 [7.079613] i40e :3b:00.0 enp59s0f0: renamed from eth1 [9.788893] i40e :3b:00.0 enp59s0f0: already using mac address ff:ff:ff:ff:ff:ff [9.819480] i40e :3b:00.1 enp59s0f1: set new mac address ff:ff:ff:ff:ff:ff [9.728194] bond0: Setting MII monitoring interval to 100 [9.788690] bond0: Adding slave enp59s0f0 [9.805195] bond0: Enslaving enp59s0f0 as a backup interface with a down link [9.819470] bond0: Adding slave enp59s0f1 [9.836360] bond0: making interface enp59s0f1 the new active one [9.836614] bond0: Enslaving enp59s0f1 as an active interface with an up link Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011) Bonding Mode: fault-tolerance (active-backup) Primary Slave: None Currently Active Slave: enp59s0f1 MII Status: up MII Polling Interval (ms): 100 Up Delay (ms): 0 Down Delay (ms): 0 Slave Interface: enp59s0f0 MII Status: down Speed: Unknown Duplex: Unknown Link Failure Count: 0 Permanent HW addr: ff:ff:ff:ff:ff:ff Slave queue ID: 0 Slave Interface: enp59s0f1 MII Status: up Speed: Unknown Duplex: Unknown Link Failure Count: 0 Permanent HW addr: ff:ff:ff:ff:ff:f1 Slave queue ID: 0 4: enp59s0f0: mtu 1500 qdisc mq master bond0 portid state DOWN group default qlen 1000 link/ether ff:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff 5: enp59s0f1: mtu 1500 qdisc mq master bond0 portid fff1 state UP group default qlen 1000 link/ether ff:ff:ff:ff:ff:f1 brd ff:ff:ff:ff:ff:ff 6: bond0: mtu 1500 qdisc noqueue state UP group default qlen 1000 link/ether ff:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff inet 123.123.123.123/24 brd 123.123.123.255 scope global bond0 valid_lft forever preferred_lft forever inet6 :::::/64 scope link valid_lft forever preferred_lft forever bond0 Link encap:Ethernet HWaddr ff:ff:ff:ff:ff:ff inet addr:123.123.123.123 Bcast:123.123.123.255 Mask:255.255.255.0 inet6 addr: :::::/64 Scope:Link UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1 RX packets:4392 errors:0 dropped:10 overruns:0 frame:0 TX packets:3585 errors:0 dropped:0
[Kernel-packages] [Bug 1811963] Re: Sporadic problems with X710 (i40e) and bonding where one interface is shown as "state DOWN" and without LOWER_UP
Hi Malte, Was this issue resolved for you? There are several other possibilities that it could be - and if it's still a problem with current mainline, please let us know. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1811963 Title: Sporadic problems with X710 (i40e) and bonding where one interface is shown as "state DOWN" and without LOWER_UP Status in linux package in Ubuntu: Confirmed Bug description: After rebooting the physical server there is a 50/50 chance of all connected interfaces coming up. This affects Dell EMC R740's and R440's equipped with the X710 network cards. As far as I noticed (~20 reboots on different machines), this happens only when using bonding (in this case active-backup or mode 1, did not test different modes yet). The networking-hardware on the other side shows the ports "connected". tcpdump shows frames being received, even if the interface is in "state DOWN". Tried with: Ubuntu 16.04, kernel 4.4.0-141, driver 2.7.26 (from the Intel-website), firmware 18.8.9 Ubuntu 16.04, kernel 4.4.0-141, driver 1.4.25-k, firmware 18.8.9 Ubuntu 16.04, kernel 4.15.0-43 (hwe), driver 2.1.14-k, firmware 18.8.9 The following excerpts are made using Intels driver in version 2.7.26, therefore tainting the kernel, but the same happens using the original kernel's version or the hardware enablement kernel's version. Sporadic failure case: [6.319226] i40e: loading out-of-tree module taints kernel. [6.319227] i40e: loading out-of-tree module taints kernel. [6.319422] i40e: module verification failed: signature and/or required key missing - tainting kernel [6.410837] i40e: Intel(R) 40-10 Gigabit Ethernet Connection Network Driver - version 2.7.26 [6.410838] i40e: Copyright(c) 2013 - 2018 Intel Corporation. [6.423542] i40e :3b:00.0: fw 6.81.49447 api 1.7 nvm 6.80 0x80003d72 18.8.9 [6.658526] i40e :3b:00.0: MAC address: ff:ff:ff:ff:ff:ff [6.710391] i40e :3b:00.0: PCI-Express: Speed 8.0GT/s Width x8 [6.725692] i40e :3b:00.0: Features: PF-id[0] VFs: 64 VSIs: 2 QP: 40 RSS FD_ATR FD_SB NTUPLE CloudF DCB VxLAN Geneve NVGRE PTP VEPA [6.750239] i40e :3b:00.1: fw 6.81.49447 api 1.7 nvm 6.80 0x80003d72 18.8.9 [6.987874] i40e :3b:00.1: MAC address: ff:ff:ff:ff:ff:f1 [7.005397] i40e :3b:00.1 eth0: NIC Link is Up, 10 Gbps Full Duplex, Flow Control: None [7.024993] i40e :3b:00.1: PCI-Express: Speed 8.0GT/s Width x8 [7.040298] i40e :3b:00.1: Features: PF-id[1] VFs: 64 VSIs: 2 QP: 40 RSS FD_ATR FD_SB NTUPLE CloudF DCB VxLAN Geneve NVGRE PTP VEPA [7.054384] i40e :3b:00.1 enp59s0f1: renamed from eth0 [7.079613] i40e :3b:00.0 enp59s0f0: renamed from eth1 [9.788893] i40e :3b:00.0 enp59s0f0: already using mac address ff:ff:ff:ff:ff:ff [9.819480] i40e :3b:00.1 enp59s0f1: set new mac address ff:ff:ff:ff:ff:ff [9.728194] bond0: Setting MII monitoring interval to 100 [9.788690] bond0: Adding slave enp59s0f0 [9.805195] bond0: Enslaving enp59s0f0 as a backup interface with a down link [9.819470] bond0: Adding slave enp59s0f1 [9.836360] bond0: making interface enp59s0f1 the new active one [9.836614] bond0: Enslaving enp59s0f1 as an active interface with an up link Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011) Bonding Mode: fault-tolerance (active-backup) Primary Slave: None Currently Active Slave: enp59s0f1 MII Status: up MII Polling Interval (ms): 100 Up Delay (ms): 0 Down Delay (ms): 0 Slave Interface: enp59s0f0 MII Status: down Speed: Unknown Duplex: Unknown Link Failure Count: 0 Permanent HW addr: ff:ff:ff:ff:ff:ff Slave queue ID: 0 Slave Interface: enp59s0f1 MII Status: up Speed: Unknown Duplex: Unknown Link Failure Count: 0 Permanent HW addr: ff:ff:ff:ff:ff:f1 Slave queue ID: 0 4: enp59s0f0: mtu 1500 qdisc mq master bond0 portid state DOWN group default qlen 1000 link/ether ff:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff 5: enp59s0f1: mtu 1500 qdisc mq master bond0 portid fff1 state UP group default qlen 1000 link/ether ff:ff:ff:ff:ff:f1 brd ff:ff:ff:ff:ff:ff 6: bond0: mtu 1500 qdisc noqueue state UP group default qlen 1000 link/ether ff:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff inet 123.123.123.123/24 brd 123.123.123.255 scope global bond0 valid_lft forever preferred_lft forever inet6 :::::/64 scope link valid_lft forever preferred_lft forever bond0 Link encap:Ethernet HWaddr ff:ff:ff:ff:ff:ff inet addr:123.123.123.123 Bcast:123.123.123.255 Mask:255.255.255.0 inet6 addr: :::::/64 Scope:Link UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1
[Kernel-packages] [Bug 1811963] Re: Sporadic problems with X710 (i40e) and bonding where one interface is shown as "state DOWN" and without LOWER_UP
Please file a bug to Intel ethernet developers. Commit 3647cd6eaf83d7f6145a3ccac73f5286496490d2 and 3f8af41262697a4d6742f030fbe0ceb9e1a048a6 in linux-next may worth trying. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1811963 Title: Sporadic problems with X710 (i40e) and bonding where one interface is shown as "state DOWN" and without LOWER_UP Status in linux package in Ubuntu: Confirmed Bug description: After rebooting the physical server there is a 50/50 chance of all connected interfaces coming up. This affects Dell EMC R740's and R440's equipped with the X710 network cards. As far as I noticed (~20 reboots on different machines), this happens only when using bonding (in this case active-backup or mode 1, did not test different modes yet). The networking-hardware on the other side shows the ports "connected". tcpdump shows frames being received, even if the interface is in "state DOWN". Tried with: Ubuntu 16.04, kernel 4.4.0-141, driver 2.7.26 (from the Intel-website), firmware 18.8.9 Ubuntu 16.04, kernel 4.4.0-141, driver 1.4.25-k, firmware 18.8.9 Ubuntu 16.04, kernel 4.15.0-43 (hwe), driver 2.1.14-k, firmware 18.8.9 The following excerpts are made using Intels driver in version 2.7.26, therefore tainting the kernel, but the same happens using the original kernel's version or the hardware enablement kernel's version. Sporadic failure case: [6.319226] i40e: loading out-of-tree module taints kernel. [6.319227] i40e: loading out-of-tree module taints kernel. [6.319422] i40e: module verification failed: signature and/or required key missing - tainting kernel [6.410837] i40e: Intel(R) 40-10 Gigabit Ethernet Connection Network Driver - version 2.7.26 [6.410838] i40e: Copyright(c) 2013 - 2018 Intel Corporation. [6.423542] i40e :3b:00.0: fw 6.81.49447 api 1.7 nvm 6.80 0x80003d72 18.8.9 [6.658526] i40e :3b:00.0: MAC address: ff:ff:ff:ff:ff:ff [6.710391] i40e :3b:00.0: PCI-Express: Speed 8.0GT/s Width x8 [6.725692] i40e :3b:00.0: Features: PF-id[0] VFs: 64 VSIs: 2 QP: 40 RSS FD_ATR FD_SB NTUPLE CloudF DCB VxLAN Geneve NVGRE PTP VEPA [6.750239] i40e :3b:00.1: fw 6.81.49447 api 1.7 nvm 6.80 0x80003d72 18.8.9 [6.987874] i40e :3b:00.1: MAC address: ff:ff:ff:ff:ff:f1 [7.005397] i40e :3b:00.1 eth0: NIC Link is Up, 10 Gbps Full Duplex, Flow Control: None [7.024993] i40e :3b:00.1: PCI-Express: Speed 8.0GT/s Width x8 [7.040298] i40e :3b:00.1: Features: PF-id[1] VFs: 64 VSIs: 2 QP: 40 RSS FD_ATR FD_SB NTUPLE CloudF DCB VxLAN Geneve NVGRE PTP VEPA [7.054384] i40e :3b:00.1 enp59s0f1: renamed from eth0 [7.079613] i40e :3b:00.0 enp59s0f0: renamed from eth1 [9.788893] i40e :3b:00.0 enp59s0f0: already using mac address ff:ff:ff:ff:ff:ff [9.819480] i40e :3b:00.1 enp59s0f1: set new mac address ff:ff:ff:ff:ff:ff [9.728194] bond0: Setting MII monitoring interval to 100 [9.788690] bond0: Adding slave enp59s0f0 [9.805195] bond0: Enslaving enp59s0f0 as a backup interface with a down link [9.819470] bond0: Adding slave enp59s0f1 [9.836360] bond0: making interface enp59s0f1 the new active one [9.836614] bond0: Enslaving enp59s0f1 as an active interface with an up link Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011) Bonding Mode: fault-tolerance (active-backup) Primary Slave: None Currently Active Slave: enp59s0f1 MII Status: up MII Polling Interval (ms): 100 Up Delay (ms): 0 Down Delay (ms): 0 Slave Interface: enp59s0f0 MII Status: down Speed: Unknown Duplex: Unknown Link Failure Count: 0 Permanent HW addr: ff:ff:ff:ff:ff:ff Slave queue ID: 0 Slave Interface: enp59s0f1 MII Status: up Speed: Unknown Duplex: Unknown Link Failure Count: 0 Permanent HW addr: ff:ff:ff:ff:ff:f1 Slave queue ID: 0 4: enp59s0f0: mtu 1500 qdisc mq master bond0 portid state DOWN group default qlen 1000 link/ether ff:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff 5: enp59s0f1: mtu 1500 qdisc mq master bond0 portid fff1 state UP group default qlen 1000 link/ether ff:ff:ff:ff:ff:f1 brd ff:ff:ff:ff:ff:ff 6: bond0: mtu 1500 qdisc noqueue state UP group default qlen 1000 link/ether ff:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff inet 123.123.123.123/24 brd 123.123.123.255 scope global bond0 valid_lft forever preferred_lft forever inet6 :::::/64 scope link valid_lft forever preferred_lft forever bond0 Link encap:Ethernet HWaddr ff:ff:ff:ff:ff:ff inet addr:123.123.123.123 Bcast:123.123.123.255 Mask:255.255.255.0 inet6 addr: :::::/64 Scope:Link UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1
[Kernel-packages] [Bug 1811963] Re: Sporadic problems with X710 (i40e) and bonding where one interface is shown as "state DOWN" and without LOWER_UP
I tested and can reproduce the problem with this combination: Ubuntu 16.04, kernel 5.0.0-05.201903032031, driver 2.7.6-k, firmware 18.8.9 I set the tag accordingly. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1811963 Title: Sporadic problems with X710 (i40e) and bonding where one interface is shown as "state DOWN" and without LOWER_UP Status in linux package in Ubuntu: Confirmed Bug description: After rebooting the physical server there is a 50/50 chance of all connected interfaces coming up. This affects Dell EMC R740's and R440's equipped with the X710 network cards. As far as I noticed (~20 reboots on different machines), this happens only when using bonding (in this case active-backup or mode 1, did not test different modes yet). The networking-hardware on the other side shows the ports "connected". tcpdump shows frames being received, even if the interface is in "state DOWN". Tried with: Ubuntu 16.04, kernel 4.4.0-141, driver 2.7.26 (from the Intel-website), firmware 18.8.9 Ubuntu 16.04, kernel 4.4.0-141, driver 1.4.25-k, firmware 18.8.9 Ubuntu 16.04, kernel 4.15.0-43 (hwe), driver 2.1.14-k, firmware 18.8.9 The following excerpts are made using Intels driver in version 2.7.26, therefore tainting the kernel, but the same happens using the original kernel's version or the hardware enablement kernel's version. Sporadic failure case: [6.319226] i40e: loading out-of-tree module taints kernel. [6.319227] i40e: loading out-of-tree module taints kernel. [6.319422] i40e: module verification failed: signature and/or required key missing - tainting kernel [6.410837] i40e: Intel(R) 40-10 Gigabit Ethernet Connection Network Driver - version 2.7.26 [6.410838] i40e: Copyright(c) 2013 - 2018 Intel Corporation. [6.423542] i40e :3b:00.0: fw 6.81.49447 api 1.7 nvm 6.80 0x80003d72 18.8.9 [6.658526] i40e :3b:00.0: MAC address: ff:ff:ff:ff:ff:ff [6.710391] i40e :3b:00.0: PCI-Express: Speed 8.0GT/s Width x8 [6.725692] i40e :3b:00.0: Features: PF-id[0] VFs: 64 VSIs: 2 QP: 40 RSS FD_ATR FD_SB NTUPLE CloudF DCB VxLAN Geneve NVGRE PTP VEPA [6.750239] i40e :3b:00.1: fw 6.81.49447 api 1.7 nvm 6.80 0x80003d72 18.8.9 [6.987874] i40e :3b:00.1: MAC address: ff:ff:ff:ff:ff:f1 [7.005397] i40e :3b:00.1 eth0: NIC Link is Up, 10 Gbps Full Duplex, Flow Control: None [7.024993] i40e :3b:00.1: PCI-Express: Speed 8.0GT/s Width x8 [7.040298] i40e :3b:00.1: Features: PF-id[1] VFs: 64 VSIs: 2 QP: 40 RSS FD_ATR FD_SB NTUPLE CloudF DCB VxLAN Geneve NVGRE PTP VEPA [7.054384] i40e :3b:00.1 enp59s0f1: renamed from eth0 [7.079613] i40e :3b:00.0 enp59s0f0: renamed from eth1 [9.788893] i40e :3b:00.0 enp59s0f0: already using mac address ff:ff:ff:ff:ff:ff [9.819480] i40e :3b:00.1 enp59s0f1: set new mac address ff:ff:ff:ff:ff:ff [9.728194] bond0: Setting MII monitoring interval to 100 [9.788690] bond0: Adding slave enp59s0f0 [9.805195] bond0: Enslaving enp59s0f0 as a backup interface with a down link [9.819470] bond0: Adding slave enp59s0f1 [9.836360] bond0: making interface enp59s0f1 the new active one [9.836614] bond0: Enslaving enp59s0f1 as an active interface with an up link Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011) Bonding Mode: fault-tolerance (active-backup) Primary Slave: None Currently Active Slave: enp59s0f1 MII Status: up MII Polling Interval (ms): 100 Up Delay (ms): 0 Down Delay (ms): 0 Slave Interface: enp59s0f0 MII Status: down Speed: Unknown Duplex: Unknown Link Failure Count: 0 Permanent HW addr: ff:ff:ff:ff:ff:ff Slave queue ID: 0 Slave Interface: enp59s0f1 MII Status: up Speed: Unknown Duplex: Unknown Link Failure Count: 0 Permanent HW addr: ff:ff:ff:ff:ff:f1 Slave queue ID: 0 4: enp59s0f0: mtu 1500 qdisc mq master bond0 portid state DOWN group default qlen 1000 link/ether ff:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff 5: enp59s0f1: mtu 1500 qdisc mq master bond0 portid fff1 state UP group default qlen 1000 link/ether ff:ff:ff:ff:ff:f1 brd ff:ff:ff:ff:ff:ff 6: bond0: mtu 1500 qdisc noqueue state UP group default qlen 1000 link/ether ff:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff inet 123.123.123.123/24 brd 123.123.123.255 scope global bond0 valid_lft forever preferred_lft forever inet6 :::::/64 scope link valid_lft forever preferred_lft forever bond0 Link encap:Ethernet HWaddr ff:ff:ff:ff:ff:ff inet addr:123.123.123.123 Bcast:123.123.123.255 Mask:255.255.255.0 inet6 addr: :::::/64 Scope:Link UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1
[Kernel-packages] [Bug 1811963] Re: Sporadic problems with X710 (i40e) and bonding where one interface is shown as "state DOWN" and without LOWER_UP
** Tags added: kernel-bug-exists-upstream -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1811963 Title: Sporadic problems with X710 (i40e) and bonding where one interface is shown as "state DOWN" and without LOWER_UP Status in linux package in Ubuntu: Confirmed Bug description: After rebooting the physical server there is a 50/50 chance of all connected interfaces coming up. This affects Dell EMC R740's and R440's equipped with the X710 network cards. As far as I noticed (~20 reboots on different machines), this happens only when using bonding (in this case active-backup or mode 1, did not test different modes yet). The networking-hardware on the other side shows the ports "connected". tcpdump shows frames being received, even if the interface is in "state DOWN". Tried with: Ubuntu 16.04, kernel 4.4.0-141, driver 2.7.26 (from the Intel-website), firmware 18.8.9 Ubuntu 16.04, kernel 4.4.0-141, driver 1.4.25-k, firmware 18.8.9 Ubuntu 16.04, kernel 4.15.0-43 (hwe), driver 2.1.14-k, firmware 18.8.9 The following excerpts are made using Intels driver in version 2.7.26, therefore tainting the kernel, but the same happens using the original kernel's version or the hardware enablement kernel's version. Sporadic failure case: [6.319226] i40e: loading out-of-tree module taints kernel. [6.319227] i40e: loading out-of-tree module taints kernel. [6.319422] i40e: module verification failed: signature and/or required key missing - tainting kernel [6.410837] i40e: Intel(R) 40-10 Gigabit Ethernet Connection Network Driver - version 2.7.26 [6.410838] i40e: Copyright(c) 2013 - 2018 Intel Corporation. [6.423542] i40e :3b:00.0: fw 6.81.49447 api 1.7 nvm 6.80 0x80003d72 18.8.9 [6.658526] i40e :3b:00.0: MAC address: ff:ff:ff:ff:ff:ff [6.710391] i40e :3b:00.0: PCI-Express: Speed 8.0GT/s Width x8 [6.725692] i40e :3b:00.0: Features: PF-id[0] VFs: 64 VSIs: 2 QP: 40 RSS FD_ATR FD_SB NTUPLE CloudF DCB VxLAN Geneve NVGRE PTP VEPA [6.750239] i40e :3b:00.1: fw 6.81.49447 api 1.7 nvm 6.80 0x80003d72 18.8.9 [6.987874] i40e :3b:00.1: MAC address: ff:ff:ff:ff:ff:f1 [7.005397] i40e :3b:00.1 eth0: NIC Link is Up, 10 Gbps Full Duplex, Flow Control: None [7.024993] i40e :3b:00.1: PCI-Express: Speed 8.0GT/s Width x8 [7.040298] i40e :3b:00.1: Features: PF-id[1] VFs: 64 VSIs: 2 QP: 40 RSS FD_ATR FD_SB NTUPLE CloudF DCB VxLAN Geneve NVGRE PTP VEPA [7.054384] i40e :3b:00.1 enp59s0f1: renamed from eth0 [7.079613] i40e :3b:00.0 enp59s0f0: renamed from eth1 [9.788893] i40e :3b:00.0 enp59s0f0: already using mac address ff:ff:ff:ff:ff:ff [9.819480] i40e :3b:00.1 enp59s0f1: set new mac address ff:ff:ff:ff:ff:ff [9.728194] bond0: Setting MII monitoring interval to 100 [9.788690] bond0: Adding slave enp59s0f0 [9.805195] bond0: Enslaving enp59s0f0 as a backup interface with a down link [9.819470] bond0: Adding slave enp59s0f1 [9.836360] bond0: making interface enp59s0f1 the new active one [9.836614] bond0: Enslaving enp59s0f1 as an active interface with an up link Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011) Bonding Mode: fault-tolerance (active-backup) Primary Slave: None Currently Active Slave: enp59s0f1 MII Status: up MII Polling Interval (ms): 100 Up Delay (ms): 0 Down Delay (ms): 0 Slave Interface: enp59s0f0 MII Status: down Speed: Unknown Duplex: Unknown Link Failure Count: 0 Permanent HW addr: ff:ff:ff:ff:ff:ff Slave queue ID: 0 Slave Interface: enp59s0f1 MII Status: up Speed: Unknown Duplex: Unknown Link Failure Count: 0 Permanent HW addr: ff:ff:ff:ff:ff:f1 Slave queue ID: 0 4: enp59s0f0: mtu 1500 qdisc mq master bond0 portid state DOWN group default qlen 1000 link/ether ff:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff 5: enp59s0f1: mtu 1500 qdisc mq master bond0 portid fff1 state UP group default qlen 1000 link/ether ff:ff:ff:ff:ff:f1 brd ff:ff:ff:ff:ff:ff 6: bond0: mtu 1500 qdisc noqueue state UP group default qlen 1000 link/ether ff:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff inet 123.123.123.123/24 brd 123.123.123.255 scope global bond0 valid_lft forever preferred_lft forever inet6 :::::/64 scope link valid_lft forever preferred_lft forever bond0 Link encap:Ethernet HWaddr ff:ff:ff:ff:ff:ff inet addr:123.123.123.123 Bcast:123.123.123.255 Mask:255.255.255.0 inet6 addr: :::::/64 Scope:Link UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1 RX packets:4392 errors:0 dropped:10 overruns:0 frame:0 TX packets:3585 errors:0 dropped:0 overruns:0 carrier:0
[Kernel-packages] [Bug 1811963] Re: Sporadic problems with X710 (i40e) and bonding where one interface is shown as "state DOWN" and without LOWER_UP
Would it be possible for you to test the latest upstream kernel? Refer to https://wiki.ubuntu.com/KernelMainlineBuilds . Please test the latest v5.0 kernel [0]. If this bug is fixed in the mainline kernel, please add the following tag 'kernel-fixed-upstream'. If the mainline kernel does not fix this bug, please add the tag: 'kernel-bug-exists-upstream'. Once testing of the upstream kernel is complete, please mark this bug as "Confirmed". Thanks in advance. [0] https://kernel.ubuntu.com/~kernel-ppa/mainline/v5.0/ -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1811963 Title: Sporadic problems with X710 (i40e) and bonding where one interface is shown as "state DOWN" and without LOWER_UP Status in linux package in Ubuntu: Confirmed Bug description: After rebooting the physical server there is a 50/50 chance of all connected interfaces coming up. This affects Dell EMC R740's and R440's equipped with the X710 network cards. As far as I noticed (~20 reboots on different machines), this happens only when using bonding (in this case active-backup or mode 1, did not test different modes yet). The networking-hardware on the other side shows the ports "connected". tcpdump shows frames being received, even if the interface is in "state DOWN". Tried with: Ubuntu 16.04, kernel 4.4.0-141, driver 2.7.26 (from the Intel-website), firmware 18.8.9 Ubuntu 16.04, kernel 4.4.0-141, driver 1.4.25-k, firmware 18.8.9 Ubuntu 16.04, kernel 4.15.0-43 (hwe), driver 2.1.14-k, firmware 18.8.9 The following excerpts are made using Intels driver in version 2.7.26, therefore tainting the kernel, but the same happens using the original kernel's version or the hardware enablement kernel's version. Sporadic failure case: [6.319226] i40e: loading out-of-tree module taints kernel. [6.319227] i40e: loading out-of-tree module taints kernel. [6.319422] i40e: module verification failed: signature and/or required key missing - tainting kernel [6.410837] i40e: Intel(R) 40-10 Gigabit Ethernet Connection Network Driver - version 2.7.26 [6.410838] i40e: Copyright(c) 2013 - 2018 Intel Corporation. [6.423542] i40e :3b:00.0: fw 6.81.49447 api 1.7 nvm 6.80 0x80003d72 18.8.9 [6.658526] i40e :3b:00.0: MAC address: ff:ff:ff:ff:ff:ff [6.710391] i40e :3b:00.0: PCI-Express: Speed 8.0GT/s Width x8 [6.725692] i40e :3b:00.0: Features: PF-id[0] VFs: 64 VSIs: 2 QP: 40 RSS FD_ATR FD_SB NTUPLE CloudF DCB VxLAN Geneve NVGRE PTP VEPA [6.750239] i40e :3b:00.1: fw 6.81.49447 api 1.7 nvm 6.80 0x80003d72 18.8.9 [6.987874] i40e :3b:00.1: MAC address: ff:ff:ff:ff:ff:f1 [7.005397] i40e :3b:00.1 eth0: NIC Link is Up, 10 Gbps Full Duplex, Flow Control: None [7.024993] i40e :3b:00.1: PCI-Express: Speed 8.0GT/s Width x8 [7.040298] i40e :3b:00.1: Features: PF-id[1] VFs: 64 VSIs: 2 QP: 40 RSS FD_ATR FD_SB NTUPLE CloudF DCB VxLAN Geneve NVGRE PTP VEPA [7.054384] i40e :3b:00.1 enp59s0f1: renamed from eth0 [7.079613] i40e :3b:00.0 enp59s0f0: renamed from eth1 [9.788893] i40e :3b:00.0 enp59s0f0: already using mac address ff:ff:ff:ff:ff:ff [9.819480] i40e :3b:00.1 enp59s0f1: set new mac address ff:ff:ff:ff:ff:ff [9.728194] bond0: Setting MII monitoring interval to 100 [9.788690] bond0: Adding slave enp59s0f0 [9.805195] bond0: Enslaving enp59s0f0 as a backup interface with a down link [9.819470] bond0: Adding slave enp59s0f1 [9.836360] bond0: making interface enp59s0f1 the new active one [9.836614] bond0: Enslaving enp59s0f1 as an active interface with an up link Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011) Bonding Mode: fault-tolerance (active-backup) Primary Slave: None Currently Active Slave: enp59s0f1 MII Status: up MII Polling Interval (ms): 100 Up Delay (ms): 0 Down Delay (ms): 0 Slave Interface: enp59s0f0 MII Status: down Speed: Unknown Duplex: Unknown Link Failure Count: 0 Permanent HW addr: ff:ff:ff:ff:ff:ff Slave queue ID: 0 Slave Interface: enp59s0f1 MII Status: up Speed: Unknown Duplex: Unknown Link Failure Count: 0 Permanent HW addr: ff:ff:ff:ff:ff:f1 Slave queue ID: 0 4: enp59s0f0: mtu 1500 qdisc mq master bond0 portid state DOWN group default qlen 1000 link/ether ff:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff 5: enp59s0f1: mtu 1500 qdisc mq master bond0 portid fff1 state UP group default qlen 1000 link/ether ff:ff:ff:ff:ff:f1 brd ff:ff:ff:ff:ff:ff 6: bond0: mtu 1500 qdisc noqueue state UP group default qlen 1000 link/ether ff:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff inet 123.123.123.123/24 brd 123.123.123.255 scope global bond0 valid_lft forever preferred_lft forever inet6
[Kernel-packages] [Bug 1811963] Re: Sporadic problems with X710 (i40e) and bonding where one interface is shown as "state DOWN" and without LOWER_UP
Can anyone follow up on this? I am ready to provide specific logs and try alternative methods/workarounds on demand. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1811963 Title: Sporadic problems with X710 (i40e) and bonding where one interface is shown as "state DOWN" and without LOWER_UP Status in linux package in Ubuntu: Confirmed Bug description: After rebooting the physical server there is a 50/50 chance of all connected interfaces coming up. This affects Dell EMC R740's and R440's equipped with the X710 network cards. As far as I noticed (~20 reboots on different machines), this happens only when using bonding (in this case active-backup or mode 1, did not test different modes yet). The networking-hardware on the other side shows the ports "connected". tcpdump shows frames being received, even if the interface is in "state DOWN". Tried with: Ubuntu 16.04, kernel 4.4.0-141, driver 2.7.26 (from the Intel-website), firmware 18.8.9 Ubuntu 16.04, kernel 4.4.0-141, driver 1.4.25-k, firmware 18.8.9 Ubuntu 16.04, kernel 4.15.0-43 (hwe), driver 2.1.14-k, firmware 18.8.9 The following excerpts are made using Intels driver in version 2.7.26, therefore tainting the kernel, but the same happens using the original kernel's version or the hardware enablement kernel's version. Sporadic failure case: [6.319226] i40e: loading out-of-tree module taints kernel. [6.319227] i40e: loading out-of-tree module taints kernel. [6.319422] i40e: module verification failed: signature and/or required key missing - tainting kernel [6.410837] i40e: Intel(R) 40-10 Gigabit Ethernet Connection Network Driver - version 2.7.26 [6.410838] i40e: Copyright(c) 2013 - 2018 Intel Corporation. [6.423542] i40e :3b:00.0: fw 6.81.49447 api 1.7 nvm 6.80 0x80003d72 18.8.9 [6.658526] i40e :3b:00.0: MAC address: ff:ff:ff:ff:ff:ff [6.710391] i40e :3b:00.0: PCI-Express: Speed 8.0GT/s Width x8 [6.725692] i40e :3b:00.0: Features: PF-id[0] VFs: 64 VSIs: 2 QP: 40 RSS FD_ATR FD_SB NTUPLE CloudF DCB VxLAN Geneve NVGRE PTP VEPA [6.750239] i40e :3b:00.1: fw 6.81.49447 api 1.7 nvm 6.80 0x80003d72 18.8.9 [6.987874] i40e :3b:00.1: MAC address: ff:ff:ff:ff:ff:f1 [7.005397] i40e :3b:00.1 eth0: NIC Link is Up, 10 Gbps Full Duplex, Flow Control: None [7.024993] i40e :3b:00.1: PCI-Express: Speed 8.0GT/s Width x8 [7.040298] i40e :3b:00.1: Features: PF-id[1] VFs: 64 VSIs: 2 QP: 40 RSS FD_ATR FD_SB NTUPLE CloudF DCB VxLAN Geneve NVGRE PTP VEPA [7.054384] i40e :3b:00.1 enp59s0f1: renamed from eth0 [7.079613] i40e :3b:00.0 enp59s0f0: renamed from eth1 [9.788893] i40e :3b:00.0 enp59s0f0: already using mac address ff:ff:ff:ff:ff:ff [9.819480] i40e :3b:00.1 enp59s0f1: set new mac address ff:ff:ff:ff:ff:ff [9.728194] bond0: Setting MII monitoring interval to 100 [9.788690] bond0: Adding slave enp59s0f0 [9.805195] bond0: Enslaving enp59s0f0 as a backup interface with a down link [9.819470] bond0: Adding slave enp59s0f1 [9.836360] bond0: making interface enp59s0f1 the new active one [9.836614] bond0: Enslaving enp59s0f1 as an active interface with an up link Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011) Bonding Mode: fault-tolerance (active-backup) Primary Slave: None Currently Active Slave: enp59s0f1 MII Status: up MII Polling Interval (ms): 100 Up Delay (ms): 0 Down Delay (ms): 0 Slave Interface: enp59s0f0 MII Status: down Speed: Unknown Duplex: Unknown Link Failure Count: 0 Permanent HW addr: ff:ff:ff:ff:ff:ff Slave queue ID: 0 Slave Interface: enp59s0f1 MII Status: up Speed: Unknown Duplex: Unknown Link Failure Count: 0 Permanent HW addr: ff:ff:ff:ff:ff:f1 Slave queue ID: 0 4: enp59s0f0: mtu 1500 qdisc mq master bond0 portid state DOWN group default qlen 1000 link/ether ff:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff 5: enp59s0f1: mtu 1500 qdisc mq master bond0 portid fff1 state UP group default qlen 1000 link/ether ff:ff:ff:ff:ff:f1 brd ff:ff:ff:ff:ff:ff 6: bond0: mtu 1500 qdisc noqueue state UP group default qlen 1000 link/ether ff:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff inet 123.123.123.123/24 brd 123.123.123.255 scope global bond0 valid_lft forever preferred_lft forever inet6 :::::/64 scope link valid_lft forever preferred_lft forever bond0 Link encap:Ethernet HWaddr ff:ff:ff:ff:ff:ff inet addr:123.123.123.123 Bcast:123.123.123.255 Mask:255.255.255.0 inet6 addr: :::::/64 Scope:Link UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1 RX packets:4392 errors:0 dropped:10 overruns:0
[Kernel-packages] [Bug 1811963] Re: Sporadic problems with X710 (i40e) and bonding where one interface is shown as "state DOWN" and without LOWER_UP
Unable to serve the logs via apport-collect due to policy restrictions. ** Changed in: linux (Ubuntu) Status: Incomplete => Confirmed -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1811963 Title: Sporadic problems with X710 (i40e) and bonding where one interface is shown as "state DOWN" and without LOWER_UP Status in linux package in Ubuntu: Confirmed Bug description: After rebooting the physical server there is a 50/50 chance of all connected interfaces coming up. This affects Dell EMC R740's and R440's equipped with the X710 network cards. As far as I noticed (~20 reboots on different machines), this happens only when using bonding (in this case active-backup or mode 1, did not test different modes yet). The networking-hardware on the other side shows the ports "connected". tcpdump shows frames being received, even if the interface is in "state DOWN". Tried with: Ubuntu 16.04, kernel 4.4.0-141, driver 2.7.26 (from the Intel-website), firmware 18.8.9 Ubuntu 16.04, kernel 4.4.0-141, driver 1.4.25-k, firmware 18.8.9 Ubuntu 16.04, kernel 4.15.0-43 (hwe), driver 2.1.14-k, firmware 18.8.9 The following excerpts are made using Intels driver in version 2.7.26, therefore tainting the kernel, but the same happens using the original kernel's version or the hardware enablement kernel's version. Sporadic failure case: [6.319226] i40e: loading out-of-tree module taints kernel. [6.319227] i40e: loading out-of-tree module taints kernel. [6.319422] i40e: module verification failed: signature and/or required key missing - tainting kernel [6.410837] i40e: Intel(R) 40-10 Gigabit Ethernet Connection Network Driver - version 2.7.26 [6.410838] i40e: Copyright(c) 2013 - 2018 Intel Corporation. [6.423542] i40e :3b:00.0: fw 6.81.49447 api 1.7 nvm 6.80 0x80003d72 18.8.9 [6.658526] i40e :3b:00.0: MAC address: ff:ff:ff:ff:ff:ff [6.710391] i40e :3b:00.0: PCI-Express: Speed 8.0GT/s Width x8 [6.725692] i40e :3b:00.0: Features: PF-id[0] VFs: 64 VSIs: 2 QP: 40 RSS FD_ATR FD_SB NTUPLE CloudF DCB VxLAN Geneve NVGRE PTP VEPA [6.750239] i40e :3b:00.1: fw 6.81.49447 api 1.7 nvm 6.80 0x80003d72 18.8.9 [6.987874] i40e :3b:00.1: MAC address: ff:ff:ff:ff:ff:f1 [7.005397] i40e :3b:00.1 eth0: NIC Link is Up, 10 Gbps Full Duplex, Flow Control: None [7.024993] i40e :3b:00.1: PCI-Express: Speed 8.0GT/s Width x8 [7.040298] i40e :3b:00.1: Features: PF-id[1] VFs: 64 VSIs: 2 QP: 40 RSS FD_ATR FD_SB NTUPLE CloudF DCB VxLAN Geneve NVGRE PTP VEPA [7.054384] i40e :3b:00.1 enp59s0f1: renamed from eth0 [7.079613] i40e :3b:00.0 enp59s0f0: renamed from eth1 [9.788893] i40e :3b:00.0 enp59s0f0: already using mac address ff:ff:ff:ff:ff:ff [9.819480] i40e :3b:00.1 enp59s0f1: set new mac address ff:ff:ff:ff:ff:ff [9.728194] bond0: Setting MII monitoring interval to 100 [9.788690] bond0: Adding slave enp59s0f0 [9.805195] bond0: Enslaving enp59s0f0 as a backup interface with a down link [9.819470] bond0: Adding slave enp59s0f1 [9.836360] bond0: making interface enp59s0f1 the new active one [9.836614] bond0: Enslaving enp59s0f1 as an active interface with an up link Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011) Bonding Mode: fault-tolerance (active-backup) Primary Slave: None Currently Active Slave: enp59s0f1 MII Status: up MII Polling Interval (ms): 100 Up Delay (ms): 0 Down Delay (ms): 0 Slave Interface: enp59s0f0 MII Status: down Speed: Unknown Duplex: Unknown Link Failure Count: 0 Permanent HW addr: ff:ff:ff:ff:ff:ff Slave queue ID: 0 Slave Interface: enp59s0f1 MII Status: up Speed: Unknown Duplex: Unknown Link Failure Count: 0 Permanent HW addr: ff:ff:ff:ff:ff:f1 Slave queue ID: 0 4: enp59s0f0: mtu 1500 qdisc mq master bond0 portid state DOWN group default qlen 1000 link/ether ff:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff 5: enp59s0f1: mtu 1500 qdisc mq master bond0 portid fff1 state UP group default qlen 1000 link/ether ff:ff:ff:ff:ff:f1 brd ff:ff:ff:ff:ff:ff 6: bond0: mtu 1500 qdisc noqueue state UP group default qlen 1000 link/ether ff:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff inet 123.123.123.123/24 brd 123.123.123.255 scope global bond0 valid_lft forever preferred_lft forever inet6 :::::/64 scope link valid_lft forever preferred_lft forever bond0 Link encap:Ethernet HWaddr ff:ff:ff:ff:ff:ff inet addr:123.123.123.123 Bcast:123.123.123.255 Mask:255.255.255.0 inet6 addr: :::::/64 Scope:Link UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1 RX packets:4392 errors:0
[Kernel-packages] [Bug 1811963] Re: Sporadic problems with X710 (i40e) and bonding where one interface is shown as "state DOWN" and without LOWER_UP
** Attachment added: "version_signature" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1811963/+attachment/5229569/+files/version.log -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1811963 Title: Sporadic problems with X710 (i40e) and bonding where one interface is shown as "state DOWN" and without LOWER_UP Status in linux package in Ubuntu: New Bug description: After rebooting the physical server there is a 50/50 chance of all connected interfaces coming up. This affects Dell EMC R740's and R440's equipped with the X710 network cards. As far as I noticed (~20 reboots on different machines), this happens only when using bonding (in this case active-backup or mode 1, did not test different modes yet). The networking-hardware on the other side shows the ports "connected". tcpdump shows frames being received, even if the interface is in "state DOWN". Tried with: Ubuntu 16.04, kernel 4.4.0-141, driver 2.7.26 (from the Intel-website), firmware 18.8.9 Ubuntu 16.04, kernel 4.4.0-141, driver 1.4.25-k, firmware 18.8.9 Ubuntu 16.04, kernel 4.15.0-43 (hwe), driver 2.1.14-k, firmware 18.8.9 The following excerpts are made using Intels driver in version 2.7.26, therefore tainting the kernel, but the same happens using the original kernel's version or the hardware enablement kernel's version. Sporadic failure case: [6.319226] i40e: loading out-of-tree module taints kernel. [6.319227] i40e: loading out-of-tree module taints kernel. [6.319422] i40e: module verification failed: signature and/or required key missing - tainting kernel [6.410837] i40e: Intel(R) 40-10 Gigabit Ethernet Connection Network Driver - version 2.7.26 [6.410838] i40e: Copyright(c) 2013 - 2018 Intel Corporation. [6.423542] i40e :3b:00.0: fw 6.81.49447 api 1.7 nvm 6.80 0x80003d72 18.8.9 [6.658526] i40e :3b:00.0: MAC address: ff:ff:ff:ff:ff:ff [6.710391] i40e :3b:00.0: PCI-Express: Speed 8.0GT/s Width x8 [6.725692] i40e :3b:00.0: Features: PF-id[0] VFs: 64 VSIs: 2 QP: 40 RSS FD_ATR FD_SB NTUPLE CloudF DCB VxLAN Geneve NVGRE PTP VEPA [6.750239] i40e :3b:00.1: fw 6.81.49447 api 1.7 nvm 6.80 0x80003d72 18.8.9 [6.987874] i40e :3b:00.1: MAC address: ff:ff:ff:ff:ff:f1 [7.005397] i40e :3b:00.1 eth0: NIC Link is Up, 10 Gbps Full Duplex, Flow Control: None [7.024993] i40e :3b:00.1: PCI-Express: Speed 8.0GT/s Width x8 [7.040298] i40e :3b:00.1: Features: PF-id[1] VFs: 64 VSIs: 2 QP: 40 RSS FD_ATR FD_SB NTUPLE CloudF DCB VxLAN Geneve NVGRE PTP VEPA [7.054384] i40e :3b:00.1 enp59s0f1: renamed from eth0 [7.079613] i40e :3b:00.0 enp59s0f0: renamed from eth1 [9.788893] i40e :3b:00.0 enp59s0f0: already using mac address ff:ff:ff:ff:ff:ff [9.819480] i40e :3b:00.1 enp59s0f1: set new mac address ff:ff:ff:ff:ff:ff [9.728194] bond0: Setting MII monitoring interval to 100 [9.788690] bond0: Adding slave enp59s0f0 [9.805195] bond0: Enslaving enp59s0f0 as a backup interface with a down link [9.819470] bond0: Adding slave enp59s0f1 [9.836360] bond0: making interface enp59s0f1 the new active one [9.836614] bond0: Enslaving enp59s0f1 as an active interface with an up link Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011) Bonding Mode: fault-tolerance (active-backup) Primary Slave: None Currently Active Slave: enp59s0f1 MII Status: up MII Polling Interval (ms): 100 Up Delay (ms): 0 Down Delay (ms): 0 Slave Interface: enp59s0f0 MII Status: down Speed: Unknown Duplex: Unknown Link Failure Count: 0 Permanent HW addr: ff:ff:ff:ff:ff:ff Slave queue ID: 0 Slave Interface: enp59s0f1 MII Status: up Speed: Unknown Duplex: Unknown Link Failure Count: 0 Permanent HW addr: ff:ff:ff:ff:ff:f1 Slave queue ID: 0 4: enp59s0f0: mtu 1500 qdisc mq master bond0 portid state DOWN group default qlen 1000 link/ether ff:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff 5: enp59s0f1: mtu 1500 qdisc mq master bond0 portid fff1 state UP group default qlen 1000 link/ether ff:ff:ff:ff:ff:f1 brd ff:ff:ff:ff:ff:ff 6: bond0: mtu 1500 qdisc noqueue state UP group default qlen 1000 link/ether ff:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff inet 123.123.123.123/24 brd 123.123.123.255 scope global bond0 valid_lft forever preferred_lft forever inet6 :::::/64 scope link valid_lft forever preferred_lft forever bond0 Link encap:Ethernet HWaddr ff:ff:ff:ff:ff:ff inet addr:123.123.123.123 Bcast:123.123.123.255 Mask:255.255.255.0 inet6 addr: :::::/64 Scope:Link UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1 RX packets:4392 errors:0
[Kernel-packages] [Bug 1811963] Re: Sporadic problems with X710 (i40e) and bonding where one interface is shown as "state DOWN" and without LOWER_UP
** Attachment added: "lspci -v" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1811963/+attachment/5229570/+files/lspci.log -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1811963 Title: Sporadic problems with X710 (i40e) and bonding where one interface is shown as "state DOWN" and without LOWER_UP Status in linux package in Ubuntu: New Bug description: After rebooting the physical server there is a 50/50 chance of all connected interfaces coming up. This affects Dell EMC R740's and R440's equipped with the X710 network cards. As far as I noticed (~20 reboots on different machines), this happens only when using bonding (in this case active-backup or mode 1, did not test different modes yet). The networking-hardware on the other side shows the ports "connected". tcpdump shows frames being received, even if the interface is in "state DOWN". Tried with: Ubuntu 16.04, kernel 4.4.0-141, driver 2.7.26 (from the Intel-website), firmware 18.8.9 Ubuntu 16.04, kernel 4.4.0-141, driver 1.4.25-k, firmware 18.8.9 Ubuntu 16.04, kernel 4.15.0-43 (hwe), driver 2.1.14-k, firmware 18.8.9 The following excerpts are made using Intels driver in version 2.7.26, therefore tainting the kernel, but the same happens using the original kernel's version or the hardware enablement kernel's version. Sporadic failure case: [6.319226] i40e: loading out-of-tree module taints kernel. [6.319227] i40e: loading out-of-tree module taints kernel. [6.319422] i40e: module verification failed: signature and/or required key missing - tainting kernel [6.410837] i40e: Intel(R) 40-10 Gigabit Ethernet Connection Network Driver - version 2.7.26 [6.410838] i40e: Copyright(c) 2013 - 2018 Intel Corporation. [6.423542] i40e :3b:00.0: fw 6.81.49447 api 1.7 nvm 6.80 0x80003d72 18.8.9 [6.658526] i40e :3b:00.0: MAC address: ff:ff:ff:ff:ff:ff [6.710391] i40e :3b:00.0: PCI-Express: Speed 8.0GT/s Width x8 [6.725692] i40e :3b:00.0: Features: PF-id[0] VFs: 64 VSIs: 2 QP: 40 RSS FD_ATR FD_SB NTUPLE CloudF DCB VxLAN Geneve NVGRE PTP VEPA [6.750239] i40e :3b:00.1: fw 6.81.49447 api 1.7 nvm 6.80 0x80003d72 18.8.9 [6.987874] i40e :3b:00.1: MAC address: ff:ff:ff:ff:ff:f1 [7.005397] i40e :3b:00.1 eth0: NIC Link is Up, 10 Gbps Full Duplex, Flow Control: None [7.024993] i40e :3b:00.1: PCI-Express: Speed 8.0GT/s Width x8 [7.040298] i40e :3b:00.1: Features: PF-id[1] VFs: 64 VSIs: 2 QP: 40 RSS FD_ATR FD_SB NTUPLE CloudF DCB VxLAN Geneve NVGRE PTP VEPA [7.054384] i40e :3b:00.1 enp59s0f1: renamed from eth0 [7.079613] i40e :3b:00.0 enp59s0f0: renamed from eth1 [9.788893] i40e :3b:00.0 enp59s0f0: already using mac address ff:ff:ff:ff:ff:ff [9.819480] i40e :3b:00.1 enp59s0f1: set new mac address ff:ff:ff:ff:ff:ff [9.728194] bond0: Setting MII monitoring interval to 100 [9.788690] bond0: Adding slave enp59s0f0 [9.805195] bond0: Enslaving enp59s0f0 as a backup interface with a down link [9.819470] bond0: Adding slave enp59s0f1 [9.836360] bond0: making interface enp59s0f1 the new active one [9.836614] bond0: Enslaving enp59s0f1 as an active interface with an up link Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011) Bonding Mode: fault-tolerance (active-backup) Primary Slave: None Currently Active Slave: enp59s0f1 MII Status: up MII Polling Interval (ms): 100 Up Delay (ms): 0 Down Delay (ms): 0 Slave Interface: enp59s0f0 MII Status: down Speed: Unknown Duplex: Unknown Link Failure Count: 0 Permanent HW addr: ff:ff:ff:ff:ff:ff Slave queue ID: 0 Slave Interface: enp59s0f1 MII Status: up Speed: Unknown Duplex: Unknown Link Failure Count: 0 Permanent HW addr: ff:ff:ff:ff:ff:f1 Slave queue ID: 0 4: enp59s0f0: mtu 1500 qdisc mq master bond0 portid state DOWN group default qlen 1000 link/ether ff:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff 5: enp59s0f1: mtu 1500 qdisc mq master bond0 portid fff1 state UP group default qlen 1000 link/ether ff:ff:ff:ff:ff:f1 brd ff:ff:ff:ff:ff:ff 6: bond0: mtu 1500 qdisc noqueue state UP group default qlen 1000 link/ether ff:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff inet 123.123.123.123/24 brd 123.123.123.255 scope global bond0 valid_lft forever preferred_lft forever inet6 :::::/64 scope link valid_lft forever preferred_lft forever bond0 Link encap:Ethernet HWaddr ff:ff:ff:ff:ff:ff inet addr:123.123.123.123 Bcast:123.123.123.255 Mask:255.255.255.0 inet6 addr: :::::/64 Scope:Link UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1 RX packets:4392 errors:0 dropped:10
