Public bug reported:
The network device systemd service stay active after unload the module of this
network device, that call close port (ndo_stop).
once we try to load the NIC driver again, it try to start the ifup service of
his NICs and due to the service is already up, so it fail and we didn't see the
interface with the static configuration =.
below simple reproduce with the Mellanox ConnectX4 device (driver name
mlx5_core).
Also we see this issue with Azure system, Ubuntu 17.04 guest over
Hyper-v, the VF failed to start after re-enable SR-IOV from VM's vNIC.
For now we have a Work Around that to add a udev rule,
echo DRIVERS==\"*mlx*\", SUBSYSTEM==\"net\",
ACTION==\"add\",RUN+=\"/sbin/ifup --force $env{INTERFACE}\" >
/lib/udev/rules.d/100-up.rules
Example:
#:/lib/udev/rules.d# cat 100-up.rules
DRIVERS=="*mlx*", SUBSYSTEM=="net", ACTION=="add",RUN+="/sbin/ifup --force
$env{INTERFACE}"
***
* More info and reproduce *
***
# ifdown ens1f0
RTNETLINK answers: Cannot assign requested address
# ifup ens1f0
# ifconfig ens1f0
ens1f0: flags=4163 mtu 1500
inet 123.12.23.1 netmask 255.255.0.0 broadcast 123.12.255.255
inet6 fe80::268a:7ff:fea1:fbdc prefixlen 64 scopeid 0x20
ether 24:8a:07:a1:fb:dc txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 17 bytes 1392 (1.3 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
# ethtool -i ens1f0 |grep driv
driver: mlx5_core
# systemctl status ifup@ens1f
ifup@ens1f0.service ifup@ens1f1.service
# systemctl status ifup@ens1f0.service
* ifup@ens1f0.service - ifup for ens1f0
Loaded: loaded (/lib/systemd/system/ifup@.service; static; vendor preset:
enabled)
Active: active (exited) since Sun 2017-03-12 09:40:04 IST; 2h 26min ago
Main PID: 1608 (code=exited, status=0/SUCCESS)
CGroup: /system.slice/ifup@ens1f0.service
Mar 12 09:40:04 qa-h-vrt-039 systemd[1]: Started ifup for ens1f0.
Mar 12 09:40:04 qa-h-vrt-039 sh[1608]: ifup: interface ens1f0 already configured
root@qa-h-vrt-039:/tmp# modprobe -rv mlx5_ib
rmmod mlx5_ib
rmmod mlx5_core
# modprobe -rv mlx5_core
# ifconfig -a |grep ens1f0
# lsmod |grep mlx5
# systemctl status ifup@ens1f0.service
* ifup@ens1f0.service - ifup for ens1f0
Loaded: loaded (/lib/systemd/system/ifup@.service; static; vendor preset:
enabled)
Active: active (exited) since Sun 2017-03-12 09:40:04 IST; 2h 27min ago
Main PID: 1608 (code=exited, status=0/SUCCESS)
CGroup: /system.slice/ifup@ens1f0.service
Mar 12 09:40:04 qa-h-vrt-039 systemd[1]: Started ifup for ens1f0.
Mar 12 09:40:04 qa-h-vrt-039 sh[1608]: ifup: interface ens1f0 already configured
# modprobe mlx5_core
# ifconfig ens1f0
ens1f0: flags=4098 mtu 1500
ether 24:8a:07:a1:fb:dc txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
# cat /etc/network/interfaces
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
# The loopback network interface
auto lo
iface lo inet loopback
# The primary network interface
auto eno1
iface eno1 inet dhcp
#ens1f0
auto ens1f0
iface ens1f0 inet static
address 123.12.23.1
netmask 255.255.0.0
mtu 1500
*
* Another repto and investigate *
*
once interface is created the system starts a service that is responsible for
activating it (basically runs ifup).
so, at first shot everything works.
at the second driver reload:
Good flow (on good setup 4.9.0-rc5+):
1. driver is unloaded and the interface’s “ifup” service is shutdown:
Feb 23 00:54:09 reg-l-vrt-206-006 kernel: [6.790189] mlx4_en: enP43508p0s2:
Close port called
Feb 23 00:54:09 reg-l-vrt-206-006 kernel: [6.868484] hv_netvsc
a2be13bb-7244-44ff-a31a-dea8d58a79da eth1: VF down: enP43508p0s2
Feb 23 00:54:09 reg-l-vrt-206-006 kernel: [6.868487] hv_netvsc
a2be13bb-7244-44ff-a31a-dea8d58a79da eth1: Data path switched from VF:
enP43508p0s2
Feb 23 00:54:09 reg-l-vrt-206-006 kernel: [6.869575] hv_netvsc
a2be13bb-7244-44ff-a31a-dea8d58a79da eth1: VF unregistering: enP43508p0s2
Feb 23 00:54:09 reg-l-vrt-206-006 systemd1: Stopping ifup for enP43508p0s2...
Feb 23 00:54:09 reg-l-vrt-206-006 ifdown47196: Cannot find device
"enP43508p0s2"
Feb 23 00:54:09 reg-l-vrt-206-006
