weizhouapache commented on issue #5716: URL: https://github.com/apache/cloudstack/issues/5716#issuecomment-1024057976
I had some discussion and investigation with @harikrishna-patnala when router.extra.public.nics is not 0 and vmware.systemvm.nic.device.type is set to "Vmxnet3" , the network VR will have more than 3 nics: 3 (guest, linklocal, source nat) + router.extra.public.nics (for extra public ip ranges). Therefore it will hit the problem which is not fixed by vmware for many years: the nic order inside VR is not same as defined in vmx file. please refer to https://communities.vmware.com/t5/ESXi-Discussions/Problem-with-the-order-of-interfaces-using-vmxnet3/td-p/2088500 I have tested cloudstack 4.14, which seem to be working. in cloudstack VR, there is a udev rule file to rename the network device name ``` root@r-6-VM:~# cat /etc/udev/rules.d/70-persistent-net.rules SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="02:00:6e:89:00:02", NAME="eth0" SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="02:00:52:a4:00:04", NAME="eth1" SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="1e:00:37:00:00:04", NAME="eth2" SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="02:00:3b:02:00:05", NAME="eth3" SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="02:00:25:30:00:06", NAME="eth4" ``` and it works fine (please notice the device name) ``` root@r-6-VM:~# ip a |egrep 'BROADCAST|ether' 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 02:00:6e:89:00:02 brd ff:ff:ff:ff:ff:ff 3: eth4: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000 link/ether 02:00:25:30:00:06 brd ff:ff:ff:ff:ff:ff 4: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 02:00:52:a4:00:04 brd ff:ff:ff:ff:ff:ff 5: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 1e:00:37:00:00:04 brd ff:ff:ff:ff:ff:ff 6: eth3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000 link/ether 02:00:3b:02:00:05 brd ff:ff:ff:ff:ff:ff ``` Unfortunately, it does not work in 4.16 (VR cannot be started). the /etc/udev/rules.d/70-persistent-net.rules still exists, but the device name are not changed, for example eth0, eth1, eth2, eth3, eth4, not the order above. this might be caused by some code changes (e.g. /opt/cloud/bin/setup/common.sh, /opt/cloud/bin/setup/router.sh), or systemvm template changes (e.g. systemd-udevd is used in latest Debian releases). @harikrishna-patnala is currently working on removing router.extra.public.nics, which I think is a good idea and should fix the issue above (there will be only 3 nics on network VR: guest, link local and public). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
