[Yahoo-eng-team] [Bug 1841582] Re: Interface detach and attach does not work with CentOS 7 and Ubuntu 18.04 cloud images
** Project changed: nova => cloud-images -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1841582 Title: Interface detach and attach does not work with CentOS 7 and Ubuntu 18.04 cloud images Status in cloud-images: New Bug description: Hello, I uploaded bionic-server-cloudimg-amd64.img into glance. I started an Instance. The instance is reachable. Next i shutdown the instance openstack server stop detach the interface openstack server remove network afterwards I attach a new interface with the same IP as before openstack server add fixed ip --fixed-ip-address then start the instance and the instance is not reachable. I can reproduce this behaviour with. bionic-server-cloudimg-amd64.img CentOS-7-x86_64-GenericCloud-1907.qcow2 This does not happen with : CentOS-6-x86_64-GenericCloud-1907.qcow2 xenial-server-cloudimg-amd64-disk1.img trusty-server-cloudimg-amd64-disk1.img cirros-0.4.0-x86_64-disk.img The images are unchanged. I logged in via local console into ubuntu 18:04. The interface was down and I could see the following logs: Aug 26 08:38:17 os-steb-pa1 systemd[1]: Starting Apply the settings specified in cloud-config... Aug 26 08:38:17 os-steb-pa1 networkd-dispatcher[754]: No valid path found for iwconfig Aug 26 08:38:17 os-steb-pa1 networkd-dispatcher[754]: No valid path found for iw A dhclient -v started the interface and the the instance got an answer from dhcp and was reachable again. I logged in via local console into CentOS 7. The interface was also down and I could see the following logs: Aug 26 09:05:37 os-steb-cl1 network: Bringing up interface eth0: ERROR : [/etc/sysconfig/network-scripts/ifup-eth] Device eth0 has different MAC address than expected, ignoring. Aug 26 09:05:37 os-steb-cl1 /etc/sysconfig/network-scripts/ifup-eth: Device eth0 has different MAC address than expected, ignoring. Aug 26 09:05:37 os-steb-cl1 network: [FAILED] Aug 26 09:05:37 os-steb-cl1 systemd: network.service: control process exited, code=exited status=1 A dhclient -v started the interface and the the instance got an answer from dhcp and was reachable again. The problem is the old mac address in ubuntu 18.04 /etc/netplan/50-cloud-init.yaml centos 7 /etc/sysconfig/network-scripts/ifcfg-eth0 Manually changing the mac address in these files to the new one, solves the problem and the instances are reachable again after reboots. I don't know how the mechanism worked for the older operating systems to establish a network connection after the interface changed via openstack, but this seems to be broken with the newer operating systems. Environment === 1. Rocky ii nova-api 2:18.1.0-0ubuntu1~cloud0 all OpenStack Compute - API frontend ii nova-common 2:18.1.0-0ubuntu1~cloud0 all OpenStack Compute - common files ii nova-conductor2:18.1.0-0ubuntu1~cloud0 all OpenStack Compute - conductor service ii nova-consoleauth 2:18.1.0-0ubuntu1~cloud0 all OpenStack Compute - Console Authenticator ii nova-novncproxy 2:18.1.0-0ubuntu1~cloud0 all OpenStack Compute - NoVNC proxy ii nova-placement-api2:18.1.0-0ubuntu1~cloud0 all OpenStack Compute - placement API frontend ii nova-scheduler2:18.1.0-0ubuntu1~cloud0 all OpenStack Compute - virtual machine scheduler ii python-nova 2:18.1.0-0ubuntu1~cloud0 all OpenStack Compute Python 2 libraries 2. Libvirt + KVM ii qemu-kvm 1:2.11+dfsg-1ubuntu7.17 amd64QEMU Full virtualization on x86 hardware ii libvirt-daemon4.0.0-1ubuntu8.12 amd64Virtualization daemon ii libvirt-daemon-driver-storage-rbd 4.0.0-1ubuntu8.12 amd64Virtualization daemon RBD storage driver ii libvirt0:amd644.0.0-1ubuntu8.12 amd64library for interfacing with different virtualization systems ii python-libvirt4.0.0-1 amd64libvirt Python bindings 3. Neutron with OpenVSwitch and dvr_snat Greets To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-images/+bug/1841582/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe :
[Yahoo-eng-team] [Bug 1841582] [NEW] Interface detach and attach does not work with CentOS 7 and Ubuntu 18.04 cloud images
Public bug reported: Hello, I uploaded bionic-server-cloudimg-amd64.img into glance. I started an Instance. The instance is reachable. Next i shutdown the instance openstack server stop detach the interface openstack server remove network afterwards I attach a new interface with the same IP as before openstack server add fixed ip --fixed-ip-address then start the instance and the instance is not reachable. I can reproduce this behaviour with. bionic-server-cloudimg-amd64.img CentOS-7-x86_64-GenericCloud-1907.qcow2 This does not happen with : CentOS-6-x86_64-GenericCloud-1907.qcow2 xenial-server-cloudimg-amd64-disk1.img trusty-server-cloudimg-amd64-disk1.img cirros-0.4.0-x86_64-disk.img The images are unchanged. I logged in via local console into ubuntu 18:04. The interface was down and I could see the following logs: Aug 26 08:38:17 os-steb-pa1 systemd[1]: Starting Apply the settings specified in cloud-config... Aug 26 08:38:17 os-steb-pa1 networkd-dispatcher[754]: No valid path found for iwconfig Aug 26 08:38:17 os-steb-pa1 networkd-dispatcher[754]: No valid path found for iw A dhclient -v started the interface and the the instance got an answer from dhcp and was reachable again. I logged in via local console into CentOS 7. The interface was also down and I could see the following logs: Aug 26 09:05:37 os-steb-cl1 network: Bringing up interface eth0: ERROR : [/etc/sysconfig/network-scripts/ifup-eth] Device eth0 has different MAC address than expected, ignoring. Aug 26 09:05:37 os-steb-cl1 /etc/sysconfig/network-scripts/ifup-eth: Device eth0 has different MAC address than expected, ignoring. Aug 26 09:05:37 os-steb-cl1 network: [FAILED] Aug 26 09:05:37 os-steb-cl1 systemd: network.service: control process exited, code=exited status=1 A dhclient -v started the interface and the the instance got an answer from dhcp and was reachable again. The problem is the old mac address in ubuntu 18.04 /etc/netplan/50-cloud-init.yaml centos 7 /etc/sysconfig/network-scripts/ifcfg-eth0 Manually changing the mac address in these files to the new one, solves the problem and the instances are reachable again after reboots. I don't know how the mechanism worked for the older operating systems to establish a network connection after the interface changed via openstack, but this seems to be broken with the newer operating systems. Environment === 1. Rocky ii nova-api 2:18.1.0-0ubuntu1~cloud0 all OpenStack Compute - API frontend ii nova-common 2:18.1.0-0ubuntu1~cloud0 all OpenStack Compute - common files ii nova-conductor2:18.1.0-0ubuntu1~cloud0 all OpenStack Compute - conductor service ii nova-consoleauth 2:18.1.0-0ubuntu1~cloud0 all OpenStack Compute - Console Authenticator ii nova-novncproxy 2:18.1.0-0ubuntu1~cloud0 all OpenStack Compute - NoVNC proxy ii nova-placement-api2:18.1.0-0ubuntu1~cloud0 all OpenStack Compute - placement API frontend ii nova-scheduler2:18.1.0-0ubuntu1~cloud0 all OpenStack Compute - virtual machine scheduler ii python-nova 2:18.1.0-0ubuntu1~cloud0 all OpenStack Compute Python 2 libraries 2. Libvirt + KVM ii qemu-kvm 1:2.11+dfsg-1ubuntu7.17 amd64QEMU Full virtualization on x86 hardware ii libvirt-daemon4.0.0-1ubuntu8.12 amd64Virtualization daemon ii libvirt-daemon-driver-storage-rbd 4.0.0-1ubuntu8.12 amd64Virtualization daemon RBD storage driver ii libvirt0:amd644.0.0-1ubuntu8.12 amd64library for interfacing with different virtualization systems ii python-libvirt4.0.0-1 amd64libvirt Python bindings 3. Neutron with OpenVSwitch and dvr_snat Greets ** Affects: nova Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1841582 Title: Interface detach and attach does not work with CentOS 7 and Ubuntu 18.04 cloud images Status in OpenStack Compute (nova): New Bug description: Hello, I uploaded bionic-server-cloudimg-amd64.img into glance. I started an Instance. The instance is reachable. Next i shutdown the instance openstack server stop detach the interface openstack server remove network afterwards I attach a new interface with the same IP as before openst
[Yahoo-eng-team] [Bug 1622578] [NEW] cloud init metadata injection failed while booting VM
Public bug reported: Hello, when we start a VM the process of cloud init failed with: Inside the VM url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [62/120s]: bad status code [500] The VM is reachable and you can curl http://169.254.169.254 and get a valid result but when you curl http://169.254.169.254/latest we get: Remote metadata server experienced an internal server error. In nova-api log I see: Unauthorized: The request you have made requires authentication. (HTTP 401) Environment OS: Ubuntu 16.04.1 LTS packages on nova: ii nova-api 2:13.1.0-0ubuntu1 all OpenStack Compute - API frontend ii nova-common2:13.1.0-0ubuntu1 all OpenStack Compute - common files ii nova-conductor 2:13.1.0-0ubuntu1 all OpenStack Compute - conductor service ii nova-consoleauth 2:13.1.0-0ubuntu1 all OpenStack Compute - Console Authenticator ii nova-novncproxy2:13.1.0-0ubuntu1 all OpenStack Compute - NoVNC proxy ii nova-scheduler 2:13.1.0-0ubuntu1 all OpenStack Compute - virtual machine scheduler ii nova-spiceproxy2:13.1.0-0ubuntu1 all OpenStack Compute - spice html5 proxy ii python-nova2:13.1.0-0ubuntu1 all OpenStack Compute Python libraries ii python-novaclient 2:3.3.1-2 all client library for OpenStack Compute API - Python 2.7 ii python-neutronclient 1:4.1.1-2 all client API library for Neutron - Python 2.7 packages on neutron: ii neutron-common 2:8.1.2-0ubuntu1all Neutron is a virtual network service for Openstack - common ii neutron-dhcp-agent 2:8.1.2-0ubuntu1all Neutron is a virtual network service for Openstack - DHCP agent ii neutron-l3-agent 2:8.1.2-0ubuntu1all Neutron is a virtual network service for Openstack - l3 agent ii neutron-metadata-agent 2:8.1.2-0ubuntu1all Neutron is a virtual network service for Openstack - metadata agent ii neutron-openvswitch-agent 2:8.1.2-0ubuntu1all Neutron is a virtual network service for Openstack - Open vSwitch plugin agent ii neutron-plugin-ml2 2:8.1.2-0ubuntu1all Neutron is a virtual network service for Openstack - ML2 plugin ii neutron-plugin-openvswitch-agent 2:8.1.2-0ubuntu1all Transitional package for neutron-openvswitch-agent ii neutron-server 2:8.1.2-0ubuntu1all Neutron is a virtual network service for Openstack - server ii python-neutron 2:8.1.2-0ubuntu1all Neutron is a virtual network service for Openstack - Python library ii python-neutron-fwaas 1:8.0.0-0ubuntu1all Firewall-as-a-Service driver for OpenStack Neutron ii python-neutron-lib 0.0.2-2 all Neutron shared routines and utilities - Python 2.7 ii python-neutronclient 1:4.1.1-2 all client API library for Neutron - Python 2.7 Hypervisor is KVM ii nova-compute-kvm 2:13.1.0-0ubuntu1 all OpenStack Compute - compute node (KVM) ii libvirt-bin 1.3.1-1ubuntu10.1 amd64programs for the libvirt library ii libvirt0:amd64 1.3.1-1ubuntu10.1 amd64library for interfacing with different virtualization systems ii nova-compute-libvirt 2:13.1.0-0ubuntu1 all OpenStack Compute - compute node libvirt support ii python-libvirt 1.3.1-1ubuntu1 amd64libvirt Python bindings Storage is CEPH root@openstack11:~# ceph version ceph version 10.2.2 Network is Neutron with openvswitch I tried a lot to get around this issue, I hope I didn't oversee something. Maybe someone can help or fix this bug. Thanks in advance ** Affects: nova Importance: Undecided Status: New ** Attachment added: "logs.zip" https://bugs.launchpad.net/bugs/1622578/+attachment/4739277/+files/logs.zip -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1622578 Title: cloud init metadata injection failed w
