Public bug reported: Richard Theis reported the regression here:
https://review.openstack.org/#/c/370681/9/nova/virt/libvirt/vif.py Shown here: http://logs.openstack.org/75/371175/1/check/gate-tempest-dsvm- networking- ovn/7e52927/logs/screen-n-cpu.txt.gz?level=TRACE#_2016-09-16_11_20_39_627 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [req-78de89a8-2053-42d9-899f-04aa2b8f25c7 tempest-FloatingIPsTestJSON-1913520192 tempest-FloatingIPsTestJSON-1913520192] [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] Instance failed to spawn 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] Traceback (most recent call last): 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] File "/opt/stack/new/nova/nova/compute/manager.py", line 2078, in _build_resources 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] yield resources 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] File "/opt/stack/new/nova/nova/compute/manager.py", line 1920, in _build_and_run_instance 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] block_device_info=block_device_info) 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 2583, in spawn 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] post_xml_callback=gen_confdrive) 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 4814, in _create_domain_and_network 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] self.plug_vifs(instance, network_info) 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 684, in plug_vifs 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] self.vif_driver.plug(instance, vif) 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] File "/opt/stack/new/nova/nova/virt/libvirt/vif.py", line 817, in plug 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] self._plug_os_vif(instance, vif_obj, vif) 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] File "/opt/stack/new/nova/nova/virt/libvirt/vif.py", line 799, in _plug_os_vif 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] linux_net._set_device_mtu(veth, mtu) 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] File "/opt/stack/new/nova/nova/network/linux_net.py", line 1237, in _set_device_mtu 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] check_exit_code=[0, 2, 254]) 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] File "/opt/stack/new/nova/nova/utils.py", line 295, in execute 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] return RootwrapProcessHelper().execute(*cmd, **kwargs) 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] File "/opt/stack/new/nova/nova/utils.py", line 178, in execute 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] return processutils.execute(*cmd, **kwargs) 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] File "/usr/local/lib/python2.7/dist-packages/oslo_concurrency/processutils.py", line 389, in execute 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] cmd=sanitized_cmd) 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] ProcessExecutionError: Unexpected error while running command. 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] Command: sudo nova-rootwrap /etc/nova/rootwrap.conf ip link set qvb00b6a38f-60 mtu 1442 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] Exit code: 1 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] Stdout: u'' 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] Stderr: u'Cannot find device "qvb00b6a38f-60"\n' 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] The nova team isn't sure if ovn should have those devices or not by the time we've plugged the vif, or if ovn is a special case and those just won't exist. We're going to add device exists checks around that code to be safe and get that into newton-rc2 and move forward unless someone more that knows more about OVN can tell us otherwise. ** Affects: nova Importance: Undecided Status: New ** Tags: libvirt neutron newton-rc-potential ovn -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1624383 Title: vif plugging fails with ovn when trying to set mtu on qvb device that does not exist Status in OpenStack Compute (nova): New Bug description: Richard Theis reported the regression here: https://review.openstack.org/#/c/370681/9/nova/virt/libvirt/vif.py Shown here: http://logs.openstack.org/75/371175/1/check/gate-tempest-dsvm- networking- ovn/7e52927/logs/screen-n-cpu.txt.gz?level=TRACE#_2016-09-16_11_20_39_627 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [req-78de89a8-2053-42d9-899f-04aa2b8f25c7 tempest-FloatingIPsTestJSON-1913520192 tempest-FloatingIPsTestJSON-1913520192] [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] Instance failed to spawn 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] Traceback (most recent call last): 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] File "/opt/stack/new/nova/nova/compute/manager.py", line 2078, in _build_resources 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] yield resources 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] File "/opt/stack/new/nova/nova/compute/manager.py", line 1920, in _build_and_run_instance 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] block_device_info=block_device_info) 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 2583, in spawn 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] post_xml_callback=gen_confdrive) 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 4814, in _create_domain_and_network 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] self.plug_vifs(instance, network_info) 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 684, in plug_vifs 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] self.vif_driver.plug(instance, vif) 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] File "/opt/stack/new/nova/nova/virt/libvirt/vif.py", line 817, in plug 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] self._plug_os_vif(instance, vif_obj, vif) 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] File "/opt/stack/new/nova/nova/virt/libvirt/vif.py", line 799, in _plug_os_vif 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] linux_net._set_device_mtu(veth, mtu) 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] File "/opt/stack/new/nova/nova/network/linux_net.py", line 1237, in _set_device_mtu 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] check_exit_code=[0, 2, 254]) 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] File "/opt/stack/new/nova/nova/utils.py", line 295, in execute 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] return RootwrapProcessHelper().execute(*cmd, **kwargs) 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] File "/opt/stack/new/nova/nova/utils.py", line 178, in execute 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] return processutils.execute(*cmd, **kwargs) 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] File "/usr/local/lib/python2.7/dist-packages/oslo_concurrency/processutils.py", line 389, in execute 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] cmd=sanitized_cmd) 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] ProcessExecutionError: Unexpected error while running command. 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] Command: sudo nova-rootwrap /etc/nova/rootwrap.conf ip link set qvb00b6a38f-60 mtu 1442 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] Exit code: 1 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] Stdout: u'' 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] Stderr: u'Cannot find device "qvb00b6a38f-60"\n' 2016-09-16 11:21:29.140 15810 ERROR nova.compute.manager [instance: fa58495c-1f29-4149-9efb-2bbf7624d221] The nova team isn't sure if ovn should have those devices or not by the time we've plugged the vif, or if ovn is a special case and those just won't exist. We're going to add device exists checks around that code to be safe and get that into newton-rc2 and move forward unless someone more that knows more about OVN can tell us otherwise. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1624383/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
