Public bug reported: Seen here:
http://logs.openstack.org/22/606122/7/check/nova-tox- functional/1f3126b/testr_results.html.gz There are different failures, but the tests were introduced with this change: https://review.openstack.org/#/c/591733/ For example, in nova.tests.functional.regressions.test_bug_1550919.LibvirtRbdEvacuateTest.test_evacuate_shared_instancedir I'm seeing: 2018-11-30 17:03:58,424 ERROR [nova.virt.libvirt.driver] Failed to start libvirt guest And this (during evacuate from compute0 to compute1 in the test): Traceback (most recent call last): File "nova/compute/manager.py", line 7897, in _error_out_instance_on_exception yield File "nova/compute/manager.py", line 3040, in rebuild_instance migration, request_spec) File "nova/compute/manager.py", line 3102, in _do_rebuild_instance_with_claim self._do_rebuild_instance(*args, **kwargs) File "nova/compute/manager.py", line 3265, in _do_rebuild_instance self._rebuild_default_impl(**kwargs) File "nova/compute/manager.py", line 2929, in _rebuild_default_impl block_device_info=new_block_device_info) File "nova/virt/libvirt/driver.py", line 3079, in spawn destroy_disks_on_failure=True) File "nova/virt/libvirt/driver.py", line 5648, in _create_domain_and_network destroy_disks_on_failure) File "/home/zuul/src/git.openstack.org/openstack/nova/.tox/py27/local/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ self.force_reraise() File "/home/zuul/src/git.openstack.org/openstack/nova/.tox/py27/local/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise six.reraise(self.type_, self.value, self.tb) File "nova/virt/libvirt/driver.py", line 5607, in _create_domain_and_network self.plug_vifs(instance, network_info) File "/home/zuul/src/git.openstack.org/openstack/nova/.tox/py27/local/lib/python2.7/site-packages/mock/mock.py", line 1062, in __call__ return _mock_self._mock_call(*args, **kwargs) File "/home/zuul/src/git.openstack.org/openstack/nova/.tox/py27/local/lib/python2.7/site-packages/mock/mock.py", line 1118, in _mock_call raise effect TestingException I'm guessing something is mocking global state and causing that failure...we likely need a fake virt driver that extends the libvirt driver and does what we want for these tests to avoid global mocks. In the other test, I see a timeout: nova.tests.functional.regressions.test_bug_1550919.LibvirtFlatEvacuateTest.test_evacuate_nonshared_instancedir And it seems to be stuck here: 2018-11-30 17:03:05,111 INFO [nova.virt.libvirt.driver] Creating image http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22INFO%20%5Bnova.virt.libvirt.driver%5D%20Creating%20image%5C%22%20AND%20tags%3A%5C%22console%5C%22&from=7d 32 hits in the last 48 hours, check and gate, all failures. ** Affects: nova Importance: High Status: Confirmed ** Tags: gate-failure libvirt testing -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1806126 Title: LibvirtRbdEvacuateTest and LibvirtFlatEvacuateTest tests race fail Status in OpenStack Compute (nova): Confirmed Bug description: Seen here: http://logs.openstack.org/22/606122/7/check/nova-tox- functional/1f3126b/testr_results.html.gz There are different failures, but the tests were introduced with this change: https://review.openstack.org/#/c/591733/ For example, in nova.tests.functional.regressions.test_bug_1550919.LibvirtRbdEvacuateTest.test_evacuate_shared_instancedir I'm seeing: 2018-11-30 17:03:58,424 ERROR [nova.virt.libvirt.driver] Failed to start libvirt guest And this (during evacuate from compute0 to compute1 in the test): Traceback (most recent call last): File "nova/compute/manager.py", line 7897, in _error_out_instance_on_exception yield File "nova/compute/manager.py", line 3040, in rebuild_instance migration, request_spec) File "nova/compute/manager.py", line 3102, in _do_rebuild_instance_with_claim self._do_rebuild_instance(*args, **kwargs) File "nova/compute/manager.py", line 3265, in _do_rebuild_instance self._rebuild_default_impl(**kwargs) File "nova/compute/manager.py", line 2929, in _rebuild_default_impl block_device_info=new_block_device_info) File "nova/virt/libvirt/driver.py", line 3079, in spawn destroy_disks_on_failure=True) File "nova/virt/libvirt/driver.py", line 5648, in _create_domain_and_network destroy_disks_on_failure) File "/home/zuul/src/git.openstack.org/openstack/nova/.tox/py27/local/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ self.force_reraise() File "/home/zuul/src/git.openstack.org/openstack/nova/.tox/py27/local/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise six.reraise(self.type_, self.value, self.tb) File "nova/virt/libvirt/driver.py", line 5607, in _create_domain_and_network self.plug_vifs(instance, network_info) File "/home/zuul/src/git.openstack.org/openstack/nova/.tox/py27/local/lib/python2.7/site-packages/mock/mock.py", line 1062, in __call__ return _mock_self._mock_call(*args, **kwargs) File "/home/zuul/src/git.openstack.org/openstack/nova/.tox/py27/local/lib/python2.7/site-packages/mock/mock.py", line 1118, in _mock_call raise effect TestingException I'm guessing something is mocking global state and causing that failure...we likely need a fake virt driver that extends the libvirt driver and does what we want for these tests to avoid global mocks. In the other test, I see a timeout: nova.tests.functional.regressions.test_bug_1550919.LibvirtFlatEvacuateTest.test_evacuate_nonshared_instancedir And it seems to be stuck here: 2018-11-30 17:03:05,111 INFO [nova.virt.libvirt.driver] Creating image http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22INFO%20%5Bnova.virt.libvirt.driver%5D%20Creating%20image%5C%22%20AND%20tags%3A%5C%22console%5C%22&from=7d 32 hits in the last 48 hours, check and gate, all failures. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1806126/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
