This is happening very frequently now. Every time with a different libvirt error.
Like the one below: http://logs.openstack.org/93/462293/3/gate/gate-heat-dsvm-functional- convg-mysql-lbaasv2-non-apache-ubuntu- xenial/ea8c446/logs/screen-n-cpu.txt.gz?level=ERROR Jun 06 05:25:27.340500 ubuntu-xenial-osic-cloud1-s3500-9151671 nova-compute[4359]: ERROR oslo.service.loopingcall [-] Dynamic interval looping call 'oslo_service.loopingcall._func' failed: libvirtError: internal error: End of file from qemu monitor Jun 06 05:25:27.340614 ubuntu-xenial-osic-cloud1-s3500-9151671 nova-compute[4359]: ERROR oslo.service.loopingcall Traceback (most recent call last): Jun 06 05:25:27.340696 ubuntu-xenial-osic-cloud1-s3500-9151671 nova-compute[4359]: ERROR oslo.service.loopingcall File "/usr/local/lib/python2.7/dist-packages/oslo_service/loopingcall.py", line 137, in _run_loop Jun 06 05:25:27.340773 ubuntu-xenial-osic-cloud1-s3500-9151671 nova-compute[4359]: ERROR oslo.service.loopingcall result = func(*self.args, **self.kw) Jun 06 05:25:27.340850 ubuntu-xenial-osic-cloud1-s3500-9151671 nova-compute[4359]: ERROR oslo.service.loopingcall File "/usr/local/lib/python2.7/dist-packages/oslo_service/loopingcall.py", line 394, in _func Jun 06 05:25:27.340927 ubuntu-xenial-osic-cloud1-s3500-9151671 nova-compute[4359]: ERROR oslo.service.loopingcall result = f(*args, **kwargs) Jun 06 05:25:27.341001 ubuntu-xenial-osic-cloud1-s3500-9151671 nova-compute[4359]: ERROR oslo.service.loopingcall File "/opt/stack/new/nova/nova/virt/libvirt/guest.py", line 446, in _do_wait_and_retry_detach Jun 06 05:25:27.341091 ubuntu-xenial-osic-cloud1-s3500-9151671 nova-compute[4359]: ERROR oslo.service.loopingcall _try_detach_device(config, persistent=False, live=live) Jun 06 05:25:27.341172 ubuntu-xenial-osic-cloud1-s3500-9151671 nova-compute[4359]: ERROR oslo.service.loopingcall File "/opt/stack/new/nova/nova/virt/libvirt/guest.py", line 427, in _try_detach_device Jun 06 05:25:27.341257 ubuntu-xenial-osic-cloud1-s3500-9151671 nova-compute[4359]: ERROR oslo.service.loopingcall device=alternative_device_name) Jun 06 05:25:27.341347 ubuntu-xenial-osic-cloud1-s3500-9151671 nova-compute[4359]: ERROR oslo.service.loopingcall File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ Jun 06 05:25:27.341424 ubuntu-xenial-osic-cloud1-s3500-9151671 nova-compute[4359]: ERROR oslo.service.loopingcall self.force_reraise() Jun 06 05:25:27.341509 ubuntu-xenial-osic-cloud1-s3500-9151671 nova-compute[4359]: ERROR oslo.service.loopingcall File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise Jun 06 05:25:27.341602 ubuntu-xenial-osic-cloud1-s3500-9151671 nova-compute[4359]: ERROR oslo.service.loopingcall six.reraise(self.type_, self.value, self.tb) Jun 06 05:25:27.341688 ubuntu-xenial-osic-cloud1-s3500-9151671 nova-compute[4359]: ERROR oslo.service.loopingcall File "/opt/stack/new/nova/nova/virt/libvirt/guest.py", line 407, in _try_detach_device Jun 06 05:25:27.341766 ubuntu-xenial-osic-cloud1-s3500-9151671 nova-compute[4359]: ERROR oslo.service.loopingcall self.detach_device(conf, persistent=persistent, live=live) Jun 06 05:25:27.341843 ubuntu-xenial-osic-cloud1-s3500-9151671 nova-compute[4359]: ERROR oslo.service.loopingcall File "/opt/stack/new/nova/nova/virt/libvirt/guest.py", line 471, in detach_device Jun 06 05:25:27.341930 ubuntu-xenial-osic-cloud1-s3500-9151671 nova-compute[4359]: ERROR oslo.service.loopingcall self._domain.detachDeviceFlags(device_xml, flags=flags) Jun 06 05:25:27.342391 ubuntu-xenial-osic-cloud1-s3500-9151671 nova-compute[4359]: ERROR oslo.service.loopingcall File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 186, in doit Jun 06 05:25:27.342483 ubuntu-xenial-osic-cloud1-s3500-9151671 nova-compute[4359]: ERROR oslo.service.loopingcall result = proxy_call(self._autowrap, f, *args, **kwargs) Jun 06 05:25:27.342570 ubuntu-xenial-osic-cloud1-s3500-9151671 nova-compute[4359]: ERROR oslo.service.loopingcall File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 144, in proxy_call Jun 06 05:25:27.342661 ubuntu-xenial-osic-cloud1-s3500-9151671 nova-compute[4359]: ERROR oslo.service.loopingcall rv = execute(f, *args, **kwargs) Jun 06 05:25:27.342738 ubuntu-xenial-osic-cloud1-s3500-9151671 nova-compute[4359]: ERROR oslo.service.loopingcall File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 125, in execute Jun 06 05:25:27.342814 ubuntu-xenial-osic-cloud1-s3500-9151671 nova-compute[4359]: ERROR oslo.service.loopingcall six.reraise(c, e, tb) Jun 06 05:25:27.342897 ubuntu-xenial-osic-cloud1-s3500-9151671 nova-compute[4359]: ERROR oslo.service.loopingcall File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 83, in tworker Jun 06 05:25:27.342985 ubuntu-xenial-osic-cloud1-s3500-9151671 nova-compute[4359]: ERROR oslo.service.loopingcall rv = meth(*args, **kwargs) Jun 06 05:25:27.343068 ubuntu-xenial-osic-cloud1-s3500-9151671 nova-compute[4359]: ERROR oslo.service.loopingcall File "/usr/local/lib/python2.7/dist-packages/libvirt.py", line 1215, in detachDeviceFlags Jun 06 05:25:27.343149 ubuntu-xenial-osic-cloud1-s3500-9151671 nova-compute[4359]: ERROR oslo.service.loopingcall if ret == -1: raise libvirtError ('virDomainDetachDeviceFlags() failed', dom=self) Jun 06 05:25:27.343224 ubuntu-xenial-osic-cloud1-s3500-9151671 nova-compute[4359]: ERROR oslo.service.loopingcall libvirtError: internal error: End of file from qemu monitor Jun 06 05:25:27.343298 ubuntu-xenial-osic-cloud1-s3500-9151671 nova-compute[4359]: ERROR oslo.service.loopingcall ** Changed in: heat Importance: Undecided => High ** Also affects: nova Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1585858 Title: InterfaceDetachFailed: resources.server: Failed to detach interface Status in heat: Triaged Status in OpenStack Compute (nova): New Bug description: It seems the number of attempts to detach interfaces in not enough in some cases. heat_integrationtests.functional.test_create_update.UpdateStackTest.test_stack_update_with_replacing_userdata 2016-05-25 10:37:16.151 | 2016-05-25 10:37:16.020 | ------------------------------------------------------------------------------------------------------------- 2016-05-25 10:37:16.151 | 2016-05-25 10:37:16.022 | 2016-05-25 10:37:16.152 | 2016-05-25 10:37:16.025 | Captured traceback: 2016-05-25 10:37:16.152 | 2016-05-25 10:37:16.031 | ~~~~~~~~~~~~~~~~~~~ 2016-05-25 10:37:16.152 | 2016-05-25 10:37:16.034 | Traceback (most recent call last): 2016-05-25 10:37:16.152 | 2016-05-25 10:37:16.038 | File "/opt/stack/new/heat/heat_integrationtests/functional/test_create_update.py", line 534, in test_stack_update_with_replacing_userdata 2016-05-25 10:37:16.152 | 2016-05-25 10:37:16.040 | parameters=parms_updated) 2016-05-25 10:37:16.153 | 2016-05-25 10:37:16.044 | File "/opt/stack/new/heat/heat_integrationtests/common/test.py", line 402, in update_stack 2016-05-25 10:37:16.153 | 2016-05-25 10:37:16.046 | self._wait_for_stack_status(**kwargs) 2016-05-25 10:37:16.153 | 2016-05-25 10:37:16.048 | File "/opt/stack/new/heat/heat_integrationtests/common/test.py", line 334, in _wait_for_stack_status 2016-05-25 10:37:16.153 | 2016-05-25 10:37:16.051 | fail_regexp): 2016-05-25 10:37:16.153 | 2016-05-25 10:37:16.054 | File "/opt/stack/new/heat/heat_integrationtests/common/test.py", line 295, in _verify_status 2016-05-25 10:37:16.153 | 2016-05-25 10:37:16.056 | stack_status_reason=stack.stack_status_reason) 2016-05-25 10:37:16.154 | 2016-05-25 10:37:16.058 | heat_integrationtests.common.exceptions.StackBuildErrorException: Stack UpdateStackTest-2129656241/3c284793-623d-47bd-901a-6d71e75bdbca is in UPDATE_FAILED status due to 'InterfaceDetachFailed: resources.server: Failed to detach interface (bdbc4a8c-114b-42c0-a49a-a27311f5f2e2) from server (259f3fff-6e38-4722-9a51-7c7166d78812)' To manage notifications about this bug go to: https://bugs.launchpad.net/heat/+bug/1585858/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
