Public bug reported:
Description
===========
The test_attach_attached_volume_to_same_server tests within the nova-
next job fail during detaching the volume:
File "/usr/local/lib/python3.8/dist-packages/libvirt.py", line 1515, in
detachDeviceFlags
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5] raise
libvirtError('virDomainDetachDeviceFlags() failed')
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5] libvirt.libvirtError: operation
failed: disk vdb not found
Steps to reproduce
==================
Only seen as part of the nova-next job at present.
Expected result
===============
detach succeeds
Actual result
=============
test fails as the volume state
Environment
===========
1. Exact version of OpenStack you are running. See the following
list for all releases: http://docs.openstack.org/releases/
master
2. Which hypervisor did you use?
(For example: Libvirt + KVM, Libvirt + XEN, Hyper-V, PowerKVM, ...)
What's the version of that?
libvirt
2. Which storage type did you use?
(For example: Ceph, LVM, GPFS, ...)
What's the version of that?
cinder volume
3. Which networking type did you use?
(For example: nova-network, Neutron with OpenVSwitch, ...)
N/A
Logs & Configs
==============
https://zuul.opendev.org/t/openstack/build/02e6a99bf1574c978c663eb434705cbb/log/controller/logs/screen-n-cpu.txt?severity=0#34811
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [None req-aceedc3b-f03b-
4bd5-8e8e-cfe66170a2e2 tempest-AttachVolumeNegativeTest-1273499975
tempest-AttachVolumeNegativeTest-1273499975-project] [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5] Failed to detach volume
2939aa92-0156-45d6-9689-ed48dcc8fd8a from /dev/vdb:
libvirt.libvirtError: operation failed: disk vdb not found
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5] Traceback (most recent call last):
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5] File
"/opt/stack/nova/nova/virt/block_device.py", line 328, in driver_detach
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5]
virt_driver.detach_volume(context, connection_info, instance, mp,
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5] File
"/opt/stack/nova/nova/virt/libvirt/driver.py", line 2555, in
detach_volume
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5] self._detach_with_retry(
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5] File
"/opt/stack/nova/nova/virt/libvirt/driver.py", line 2308, in
_detach_with_retry
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5]
self._detach_from_live_with_retry(
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5] File
"/opt/stack/nova/nova/virt/libvirt/driver.py", line 2364, in
_detach_from_live_with_retry
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5]
self._detach_from_live_and_wait_for_event(
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5] File
"/opt/stack/nova/nova/virt/libvirt/driver.py", line 2426, in
_detach_from_live_and_wait_for_event
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5] self._detach_sync(
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5] File
"/opt/stack/nova/nova/virt/libvirt/driver.py", line 2498, in
_detach_sync
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5] guest.detach_device(dev,
persistent=persistent, live=live)
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5] File
"/opt/stack/nova/nova/virt/libvirt/guest.py", line 441, in detach_device
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5]
self._domain.detachDeviceFlags(device_xml, flags=flags)
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5] File "/usr/local/lib/python3.8
/dist-packages/eventlet/tpool.py", line 190, in doit
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5] result =
proxy_call(self._autowrap, f, *args, **kwargs)
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5] File "/usr/local/lib/python3.8
/dist-packages/eventlet/tpool.py", line 148, in proxy_call
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5] rv = execute(f, *args,
**kwargs)
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5] File "/usr/local/lib/python3.8
/dist-packages/eventlet/tpool.py", line 129, in execute
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR
nova.virt.block_device [instance: e3d6fd2b-7357-4d19-bcb4-482d97d332e5]
six.reraise(c, e, tb)
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5] File "/usr/local/lib/python3.8
/dist-packages/six.py", line 719, in reraise
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5] raise value
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5] File "/usr/local/lib/python3.8
/dist-packages/eventlet/tpool.py", line 83, in tworker
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5] rv = meth(*args, **kwargs)
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5] File "/usr/local/lib/python3.8
/dist-packages/libvirt.py", line 1515, in detachDeviceFlags
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5] raise
libvirtError('virDomainDetachDeviceFlags() failed')
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5] libvirt.libvirtError: operation
failed: disk vdb not found
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5]
** Affects: nova
Importance: Undecided
Status: New
** Tags: gate-failure
** Tags added: gate-failure
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1931716
Title:
Detaching volume from the live domain: virDomainDetachDeviceFlags():
libvirt.libvirtError: operation failed: disk vdb not found
Status in OpenStack Compute (nova):
New
Bug description:
Description
===========
The test_attach_attached_volume_to_same_server tests within the nova-
next job fail during detaching the volume:
File "/usr/local/lib/python3.8/dist-packages/libvirt.py", line 1515,
in detachDeviceFlags
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5] raise
libvirtError('virDomainDetachDeviceFlags() failed')
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5] libvirt.libvirtError: operation
failed: disk vdb not found
Steps to reproduce
==================
Only seen as part of the nova-next job at present.
Expected result
===============
detach succeeds
Actual result
=============
test fails as the volume state
Environment
===========
1. Exact version of OpenStack you are running. See the following
list for all releases: http://docs.openstack.org/releases/
master
2. Which hypervisor did you use?
(For example: Libvirt + KVM, Libvirt + XEN, Hyper-V, PowerKVM, ...)
What's the version of that?
libvirt
2. Which storage type did you use?
(For example: Ceph, LVM, GPFS, ...)
What's the version of that?
cinder volume
3. Which networking type did you use?
(For example: nova-network, Neutron with OpenVSwitch, ...)
N/A
Logs & Configs
==============
https://zuul.opendev.org/t/openstack/build/02e6a99bf1574c978c663eb434705cbb/log/controller/logs/screen-n-cpu.txt?severity=0#34811
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [None req-aceedc3b-f03b-
4bd5-8e8e-cfe66170a2e2 tempest-AttachVolumeNegativeTest-1273499975
tempest-AttachVolumeNegativeTest-1273499975-project] [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5] Failed to detach volume
2939aa92-0156-45d6-9689-ed48dcc8fd8a from /dev/vdb:
libvirt.libvirtError: operation failed: disk vdb not found
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5] Traceback (most recent call
last):
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5] File
"/opt/stack/nova/nova/virt/block_device.py", line 328, in
driver_detach
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5]
virt_driver.detach_volume(context, connection_info, instance, mp,
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5] File
"/opt/stack/nova/nova/virt/libvirt/driver.py", line 2555, in
detach_volume
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5] self._detach_with_retry(
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5] File
"/opt/stack/nova/nova/virt/libvirt/driver.py", line 2308, in
_detach_with_retry
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5]
self._detach_from_live_with_retry(
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5] File
"/opt/stack/nova/nova/virt/libvirt/driver.py", line 2364, in
_detach_from_live_with_retry
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5]
self._detach_from_live_and_wait_for_event(
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5] File
"/opt/stack/nova/nova/virt/libvirt/driver.py", line 2426, in
_detach_from_live_and_wait_for_event
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5] self._detach_sync(
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5] File
"/opt/stack/nova/nova/virt/libvirt/driver.py", line 2498, in
_detach_sync
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5] guest.detach_device(dev,
persistent=persistent, live=live)
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5] File
"/opt/stack/nova/nova/virt/libvirt/guest.py", line 441, in
detach_device
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5]
self._domain.detachDeviceFlags(device_xml, flags=flags)
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5] File "/usr/local/lib/python3.8
/dist-packages/eventlet/tpool.py", line 190, in doit
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5] result =
proxy_call(self._autowrap, f, *args, **kwargs)
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5] File "/usr/local/lib/python3.8
/dist-packages/eventlet/tpool.py", line 148, in proxy_call
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5] rv = execute(f, *args,
**kwargs)
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5] File "/usr/local/lib/python3.8
/dist-packages/eventlet/tpool.py", line 129, in execute
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR
nova.virt.block_device [instance: e3d6fd2b-7357-4d19-bcb4-482d97d332e5]
six.reraise(c, e, tb)
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5] File "/usr/local/lib/python3.8
/dist-packages/six.py", line 719, in reraise
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5] raise value
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5] File "/usr/local/lib/python3.8
/dist-packages/eventlet/tpool.py", line 83, in tworker
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5] rv = meth(*args, **kwargs)
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5] File "/usr/local/lib/python3.8
/dist-packages/libvirt.py", line 1515, in detachDeviceFlags
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5] raise
libvirtError('virDomainDetachDeviceFlags() failed')
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5] libvirt.libvirtError: operation
failed: disk vdb not found
Jun 11 13:30:47.071650 ubuntu-focal-inap-mtl01-0025074130 nova-
compute[108182]: ERROR nova.virt.block_device [instance:
e3d6fd2b-7357-4d19-bcb4-482d97d332e5]
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1931716/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
