Reviewed: https://review.opendev.org/c/openstack/nova/+/852737 Committed: https://opendev.org/openstack/nova/commit/6eed55bf55469f4ceaa7d4d4eb1be635e14bc73b Submitter: "Zuul (22348)" Branch: master
commit 6eed55bf55469f4ceaa7d4d4eb1be635e14bc73b Author: Rajesh Tailor <ratai...@redhat.com> Date: Wed Aug 10 18:15:04 2022 +0530 Fix rescue volume-based instance As of now, when attempting to rescue a volume-based instance using an image without the hw_rescue_device and/or hw_rescue_bus properties set, the rescue api call fails (as non-stable rescue for volume-based instances are not supported) leaving the instance in error state. This change checks for hw_rescue_device/hw_rescue_bus image properties before attempting to rescue and if the property is not set, then fail with proper error message, without changing instance state. Related-Bug: #1978958 Closes-Bug: #1926601 Change-Id: Id4c8c5f3b32985ac7d3d7c833b82e0876f7367c1 ** Changed in: nova Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1926601 Title: Rescuing RBD volume-backed instance does not work Status in OpenStack Compute (nova): Fix Released Bug description: Context: - Openstack Victoria deployment - Separate ceph pools for Cinder volumes and Nova VMs. - Trying to rescue a volume-backed instance with --image parameter. The rescue disk is not even created in the VMs pool, so the instance is put into ERROR because of the following exception: libvirt.libvirtError: internal error: process exited while connecting to monitor: 2021-04-29T11:19:00.527948Z qemu-system-x86_64: -blockdev {"driver":"rbd","pool":"vms","image":"f2dfce55-94cb-43e2-b799-9f9a2671c38b_disk","server":[{"host":"10.0.1.81","port":"6789"},{"host":"10.0.1.82","port":"6789"},{"host":"10.0.1.83","port":"6789"}],"user":"nova","auth- client-required":["cephx","none"],"key-secret":"libvirt-1-storage- secret0","node-name":"libvirt-1-storage","cache":{"direct":false,"no- flush":false},"auto-read-only":true,"discard":"unmap"}: error reading header from f2dfce55-94cb-43e2-b799-9f9a2671c38b_disk: No such file or directory Stacktrace 2021-04-29 11:19:00.774 6 ERROR nova.compute.manager [instance: f2dfce55-94cb-43e2-b799-9f9a2671c38b] Traceback (most recent call last): 2021-04-29 11:19:00.774 6 ERROR nova.compute.manager [instance: f2dfce55-94cb-43e2-b799-9f9a2671c38b] File "/var/lib/kolla/venv/lib/python3.8/site-packages/nova/compute/manager.py", line 4178, in rescue_instance 2021-04-29 11:19:00.774 6 ERROR nova.compute.manager [instance: f2dfce55-94cb-43e2-b799-9f9a2671c38b] self.driver.rescue(context, instance, network_info, 2021-04-29 11:19:00.774 6 ERROR nova.compute.manager [instance: f2dfce55-94cb-43e2-b799-9f9a2671c38b] File "/var/lib/kolla/venv/lib/python3.8/site-packages/nova/virt/libvirt/driver.py", line 3668, in rescue 2021-04-29 11:19:00.774 6 ERROR nova.compute.manager [instance: f2dfce55-94cb-43e2-b799-9f9a2671c38b] self._create_guest( 2021-04-29 11:19:00.774 6 ERROR nova.compute.manager [instance: f2dfce55-94cb-43e2-b799-9f9a2671c38b] File "/var/lib/kolla/venv/lib/python3.8/site-packages/nova/virt/libvirt/driver.py", line 6637, in _create_guest 2021-04-29 11:19:00.774 6 ERROR nova.compute.manager [instance: f2dfce55-94cb-43e2-b799-9f9a2671c38b] guest.launch(pause=pause) 2021-04-29 11:19:00.774 6 ERROR nova.compute.manager [instance: f2dfce55-94cb-43e2-b799-9f9a2671c38b] File "/var/lib/kolla/venv/lib/python3.8/site-packages/nova/virt/libvirt/guest.py", line 158, in launch 2021-04-29 11:19:00.774 6 ERROR nova.compute.manager [instance: f2dfce55-94cb-43e2-b799-9f9a2671c38b] LOG.error('Error launching a defined domain ' 2021-04-29 11:19:00.774 6 ERROR nova.compute.manager [instance: f2dfce55-94cb-43e2-b799-9f9a2671c38b] File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2021-04-29 11:19:00.774 6 ERROR nova.compute.manager [instance: f2dfce55-94cb-43e2-b799-9f9a2671c38b] self.force_reraise() 2021-04-29 11:19:00.774 6 ERROR nova.compute.manager [instance: f2dfce55-94cb-43e2-b799-9f9a2671c38b] File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2021-04-29 11:19:00.774 6 ERROR nova.compute.manager [instance: f2dfce55-94cb-43e2-b799-9f9a2671c38b] six.reraise(self.type_, self.value, self.tb) 2021-04-29 11:19:00.774 6 ERROR nova.compute.manager [instance: f2dfce55-94cb-43e2-b799-9f9a2671c38b] File "/usr/local/lib/python3.8/dist-packages/six.py", line 703, in reraise 2021-04-29 11:19:00.774 6 ERROR nova.compute.manager [instance: f2dfce55-94cb-43e2-b799-9f9a2671c38b] raise value 2021-04-29 11:19:00.774 6 ERROR nova.compute.manager [instance: f2dfce55-94cb-43e2-b799-9f9a2671c38b] File "/var/lib/kolla/venv/lib/python3.8/site-packages/nova/virt/libvirt/guest.py", line 155, in launch 2021-04-29 11:19:00.774 6 ERROR nova.compute.manager [instance: f2dfce55-94cb-43e2-b799-9f9a2671c38b] return self._domain.createWithFlags(flags) 2021-04-29 11:19:00.774 6 ERROR nova.compute.manager [instance: f2dfce55-94cb-43e2-b799-9f9a2671c38b] File "/var/lib/kolla/venv/lib/python3.8/site-packages/eventlet/tpool.py", line 190, in doit 2021-04-29 11:19:00.774 6 ERROR nova.compute.manager [instance: f2dfce55-94cb-43e2-b799-9f9a2671c38b] result = proxy_call(self._autowrap, f, *args, **kwargs) 2021-04-29 11:19:00.774 6 ERROR nova.compute.manager [instance: f2dfce55-94cb-43e2-b799-9f9a2671c38b] File "/var/lib/kolla/venv/lib/python3.8/site-packages/eventlet/tpool.py", line 148, in proxy_call 2021-04-29 11:19:00.774 6 ERROR nova.compute.manager [instance: f2dfce55-94cb-43e2-b799-9f9a2671c38b] rv = execute(f, *args, **kwargs) 2021-04-29 11:19:00.774 6 ERROR nova.compute.manager [instance: f2dfce55-94cb-43e2-b799-9f9a2671c38b] File "/var/lib/kolla/venv/lib/python3.8/site-packages/eventlet/tpool.py", line 129, in execute 2021-04-29 11:19:00.774 6 ERROR nova.compute.manager [instance: f2dfce55-94cb-43e2-b799-9f9a2671c38b] six.reraise(c, e, tb) 2021-04-29 11:19:00.774 6 ERROR nova.compute.manager [instance: f2dfce55-94cb-43e2-b799-9f9a2671c38b] File "/usr/local/lib/python3.8/dist-packages/six.py", line 703, in reraise 2021-04-29 11:19:00.774 6 ERROR nova.compute.manager [instance: f2dfce55-94cb-43e2-b799-9f9a2671c38b] raise value 2021-04-29 11:19:00.774 6 ERROR nova.compute.manager [instance: f2dfce55-94cb-43e2-b799-9f9a2671c38b] File "/var/lib/kolla/venv/lib/python3.8/site-packages/eventlet/tpool.py", line 83, in tworker 2021-04-29 11:19:00.774 6 ERROR nova.compute.manager [instance: f2dfce55-94cb-43e2-b799-9f9a2671c38b] rv = meth(*args, **kwargs) 2021-04-29 11:19:00.774 6 ERROR nova.compute.manager [instance: f2dfce55-94cb-43e2-b799-9f9a2671c38b] File "/usr/lib/python3/dist-packages/libvirt.py", line 1265, in createWithFlags 2021-04-29 11:19:00.774 6 ERROR nova.compute.manager [instance: f2dfce55-94cb-43e2-b799-9f9a2671c38b] if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self) 2021-04-29 11:19:00.774 6 ERROR nova.compute.manager [instance: f2dfce55-94cb-43e2-b799-9f9a2671c38b] libvirt.libvirtError: internal error: process exited while connecting to monitor: 2021-04-29T11:19:00.527948Z qemu-system-x86_64: -blockdev {"driver":"rbd","pool":"vms","image":"f2dfce55-94cb-43e2-b799-9f9a2671c38b_disk","server":[{"host":"10.0.1.81","port":"6789"},{"host":"10.0.1.82","port":"6789"},{"host":"10.0.1.83","port":"6789"}],"user":"nova","auth-client-required":["cephx","none"],"key-secret":"libvirt-1-storage-secret0","node-name":"libvirt-1-storage","cache":{"direct":false,"no-flush":false},"auto-read-only":true,"discard":"unmap"}: error reading header from f2dfce55-94cb-43e2-b799-9f9a2671c38b_disk: No such file or directory 2021-04-29 11:19:00.774 6 ERROR nova.compute.manager [instance: f2dfce55-94cb-43e2-b799-9f9a2671c38b] To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1926601/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp