Reviewed: https://review.opendev.org/c/openstack/nova/+/815347 Committed: https://opendev.org/openstack/nova/commit/43bca185fe2d00bb70d7486fa6c6a0b9eda1fc17 Submitter: "Zuul (22348)" Branch: master
commit 43bca185fe2d00bb70d7486fa6c6a0b9eda1fc17 Author: Mitya_Eremeev <[email protected]> Date: Thu Nov 11 18:52:11 2021 +0300 Close Glance image if downloading failed. If downloding of Glance image failed we should close iterator of image body. Otherwise Glance is unable to delete the image. Change-Id: I193df2fcbf2588c10be953eb4e9eef4609b6286f Closes-Bug: 1948706 ** Changed in: nova Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1948706 Title: Glance cannot remove image if Nova boots instance from image with incorrect signature. Status in OpenStack Compute (nova): Fix Released Bug description: Description =========== Nova is configured to verify glance images: [glance] verify_glance_signatures=true Glance backend is Ceph. Steps to reproduce ================== 1. create glance image with proper signature 2. update glance image with incorrect signature 3. try to boot instance from the glance image with incorrect signature. Boot fails because Nova checks signature and verification fails. It's correct behavior. barbican_tempest_plugin.tests.scenario.test_image_signing.ImageSigningTest.test_signed_image_upload_boot_failure[compute,id-74f022d6-a6ef-4458-96b7-541deadacf99,image,smoke] ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Captured traceback: ~~~~~~~~~~~~~~~~~~~ Traceback (most recent call last): File "/var/lib/openstack/lib/python3.6/site-packages/tempest/lib/services/image/v2/images_client.py", line 103, in delete_image resp, _ = self.delete(url) File "/var/lib/openstack/lib/python3.6/site-packages/tempest/lib/common/rest_client.py", line 330, in delete return self.request('DELETE', url, extra_headers, headers, body) File "/var/lib/openstack/lib/python3.6/site-packages/tempest/lib/common/rest_client.py", line 710, in request self._error_checker(resp, resp_body) File "/var/lib/openstack/lib/python3.6/site-packages/tempest/lib/common/rest_client.py", line 831, in _error_checker raise exceptions.Conflict(resp_body, resp=resp) tempest.lib.exceptions.Conflict: Conflict with state of target resource Details: {'message': 'Image c321f6be-a4d3-42d2-bc3f-f0ea913b83b7 could not be deleted because it is in use: The image cannot be deleted because it is in use through the backend store outside of Glance.<br /><br />\n\n\n', 'code': '409 Conflict', 'title': 'Conflict'} 4. Delete the glance image right after failed instance boot. Expected result =============== Glance image was deleted successfully. Actual result ============= Glance cannot be deleted. In Glance backend we see that there are watchers that protect glance image from deletion: # rbd rm --pool images-hdd c321f6be-a4d3-42d2-bc3f-f0ea913b83b7 2021-10-15T13:25:03.862+0000 7f36b98c8700 -1 librbd::image::PreRemoveRequest: 0x562785d77a50 check_image_watchers: image has watchers - not removing Removing image: 0% complete...failed. rbd: error: image still has watchers This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. # rbd status --pool images-hdd c321f6be-a4d3-42d2-bc3f-f0ea913b83b7 Watchers: watcher=10.10.0.89:0/729945307 client.374098 cookie=140684808072160 The behavior is reproduced by tempest test: https://github.com/openstack/barbican-tempest-plugin/blob/master/barbican_tempest_plugin/tests/scenario/test_image_signing.py#L67 Environment =========== 1. Openstack version: Victoria 2. Hypervisor: KVM + libvirt 3. Glance storage: Ceph, Nova storage: local. 4. Networking: Neutron with OVS To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1948706/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
