I've added cinder and os-brick here as AFAICT this isn't a Nova issue given the description in c#0 and logs in c#1.
The attempt by Cinder to retype the detached volume doesn't appear to have encrypted anything. This might be the result of another issue in os-brick when connecting the volume to the underlying host but that isn't obvious from the compute logs. Can you reproduce this with DEBUG logs on the compute so we can see what os-brick is doing? ** Also affects: cinder Importance: Undecided Status: New ** Also affects: os-brick Importance: Undecided Status: New ** Changed in: nova Status: New => Incomplete -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1925818 Title: Retype of non-encrypted root volume to encrypted fails Status in Cinder: New Status in OpenStack Compute (nova): Incomplete Status in os-brick: New Bug description: Description =========== I tried to retype volume from unencrypted to encrypted one. That works fine for non-root volumes, however it doesn't work when volume was/is root volume. Steps to reproduce ================== 1. Create an instance with unencrypted root volume(select do not delete volume on termination) 2. Delete instance 3. Retype left volume from deleted instance to encrypted one and wait till retype will finish successfully 4. Launch an instance from this encrypted volume(increase original size by +2GB) Expected result =============== New instance is started from encrypted volume Actual result ============= Instance fails to start. At horizon we can see an error message: Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nova/conductor/manager.py", line 610, in build_instances filter_properties, instances[0].uuid) File "/usr/lib/python2.7/dist-packages/nova/scheduler/utils.py", line 680, in populate_retry raise exception.MaxRetriesExceeded(reason=msg) MaxRetriesExceeded: Exceeded maximum number of retries. Exceeded max scheduling attempts 3 for instance d19504b4-8c05-423b-abaa-b1b8c0658fa8. Last exception: Unexpected error while running command. Command: cryptsetup --batch-mode luksFormat --key-file=- --cipher aes-xts-plain64 --key-size 256 /dev/dm-19 Exit code: 5 Stdout: u'' Stderr: u'Cannot use device /dev/dm-19 which is in use (already mapped or mounted).\n' Environment =========== 1. Version of cinder: ii cinder-api 2:13.0.9-0ubuntu1~cloud1.1 all Cinder storage service - API server ii cinder-backup 2:13.0.9-0ubuntu1~cloud1.1 all Cinder storage service - Scheduler server ii cinder-common 2:13.0.9-0ubuntu1~cloud1.1 all Cinder storage service - common files ii cinder-scheduler 2:13.0.9-0ubuntu1~cloud1.1 all Cinder storage service - Scheduler server ii cinder-volume 2:13.0.9-0ubuntu1~cloud1.1 all Cinder storage service - Volume server ii python-cinder 2:13.0.9-0ubuntu1~cloud1.1 all Cinder Python 2 libraries ii python-cinderclient 1:3.5.0-0ubuntu1 all Python bindings to the OpenStack Volume API - Python 2.x Version of barbican: ii barbican-api 1:10.0.0-0ubuntu0.20.04.1~cloud0 all OpenStack Key Management Service - API Server ii barbican-common 1:10.0.0-0ubuntu0.20.04.1~cloud0 all OpenStack Key Management Service - common files ii barbican-keystone-listener 1:10.0.0-0ubuntu0.20.04.1~cloud0 all OpenStack Key Management Service - Keystone Listener ii barbican-worker 1:10.0.0-0ubuntu0.20.04.1~cloud0 all OpenStack Key Management Service - Worker Node ii python-barbican 1:6.0.1-0ubuntu1 all OpenStack Key Management Service - Python files ii python3-barbican 1:10.0.0-0ubuntu0.20.04.1~cloud0 all OpenStack Key Management Service - Python 3 files ii python3-barbicanclient 4.10.0-0ubuntu1~cloud0 all OpenStack Key Management API client - Python 3.x Version of nova: ii nova-api 2:18.3.0-0ubuntu1~cloud1 all OpenStack Compute - API frontend ii nova-common 2:18.3.0-0ubuntu1~cloud1 all OpenStack Compute - common files ii nova-conductor 2:18.3.0-0ubuntu1~cloud1 all OpenStack Compute - conductor service ii nova-consoleauth 2:18.3.0-0ubuntu1~cloud1 all OpenStack Compute - Console Authenticator ii nova-novncproxy 2:18.3.0-0ubuntu1~cloud1 all OpenStack Compute - NoVNC proxy ii nova-placement-api 2:18.3.0-0ubuntu1~cloud1 all OpenStack Compute - placement API frontend ii nova-scheduler 2:18.3.0-0ubuntu1~cloud1 all OpenStack Compute - virtual machine scheduler ii python-nova 2:18.3.0-0ubuntu1~cloud1 all OpenStack Compute Python 2 libraries ii python-novaclient 2:11.0.0-0ubuntu1~cloud0 all client library for OpenStack Compute API - Python 2.7 2. We use iSCSI storage(Huawei dorado) Logs & Configs ============== Cinder volume logs: 2021-04-23 16:33:43.288 17298 INFO cinder.volume.drivers.huawei.huawei_driver [req-19f3af30-022a-4c96-ab28-2c3aaa91764c 7d52a2a770e4402982390eb658425083 ab5b748ab5e04fa58c8b4aad9f744e2b - default default] Initialize iscsi connection for volume de2c38b7-bf67-42b7-ae51-76fca6c7c05b, connector info {u'initiator': u'iqn.1993-08.org.debian:01:3a7188e5b55f', u'ip': u'10.10.0.248', u'platform': u'x86_64', u'host': u'comp-c08', u'do_local_attach': False, u'mountpoint': u'/dev/vda', u'os_type': u'linux2', u'multipath': True}. 2021-04-23 16:34:05.883 17298 INFO cinder.volume.drivers.huawei.huawei_driver [req-969c4e48-91a7-4c46-824d-63b924503c34 7d52a2a770e4402982390eb658425083 ab5b748ab5e04fa58c8b4aad9f744e2b - default default] Terminate iscsi connection for volume de2c38b7-bf67-42b7-ae51-76fca6c7c05b, connector info {u'initiator': u'iqn.1993-08.org.debian:01:3a7188e5b55f', u'ip': u'10.10.0.248', u'platform': u'x86_64', u'host': u'comp-c08', u'do_local_attach': False, u'mountpoint': u'/dev/vda', u'os_type': u'linux2', u'multipath': True}. 2021-04-23 16:34:12.452 17298 INFO cinder.volume.drivers.huawei.huawei_driver [req-314b1423-2650-4a06 -9f2e-945bd648e70f 7d52a2a770e4402982390eb658425083 ab5b748ab5e04fa58c8b4aad9f744e2b - default default] Initialize iscsi connection for volume de2c38b7-bf67-42b7-ae51-76fca6c7c05b, connector info {u'initiator': u'iqn.1993-08.org.debian:01:a1edf5552e8', u'ip': u'10.10.0.253', u'platform': u'x86_64', u'host': u'comp-a09', u'do_local_attach': False, u'mountpoint': u'/dev/vda', u'os_type': u'linux2', u'multipath': True}. 2021-04-23 16:34:58.111 17298 INFO cinder.volume.drivers.huawei.huawei_driver [req-77231901-21bf- 42c3-99a3-5a2e0b13a066 7d52a2a770e4402982390eb658425083 ab5b748ab5e04fa58c8b4aad9f744e2b - default default] Terminate iscsi connection for volume de2c38b7-bf67-42b7-ae51-76fca6c7c05b, connector info {u'initiator': u'iqn.1993-08.org.debian:01:36b34c8cbdd8', u'ip': u'10.10.0.148', u'platform': u'x86_64', u'host': u'comp-a03', u'do_local_attach': False, u'mountpoint': u'/dev/vda', u'os_type': u'linux2', u'multipath': True}. In case some additional logs are needed, just let me know. To manage notifications about this bug go to: https://bugs.launchpad.net/cinder/+bug/1925818/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp