After further troubleshooting I realized that issue is related to barbican. Seems that value key inside vault, received an incorrect luks passphrase(do not know how right now) and when volume is trying to be migrated, it receives a wrong luks passphrase and nova responses with error code which is a correct behavior. Seems that adding a new index to vault was done incorrectly in the past(in ussuri version?) and was fixed in the meantime as it is not visible anymore. Or it's not fixed and a key value can be somehow retrieved "modified" in a future? As I can see that vault key value was not modified from the beginning, seems like it was delivered to nova as a different value. So far I cannot find a relation for that in code, but definitely it's not a nova issue.
** Project changed: nova => barbican -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1996622 Title: Cannot mount old encrypted volume to an instance with Invalid password, cannot unlock any keyslot Status in Barbican: New Bug description: Description =========== After an upgrade of barbican from ussuri to yoga version there is no possibility to attach encrypted volumes created before an upgrade to any instance, because of an error: "libvirt.libvirtError: internal error: unable to execute QEMU command 'blockdev-add': Invalid password, cannot unlock any keyslot". Encrypted volumes created after an upgrade are able to attach to instances, without such error. So far there is no workaround. Tried to detach and attach volume again, tried to convert a volume to an image and back to volume, but no luck. Steps to reproduce ================== 1. Have already created encrypted volume 2. Execute command: openstack server add volume my-new-instance my-old-encrypted-volume 3. Check attachments details by: openstack server show my-new-instance Expected result =============== my-old-encrypted-volume visible in volumes_attached list. Inside VM OS newly attached drive should be visible Actual result ============= my-old-encrypted-volume is not visible in volumes_attached list. During attachment I'm able to see such errors in nova-compute logs: https://paste.openstack.org/show/bNbPOHiQJOq8OsKZ5Gn2/ Barbican logs or cinder logs are not saying anything wrong. What is more, I can correctly retrieve a payload of a key from barbican and secret, which is used for keeping passphrase for a my-old-encrypted-volume, by command: barbican secret get --payload_content_type application/octet-stream secret-id-and-href --file my_symmetric_key.key The same procedure, executed for a freshly created volume is working fine - new encrypted disk is visible inside instance OS. Environment =========== 1. Exact version of OpenStack you are running. See the following # dpkg -l | grep nova ii nova-api 2:21.2.4-0ubuntu1 all OpenStack Compute - API frontend ii nova-common 2:21.2.4-0ubuntu1 all OpenStack Compute - common files ii nova-conductor 2:21.2.4-0ubuntu1 all OpenStack Compute - conductor service ii nova-novncproxy 2:21.2.4-0ubuntu1 all OpenStack Compute - NoVNC proxy ii nova-scheduler 2:21.2.4-0ubuntu1 all OpenStack Compute - virtual machine scheduler ii python3-nova 2:21.2.4-0ubuntu1 all OpenStack Compute Python 3 libraries ii python3-novaclient 2:17.0.0-0ubuntu1 all client library for OpenStack Compute API - 3.x # dpkg -l | grep barbican ii barbican-api 2:14.0.0-0ubuntu1~cloud0 all OpenStack Key Management Service - API Server ii barbican-common 2:14.0.0-0ubuntu1~cloud0 all OpenStack Key Management Service - common files ii barbican-keystone-listener 2:14.0.0-0ubuntu1~cloud0 all OpenStack Key Management Service - Keystone Listener ii barbican-worker 2:14.0.0-0ubuntu1~cloud0 all OpenStack Key Management Service - Worker Node ii python3-barbican 2:14.0.0-0ubuntu1~cloud0 all OpenStack Key Management Service - Python 3 files ii python3-barbicanclient 5.2.0-0ubuntu1~cloud0 all OpenStack Key Management API client - Python 3.x 2. Which hypervisor did you use? Libvirt: # dpkg -l | grep libvirt ii libvirt-daemon 6.0.0-0ubuntu8.16 amd64 Virtualization daemon ii libvirt-daemon-driver-qemu 6.0.0-0ubuntu8.16 amd64 Virtualization daemon QEMU connection driver ii libvirt-daemon-driver-storage-rbd 6.0.0-0ubuntu8.16 amd64 Virtualization daemon RBD storage driver ii libvirt0:amd64 6.0.0-0ubuntu8.16 amd64 library for interfacing with different virtualization systems ii python3-libvirt 6.1.0-1 amd64 libvirt Python 3 bindings 2. Which storage type did you use? iSCSI Huawei dorado 3. Which networking type did you use? Neutron linuxbridge Logs & Configs ============== An error message from nova-compute log: https://paste.openstack.org/show/bNbPOHiQJOq8OsKZ5Gn2/ To manage notifications about this bug go to: https://bugs.launchpad.net/barbican/+bug/1996622/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
