** Summary changed:
- Swapping encrypted volumes leaves an encrypted volume attached to the instance
+ Swapping encrypted volumes leaves an decrypted volume attached to the instance
** Changed in: nova
Status: Confirmed => Fix Released
** Changed in: nova
Assignee: (unassigned) => Lee Yarwood (lyarwood)
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1685875
Title:
Swapping encrypted volumes leaves an decrypted volume attached to the
instance
Status in OpenStack Compute (nova):
Fix Released
Bug description:
Description
===========
Swapping encrypted volumes via `nova volume-update` currently results in the
new volume being attached to the instance still encrypted and the original
volume still connected to the host.
Steps to reproduce
==================
# cinder type-create LUKS
# cinder encryption-type-create --cipher aes-xts-plain64 \
--key_size 512 --control_location front-end LUKS \
nova.volume.encryptors.luks.LuksEncryptor
# cinder create --volume-type LUKS 1
# cinder create --volume-type LUKS 1
# nova boot --image cirros-0.3.5-x86_64-disk --flavor 2 test
# nova volume-attach 445a811f-ca80-49d3-b3bd-156cb492c96c \
d7c43bc8-dbc4-4668-950a-a57397af7cbf
# nova volume-update 445a811f-ca80-49d3-b3bd-156cb492c96c \
d7c43bc8-dbc4-4668-950a-a57397af7cbf \
f998a062-8227-41b4-8400-d82a0f223ec3
Expected result
===============
The original volume is disconnected from the host, the new volume is
decrypted and attached to the instance.
Actual result
=============
f998a062, the new volume is now attached to the instance but without
being decrypted :
$ sudo virsh domblklist 445a811f-ca80-49d3-b3bd-156cb492c96c
Target Source
------------------------------------------------
vda
/opt/stack/data/nova/instances/445a811f-ca80-49d3-b3bd-156cb492c96c/disk
vdb
/dev/disk/by-path/ip-192.168.122.224:3260-iscsi-iqn.2010-10.org.openstack:volume-f998a062-8227-41b4-8400-d82a0f223ec3-lun-1
d7c43bc8, the original volumes is still connected and decrypted on the host :
$ iscsiadm -m session
tcp: [5] 192.168.122.224:3260,1
iqn.2010-10.org.openstack:volume-d7c43bc8-dbc4-4668-950a-a57397af7cbf
(non-flash)
tcp: [6] 192.168.122.224:3260,1
iqn.2010-10.org.openstack:volume-f998a062-8227-41b4-8400-d82a0f223ec3
(non-flash)
$ ll
/dev/disk/by-path/ip-192.168.122.224\:3260-iscsi-iqn.2010-10.org.openstack\:volume-d7c43bc8-dbc4-4668-950a-a57397af7cbf-lun-1
lrwxrwxrwx. 1 root root 123 Apr 24 12:39
/dev/disk/by-path/ip-192.168.122.224:3260-iscsi-iqn.2010-10.org.openstack:volume-d7c43bc8-dbc4-4668-950a-a57397af7cbf-lun-1
->
/dev/mapper/crypt-ip-192.168.122.224:3260-iscsi-iqn.2010-10.org.openstack:volume-d7c43bc8-dbc4-4668-950a-a57397af7cbf-lun-1
$ ll /dev/mapper/crypt*
lrwxrwxrwx. 1 root root 7 Apr 24 12:40
/dev/mapper/crypt-ip-192.168.122.224:3260-iscsi-iqn.2010-10.org.openstack:volume-d7c43bc8-dbc4-4668-950a-a57397af7cbf-lun-1
-> ../dm-2
Environment
===========
1. Exact version of OpenStack you are running. See the following
list for all releases: http://docs.openstack.org/releases/
$ git rev-parse HEAD
0039231719d2a02c59e7cd76631e2ff03cc86b0d
2. Which hypervisor did you use?
(For example: Libvirt + KVM, Libvirt + XEN, Hyper-V, PowerKVM, ...)
What's the version of that?
Libvirt + KVM
2. Which storage type did you use?
(For example: Ceph, LVM, GPFS, ...)
What's the version of that?
LVM/iSCSI
3. Which networking type did you use?
(For example: nova-network, Neutron with OpenVSwitch, ...)
N/A
Logs & Configs
==============
n-cpu.log
2017-04-24 12:41:00.740 ERROR nova.compute.manager
[req-95448809-bf98-4468-82cf-f4681678a8a4 admin admin] [instance:
445a811f-ca80-49d3-b3bd-156cb492c96c] Failed to swap volume d7c43bc
8-dbc4-4668-950a-a57397af7cbf for
f998a062-8227-41b4-8400-d82a0f223ec3
2017-04-24 12:41:00.740 TRACE nova.compute.manager [instance:
445a811f-ca80-49d3-b3bd-156cb492c96c] Traceback (most recent call last):
2017-04-24 12:41:00.740 TRACE nova.compute.manager [instance:
445a811f-ca80-49d3-b3bd-156cb492c96c] File
"/opt/stack/nova/nova/compute/manager.py", line 4936, in _swap_volume
2017-04-24 12:41:00.740 TRACE nova.compute.manager [instance:
445a811f-ca80-49d3-b3bd-156cb492c96c] resize_to)
2017-04-24 12:41:00.740 TRACE nova.compute.manager [instance:
445a811f-ca80-49d3-b3bd-156cb492c96c] File
"/opt/stack/nova/nova/virt/libvirt/driver.py", line 1328, in swap_volume
2017-04-24 12:41:00.740 TRACE nova.compute.manager [instance:
445a811f-ca80-49d3-b3bd-156cb492c96c]
self._disconnect_volume(old_connection_info, disk_dev)
2017-04-24 12:41:00.740 TRACE nova.compute.manager [instance:
445a811f-ca80-49d3-b3bd-156cb492c96c] File
"/opt/stack/nova/nova/virt/libvirt/driver.py", line 1176, in _disconnect_volume
2017-04-24 12:41:00.740 TRACE nova.compute.manager [instance:
445a811f-ca80-49d3-b3bd-156cb492c96c]
vol_driver.disconnect_volume(connection_info, disk_dev)
2017-04-24 12:41:00.740 TRACE nova.compute.manager [instance:
445a811f-ca80-49d3-b3bd-156cb492c96c] File
"/opt/stack/nova/nova/virt/libvirt/volume/iscsi.py", line 74, in
disconnect_volume
2017-04-24 12:41:00.740 TRACE nova.compute.manager [instance:
445a811f-ca80-49d3-b3bd-156cb492c96c]
self.connector.disconnect_volume(connection_info['data'], None)
2017-04-24 12:41:00.740 TRACE nova.compute.manager [instance:
445a811f-ca80-49d3-b3bd-156cb492c96c] File
"/usr/lib/python2.7/site-packages/os_brick/utils.py", line 150, in
trace_logging_wrapper
2017-04-24 12:41:00.740 TRACE nova.compute.manager [instance:
445a811f-ca80-49d3-b3bd-156cb492c96c] result = f(*args, **kwargs)
2017-04-24 12:41:00.740 TRACE nova.compute.manager [instance:
445a811f-ca80-49d3-b3bd-156cb492c96c] File
"/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 271, in
inner
2017-04-24 12:41:00.740 TRACE nova.compute.manager [instance:
445a811f-ca80-49d3-b3bd-156cb492c96c] return f(*args, **kwargs)
2017-04-24 12:41:00.740 TRACE nova.compute.manager [instance:
445a811f-ca80-49d3-b3bd-156cb492c96c] File
"/usr/lib/python2.7/site-packages/os_brick/initiator/connectors/iscsi.py", line
502, in disconnect_volume
2017-04-24 12:41:00.740 TRACE nova.compute.manager [instance:
445a811f-ca80-49d3-b3bd-156cb492c96c] self._disconnect_volume_iscsi(props)
2017-04-24 12:41:00.740 TRACE nova.compute.manager [instance:
445a811f-ca80-49d3-b3bd-156cb492c96c] File
"/usr/lib/python2.7/site-packages/os_brick/initiator/connectors/iscsi.py", line
523, in _disconnect_volume_ iscsi
2017-04-24 12:41:00.740 TRACE nova.compute.manager [instance:
445a811f-ca80-49d3-b3bd-156cb492c96c]
self._linuxscsi.wait_for_volume_removal(host_device)
2017-04-24 12:41:00.740 TRACE nova.compute.manager [instance:
445a811f-ca80-49d3-b3bd-156cb492c96c] File
"/usr/lib/python2.7/site-packages/os_brick/utils.py", line 61, in _wrapper
2017-04-24 12:41:00.740 TRACE nova.compute.manager [instance:
445a811f-ca80-49d3-b3bd-156cb492c96c] return r.call(f, *args, **kwargs)
2017-04-24 12:41:00.740 TRACE nova.compute.manager [instance:
445a811f-ca80-49d3-b3bd-156cb492c96c] File
"/usr/lib/python2.7/site-packages/retrying.py", line 212, in call
2017-04-24 12:41:00.740 TRACE nova.compute.manager [instance:
445a811f-ca80-49d3-b3bd-156cb492c96c] raise attempt.get()
2017-04-24 12:41:00.740 TRACE nova.compute.manager [instance:
445a811f-ca80-49d3-b3bd-156cb492c96c] File
"/usr/lib/python2.7/site-packages/retrying.py", line 247, in get
2017-04-24 12:41:00.740 TRACE nova.compute.manager [instance:
445a811f-ca80-49d3-b3bd-156cb492c96c] six.reraise(self.value[0],
self.value[1], self.value[2])
2017-04-24 12:41:00.740 TRACE nova.compute.manager [instance:
445a811f-ca80-49d3-b3bd-156cb492c96c] File
"/usr/lib/python2.7/site-packages/retrying.py", line 200, in call
2017-04-24 12:41:00.740 TRACE nova.compute.manager [instance:
445a811f-ca80-49d3-b3bd-156cb492c96c] attempt = Attempt(fn(*args,
**kwargs), attempt_number, False)
2017-04-24 12:41:00.740 TRACE nova.compute.manager [instance:
445a811f-ca80-49d3-b3bd-156cb492c96c] File
"/usr/lib/python2.7/site-packages/os_brick/initiator/linuxscsi.py", line 81, in
wait_for_volume_removal
2017-04-24 12:41:00.740 TRACE nova.compute.manager [instance:
445a811f-ca80-49d3-b3bd-156cb492c96c] volume_path=volume_path)
2017-04-24 12:41:00.740 TRACE nova.compute.manager [instance:
445a811f-ca80-49d3-b3bd-156cb492c96c] VolumePathNotRemoved: Volume path
/dev/disk/by-path/ip-192.168.122.224:3260-iscsi-iqn.2010-10.org.openstack:volume-d7c43bc8-dbc4-4668-950a-a57397af7cbf-lun-1
was not removed in time.
2017-04-24 12:41:00.740 TRACE nova.compute.manager [instance:
445a811f-ca80-49d3-b3bd-156cb492c96c]
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1685875/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : [email protected]
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
