Reviewed: https://review.openstack.org/527228 Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=4f61f9be1704d3ed51249a10360b8c48e4cd53ff Submitter: Zuul Branch: master
commit 4f61f9be1704d3ed51249a10360b8c48e4cd53ff Author: Matt Riedemann <mriedem...@gmail.com> Date: Mon Dec 11 15:33:43 2017 -0500 Update and complete volume attachments during resize With the new cinder volume attachment flow, during a resize the source compute will create a new volume attachment for any BDMs connected to the instance and delete the existing volume attachments which represents the connection to the source host. The finish_resize flow on the destination compute will then refresh the connection info on the BDMs it's working with before passing them to the virt driver. Since the source compute updated the BDM.attachment_id to point at the new attachment which is meant for the destination host, the refresh_connection_info call will get the connection_info for the new reserved attachment, which is actually empty since it hasn't been connected to the destination host yet. This results in wiping out the BDM.connection_info which has the "driver_volume_type" value which is something the virt driver on the destination compute needs to know which volume backend driver to use to connect the volume to the destination host. This change updates the volume attachments on the destination host before refreshing the connection_info in the BDMs and before we call the driver to finish the resize, and then we also complete the volume attachments once the driver has successfully finished the resize. This is similar to what would happen via _prep_block_device during the initial instance create, but since the code paths into the driver are different we have to handle this explicitly in the compute manager. Similarly, we have to perform the same incantation when reverting a resize and going back to the original source host. Closes-Bug: #1737599 Change-Id: Ifc80d07d94311534fd9e7824ede9d09223a011c2 ** Changed in: nova Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1737599 Title: Instance resize with new-style attach volume fails Status in OpenStack Compute (nova): Fix Released Bug description: The Trove gates are failing when attempting to resize an instance what has an ephemeral disk and an attached volume. The stack when it fails is this: Dec 11 03:03:28.751318 ubuntu-xenial-rax-dfw-0001351106 nova-compute[28059]: ERROR nova.compute.manager [None req-11c69857-4556-4d83-b34c-1a0191175ceb alt_demo alt_demo] [instance: 85cdb482-63a5-487a-b103-95b9383ffcc7] Setting instance vm_state to ERROR: VolumeDriverNotFound: Could not find a handler for None volume. Dec 11 03:03:28.751537 ubuntu-xenial-rax-dfw-0001351106 nova-compute[28059]: ERROR nova.compute.manager [instance: 85cdb482-63a5-487a-b103-95b9383ffcc7] Traceback (most recent call last): Dec 11 03:03:28.751683 ubuntu-xenial-rax-dfw-0001351106 nova-compute[28059]: ERROR nova.compute.manager [instance: 85cdb482-63a5-487a-b103-95b9383ffcc7] File "/opt/stack/new/nova/nova/compute/manager.py", line 7297, in _error_out_instance_on_exception Dec 11 03:03:28.751831 ubuntu-xenial-rax-dfw-0001351106 nova-compute[28059]: ERROR nova.compute.manager [instance: 85cdb482-63a5-487a-b103-95b9383ffcc7] yield Dec 11 03:03:28.751970 ubuntu-xenial-rax-dfw-0001351106 nova-compute[28059]: ERROR nova.compute.manager [instance: 85cdb482-63a5-487a-b103-95b9383ffcc7] File "/opt/stack/new/nova/nova/compute/manager.py", line 4358, in finish_resize Dec 11 03:03:28.752120 ubuntu-xenial-rax-dfw-0001351106 nova-compute[28059]: ERROR nova.compute.manager [instance: 85cdb482-63a5-487a-b103-95b9383ffcc7] disk_info, image_meta, bdms) Dec 11 03:03:28.752261 ubuntu-xenial-rax-dfw-0001351106 nova-compute[28059]: ERROR nova.compute.manager [instance: 85cdb482-63a5-487a-b103-95b9383ffcc7] File "/opt/stack/new/nova/nova/compute/manager.py", line 4326, in _finish_resize Dec 11 03:03:28.752408 ubuntu-xenial-rax-dfw-0001351106 nova-compute[28059]: ERROR nova.compute.manager [instance: 85cdb482-63a5-487a-b103-95b9383ffcc7] old_instance_type) Dec 11 03:03:28.752545 ubuntu-xenial-rax-dfw-0001351106 nova-compute[28059]: ERROR nova.compute.manager [instance: 85cdb482-63a5-487a-b103-95b9383ffcc7] File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ Dec 11 03:03:28.752680 ubuntu-xenial-rax-dfw-0001351106 nova-compute[28059]: ERROR nova.compute.manager [instance: 85cdb482-63a5-487a-b103-95b9383ffcc7] self.force_reraise() Dec 11 03:03:28.752813 ubuntu-xenial-rax-dfw-0001351106 nova-compute[28059]: ERROR nova.compute.manager [instance: 85cdb482-63a5-487a-b103-95b9383ffcc7] File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise Dec 11 03:03:28.752947 ubuntu-xenial-rax-dfw-0001351106 nova-compute[28059]: ERROR nova.compute.manager [instance: 85cdb482-63a5-487a-b103-95b9383ffcc7] six.reraise(self.type_, self.value, self.tb) Dec 11 03:03:28.753092 ubuntu-xenial-rax-dfw-0001351106 nova-compute[28059]: ERROR nova.compute.manager [instance: 85cdb482-63a5-487a-b103-95b9383ffcc7] File "/opt/stack/new/nova/nova/compute/manager.py", line 4321, in _finish_resize Dec 11 03:03:28.753229 ubuntu-xenial-rax-dfw-0001351106 nova-compute[28059]: ERROR nova.compute.manager [instance: 85cdb482-63a5-487a-b103-95b9383ffcc7] block_device_info, power_on) Dec 11 03:03:28.753363 ubuntu-xenial-rax-dfw-0001351106 nova-compute[28059]: ERROR nova.compute.manager [instance: 85cdb482-63a5-487a-b103-95b9383ffcc7] File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 7640, in finish_migration Dec 11 03:03:28.753495 ubuntu-xenial-rax-dfw-0001351106 nova-compute[28059]: ERROR nova.compute.manager [instance: 85cdb482-63a5-487a-b103-95b9383ffcc7] block_device_info=block_device_info) Dec 11 03:03:28.753628 ubuntu-xenial-rax-dfw-0001351106 nova-compute[28059]: ERROR nova.compute.manager [instance: 85cdb482-63a5-487a-b103-95b9383ffcc7] File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 5071, in _get_guest_xml Dec 11 03:03:28.753763 ubuntu-xenial-rax-dfw-0001351106 nova-compute[28059]: ERROR nova.compute.manager [instance: 85cdb482-63a5-487a-b103-95b9383ffcc7] context) Dec 11 03:03:28.753897 ubuntu-xenial-rax-dfw-0001351106 nova-compute[28059]: ERROR nova.compute.manager [instance: 85cdb482-63a5-487a-b103-95b9383ffcc7] File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 4879, in _get_guest_config Dec 11 03:03:28.754035 ubuntu-xenial-rax-dfw-0001351106 nova-compute[28059]: ERROR nova.compute.manager [instance: 85cdb482-63a5-487a-b103-95b9383ffcc7] flavor, guest.os_type) Dec 11 03:03:28.754173 ubuntu-xenial-rax-dfw-0001351106 nova-compute[28059]: ERROR nova.compute.manager [instance: 85cdb482-63a5-487a-b103-95b9383ffcc7] File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 3792, in _get_guest_storage_config Dec 11 03:03:28.754303 ubuntu-xenial-rax-dfw-0001351106 nova-compute[28059]: ERROR nova.compute.manager [instance: 85cdb482-63a5-487a-b103-95b9383ffcc7] self._connect_volume(connection_info, info, instance) Dec 11 03:03:28.754429 ubuntu-xenial-rax-dfw-0001351106 nova-compute[28059]: ERROR nova.compute.manager [instance: 85cdb482-63a5-487a-b103-95b9383ffcc7] File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 1180, in _connect_volume Dec 11 03:03:28.754597 ubuntu-xenial-rax-dfw-0001351106 nova-compute[28059]: ERROR nova.compute.manager [instance: 85cdb482-63a5-487a-b103-95b9383ffcc7] vol_driver = self._get_volume_driver(connection_info) Dec 11 03:03:28.754730 ubuntu-xenial-rax-dfw-0001351106 nova-compute[28059]: ERROR nova.compute.manager [instance: 85cdb482-63a5-487a-b103-95b9383ffcc7] File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 1176, in _get_volume_driver Dec 11 03:03:28.754873 ubuntu-xenial-rax-dfw-0001351106 nova-compute[28059]: ERROR nova.compute.manager [instance: 85cdb482-63a5-487a-b103-95b9383ffcc7] raise exception.VolumeDriverNotFound(driver_type=driver_type) Dec 11 03:03:28.755007 ubuntu-xenial-rax-dfw-0001351106 nova-compute[28059]: ERROR nova.compute.manager [instance: 85cdb482-63a5-487a-b103-95b9383ffcc7] VolumeDriverNotFound: Could not find a handler for None volume. Dec 11 03:03:28.755150 ubuntu-xenial-rax-dfw-0001351106 nova-compute[28059]: ERROR nova.compute.manager [instance: 85cdb482-63a5-487a-b103-95b9383ffcc7] A link to one of the gate's Nova logs is: http://logs.openstack.org/39/524539/1/check/legacy-trove-functional-dsvm-mysql/3ef681d/logs/screen-n-cpu.txt.gz To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1737599/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp