Public bug reported: Description =========== The following was noticed during a CI run for https://review.openstack.org/#/c/383859/ :
http://logs.openstack.org/09/395709/7/check/gate-tempest-dsvm-full- devstack-plugin-nfs- nv/a4c1057/logs/screen-n-cpu.txt.gz?level=ERROR#_2017-02-07_19_17_41_994 This is due to rebuild calling for two separate detach/disconnects of a volume when using the libvirt virt driver, once in _rebuild_default_impl in the compute layer and a second time in cleanup within the virt driver : https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L2653 - _rebuild_default_impl https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L989 - cleanup In the logs req-e976fee4-51df-4119-b505-5d68f4583186 tracks the rebuild attempt. We see the first attempt to umount succeed here : http://logs.openstack.org/09/395709/7/check/gate-tempest-dsvm-full- devstack-plugin-nfs- nv/a4c1057/logs/screen-n-cpu.txt.gz?level=ALL#_2017-02-07_19_17_39_904 We then see the second attempt here and again an ERROR is logged as we don't find the mount to be in use : http://logs.openstack.org/09/395709/7/check/gate-tempest-dsvm-full- devstack-plugin-nfs- nv/a4c1057/logs/screen-n-cpu.txt.gz?level=ALL#_2017-02-07_19_17_41_993 Steps to reproduce ================== Rebuild an instance with volumes attached Expected result =============== Only one attempt is made to detach and disconnect each volume from the original instance. Actual result ============= Two attempts are made to detach and disconnect each volume from the original instance. Environment =========== 1. Exact version of OpenStack you are running. See the following list for all releases: http://docs.openstack.org/releases/ https://review.openstack.org/#/c/383859/ - but it should reproduce against master. 2. Which hypervisor did you use? (For example: Libvirt + KVM, Libvirt + XEN, Hyper-V, PowerKVM, ...) What's the version of that? Libvirt 2. Which storage type did you use? (For example: Ceph, LVM, GPFS, ...) What's the version of that? n/a 3. Which networking type did you use? (For example: nova-network, Neutron with OpenVSwitch, ...) n/a ** Affects: nova Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1662869 Title: Multiple attempts to detach and disconnect volumes during rebuild Status in OpenStack Compute (nova): New Bug description: Description =========== The following was noticed during a CI run for https://review.openstack.org/#/c/383859/ : http://logs.openstack.org/09/395709/7/check/gate-tempest-dsvm-full- devstack-plugin-nfs- nv/a4c1057/logs/screen-n-cpu.txt.gz?level=ERROR#_2017-02-07_19_17_41_994 This is due to rebuild calling for two separate detach/disconnects of a volume when using the libvirt virt driver, once in _rebuild_default_impl in the compute layer and a second time in cleanup within the virt driver : https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L2653 - _rebuild_default_impl https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L989 - cleanup In the logs req-e976fee4-51df-4119-b505-5d68f4583186 tracks the rebuild attempt. We see the first attempt to umount succeed here : http://logs.openstack.org/09/395709/7/check/gate-tempest-dsvm-full- devstack-plugin-nfs- nv/a4c1057/logs/screen-n-cpu.txt.gz?level=ALL#_2017-02-07_19_17_39_904 We then see the second attempt here and again an ERROR is logged as we don't find the mount to be in use : http://logs.openstack.org/09/395709/7/check/gate-tempest-dsvm-full- devstack-plugin-nfs- nv/a4c1057/logs/screen-n-cpu.txt.gz?level=ALL#_2017-02-07_19_17_41_993 Steps to reproduce ================== Rebuild an instance with volumes attached Expected result =============== Only one attempt is made to detach and disconnect each volume from the original instance. Actual result ============= Two attempts are made to detach and disconnect each volume from the original instance. Environment =========== 1. Exact version of OpenStack you are running. See the following list for all releases: http://docs.openstack.org/releases/ https://review.openstack.org/#/c/383859/ - but it should reproduce against master. 2. Which hypervisor did you use? (For example: Libvirt + KVM, Libvirt + XEN, Hyper-V, PowerKVM, ...) What's the version of that? Libvirt 2. Which storage type did you use? (For example: Ceph, LVM, GPFS, ...) What's the version of that? n/a 3. Which networking type did you use? (For example: nova-network, Neutron with OpenVSwitch, ...) n/a To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1662869/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp