Reviewed: https://review.openstack.org/215613 Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=2c1b19761b3d960055ced11558dda22d022d77f4 Submitter: Jenkins Branch: master
commit 2c1b19761b3d960055ced11558dda22d022d77f4 Author: Alexis Lee <[email protected]> Date: Fri Aug 21 13:58:06 2015 +0100 Wait for device to be mapped There's a race condition when trying to perform file injection without libguestfs, which causes a fallback to nbd device. Although the kpartx command succeeds, it does so after the code has tested for success, so Nova thinks it failed. Retry a few times to avoid this. Co-Authored-By: Paul Carlton <[email protected]> Change-Id: Ie5c186562475cd56c55520ad7123f47a0130b2a4 Closes-Bug: #1428639 Closes-Bug: #1484586 ** Changed in: nova Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1484586 Title: file injection fails when using fallback method Status in OpenStack Compute (nova): Fix Released Bug description: Trying to perform file injection without libguestfs, i.e. fallback to using nbd. 2015-08-13 13:21:21 43295 DEBUG oslo_concurrency.processutils [req-e70d20d6-f42c-4d3e-8598-4dd1a76e09f5 61e0c3d7a0204bdeb647498c47599bc0 c942c664c4024ce4b5fe2bf8c3a21a3c] Running cmd (subprocess): sudo nova-rootwrap /opt/stack/service/nova-compute/etc/nova/rootwrap.conf qemu-nbd -c /dev/nbd8 //var/lib/nova/instances/e8cb4369-adf8-4e97-ad75-9d181d3c9dac/disk execute /opt/stack/venv/nova-20150805T111039Z/lib/python2.7/site-packages/oslo_concurrency/processutils.py:199 2015-08-13 13:21:21 43295 DEBUG oslo_concurrency.processutils [req-e70d20d6-f42c-4d3e-8598-4dd1a76e09f5 61e0c3d7a0204bdeb647498c47599bc0 c942c664c4024ce4b5fe2bf8c3a21a3c] CMD "sudo nova-rootwrap /opt/stack/service/nova-compute/etc/nova/rootwrap.conf qemu-nbd -c /dev/nbd8 //var/lib/nova/instances/e8cb4369-adf8-4e97-ad75-9d181d3c9dac/disk" returned: 0 in 0.096s execute /opt/stack/venv/nova-20150805T111039Z/lib/python2.7/site-packages/oslo_concurrency/processutils.py:225 2015-08-13 13:21:21 43295 DEBUG oslo_concurrency.lockutils [req-e70d20d6-f42c-4d3e-8598-4dd1a76e09f5 61e0c3d7a0204bdeb647498c47599bc0 c942c664c4024ce4b5fe2bf8c3a21a3c] Lock "nbd-allocation-lock" released by "_inner_get_dev" :: held 0.099s inner /opt/stack/venv/nova-20150805T111039Z/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:456 2015-08-13 13:21:21 43295 DEBUG nova.virt.disk.mount.api [req-e70d20d6-f42c-4d3e-8598-4dd1a76e09f5 61e0c3d7a0204bdeb647498c47599bc0 c942c664c4024ce4b5fe2bf8c3a21a3c] Map dev /dev/nbd8 map_dev /opt/stack/venv/nova-20150805T111039Z/lib/python2.7/site-packages/nova/virt/disk/mount/api.py:140 2015-08-13 13:21:21 43295 DEBUG oslo_concurrency.processutils [req-e70d20d6-f42c-4d3e-8598-4dd1a76e09f5 61e0c3d7a0204bdeb647498c47599bc0 c942c664c4024ce4b5fe2bf8c3a21a3c] Running cmd (subprocess): sudo nova-rootwrap /opt/stack/service/nova-compute/etc/nova/rootwrap.conf kpartx -a /dev/nbd8 execute /opt/stack/venv/nova-20150805T111039Z/lib/python2.7/site-packages/oslo_concurrency/processutils.py:199 2015-08-13 13:21:21 43295 DEBUG oslo_concurrency.processutils [req-e70d20d6-f42c-4d3e-8598-4dd1a76e09f5 61e0c3d7a0204bdeb647498c47599bc0 c942c664c4024ce4b5fe2bf8c3a21a3c] CMD "sudo nova-rootwrap /opt/stack/service/nova-compute/etc/nova/rootwrap.conf kpartx -a /dev/nbd8" returned: 0 in 0.093s execute /opt/stack/venv/nova-20150805T111039Z/lib/python2.7/site-packages/oslo_concurrency/processutils.py:225 2015-08-13 13:21:21 43295 DEBUG nova.virt.disk.mount.api [req-e70d20d6 -f42c-4d3e-8598-4dd1a76e09f5 61e0c3d7a0204bdeb647498c47599bc0 c942c664c4024ce4b5fe2bf8c3a21a3c] Fail to mount, tearing back down do_mount /opt/stack/venv/nova-20150805T111039Z/lib/python2.7/site- packages/nova/virt/disk/mount/api.py:223 Although the kpartx command works the check for file path fails generating an error. Inserting a short sleep before checking for the path seems to work. This issue is obviously timing related and I do not encounter this when running devstack on a libvirt host. However it occurs on some of the baremetal hypervisors in our lab very reliably. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1484586/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
