Public bug reported: Some instances are failing to spawn with the following error message:
VMwareDriverException: Cannot complete operation due to concurrent modification by another operation. It's possible this is due to a race condition in the VC driver as this does not happen frequently. This was encountered a few times by the Minesweeper CI. Affected builds include: http://208.91.1.172/logs/81905/6 http://208.91.1.172/logs/80220/3 The Traceback seen in the scheduler log is: Traceback (most recent call last): File "/opt/stack/nova/nova/compute/manager.py", line 1306, in _build_instance set_access_ip=set_access_ip) File "/opt/stack/nova/nova/compute/manager.py", line 394, in decorated_function return function(self, context, *args, **kwargs) File "/opt/stack/nova/nova/compute/manager.py", line 1718, in _spawn LOG.exception(_('Instance failed to spawn'), instance=instance) File "/opt/stack/nova/nova/openstack/common/excutils.py", line 68, in __exit__ six.reraise(self.type_, self.value, self.tb) File "/opt/stack/nova/nova/compute/manager.py", line 1715, in _spawn block_device_info) File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 611, in spawn admin_password, network_info, block_device_info) File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 593, in spawn root_gb_in_kb, linked_clone) File "/opt/stack/nova/nova/virt/vmwareapi/volumeops.py", line 75, in attach_disk_to_vm self._session._wait_for_task(reconfig_task) File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 940, in _wait_for_task ret_val = done.wait() File "/usr/local/lib/python2.7/dist-packages/eventlet/event.py", line 116, in wait return hubs.get_hub().switch() File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 187, in switch return self.greenlet.switch() VMwareDriverException: Cannot complete operation due to concurrent modification by another operation. ** Affects: nova Importance: Undecided Status: New ** Tags: driver vmware -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1296948 Title: VMware: Instance fails to spawn due to "Concurrent modification by another operation" Status in OpenStack Compute (Nova): New Bug description: Some instances are failing to spawn with the following error message: VMwareDriverException: Cannot complete operation due to concurrent modification by another operation. It's possible this is due to a race condition in the VC driver as this does not happen frequently. This was encountered a few times by the Minesweeper CI. Affected builds include: http://208.91.1.172/logs/81905/6 http://208.91.1.172/logs/80220/3 The Traceback seen in the scheduler log is: Traceback (most recent call last): File "/opt/stack/nova/nova/compute/manager.py", line 1306, in _build_instance set_access_ip=set_access_ip) File "/opt/stack/nova/nova/compute/manager.py", line 394, in decorated_function return function(self, context, *args, **kwargs) File "/opt/stack/nova/nova/compute/manager.py", line 1718, in _spawn LOG.exception(_('Instance failed to spawn'), instance=instance) File "/opt/stack/nova/nova/openstack/common/excutils.py", line 68, in __exit__ six.reraise(self.type_, self.value, self.tb) File "/opt/stack/nova/nova/compute/manager.py", line 1715, in _spawn block_device_info) File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 611, in spawn admin_password, network_info, block_device_info) File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 593, in spawn root_gb_in_kb, linked_clone) File "/opt/stack/nova/nova/virt/vmwareapi/volumeops.py", line 75, in attach_disk_to_vm self._session._wait_for_task(reconfig_task) File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 940, in _wait_for_task ret_val = done.wait() File "/usr/local/lib/python2.7/dist-packages/eventlet/event.py", line 116, in wait return hubs.get_hub().switch() File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 187, in switch return self.greenlet.switch() VMwareDriverException: Cannot complete operation due to concurrent modification by another operation. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1296948/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
