** Summary changed: - (libvirt) KeyError updating resources for some node, guest.uuid is not in BDM list + [SRU] (libvirt) KeyError updating resources for some node, guest.uuid is not in BDM list
** Description changed: + [Impact] + + There currently exists a race condition whereby the compute + resource_tracker periodic task polls extant instances and checks their + BDMs which can occur prior to any mappings having yet been created e.g. + root disk mapping for new instances. This patch ensures that instances + without any BDMs are skipped. + + [Test Case] + * deploy Openstack Mitaka with debug logging enabled (not essential but helps) + + * create an instance + + * delete its BDMs - pastebin.ubuntu.com/24287419/ + + * watch /var/log/nova/nova-compute.log on hypervisor hosting instance + and wait for next resource_tracker tick + + * ensure that exception mentioned in LP does not occur (happens after + "Auditing locally available compute resources for node") + + [Regression Potential] + + The resource tracker information is used by the scheduler when deciding + which compute hosts are able to have an instances scheduled to them. In + this case the resource tracker would be skipping instances that would + contribute to disk overcommit ratios. As such it is possible that that + scheduler will have momentarily skewed information about resource + consumption on that compute host until the next resource_tracker tick. + Since the likelihood of this race condition occurring is hopefully slim + and provided that users have a reasonable frequency for the + resource_tracker, the likelihood of this becoming a long term problem is + low since the issue will always be corrected by a subsequent tick + (although if the compute host in question were saturated that would not + be fixed until an instances was deleted or migrated). + + -------- + 2016-07-12 09:54:36.021 10056 ERROR nova.compute.manager [req-d5d5d486-b488-4429-bbb5-24c9f19ff2c0 - - - - -] Error updating resources for node controller. 2016-07-12 09:54:36.021 10056 ERROR nova.compute.manager Traceback (most recent call last): 2016-07-12 09:54:36.021 10056 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 6726, in update_available_resource 2016-07-12 09:54:36.021 10056 ERROR nova.compute.manager rt.update_available_resource(context) 2016-07-12 09:54:36.021 10056 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/compute/resource_tracker.py", line 500, in update_available_resource 2016-07-12 09:54:36.021 10056 ERROR nova.compute.manager resources = self.driver.get_available_resource(self.nodename) 2016-07-12 09:54:36.021 10056 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 5728, in get_available_resource 2016-07-12 09:54:36.021 10056 ERROR nova.compute.manager disk_over_committed = self._get_disk_over_committed_size_total() 2016-07-12 09:54:36.021 10056 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 7397, in _get_disk_over_committed_size_total 2016-07-12 09:54:36.021 10056 ERROR nova.compute.manager local_instances[guest.uuid], bdms[guest.uuid]) 2016-07-12 09:54:36.021 10056 ERROR nova.compute.manager KeyError: '0a5c5743-9555-4dfd-b26e-198449ebeee5' 2016-07-12 09:54:36.021 10056 ERROR nova.compute.manager ** Patch added: "lp1602057-xenial-mitaka.debdiff" https://bugs.launchpad.net/nova/+bug/1602057/+attachment/4851346/+files/lp1602057-xenial-mitaka.debdiff ** Tags added: sts-sponsor -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1602057 Title: [SRU] (libvirt) KeyError updating resources for some node, guest.uuid is not in BDM list To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1602057/+subscriptions -- ubuntu-bugs mailing list [email protected] https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
