Reviewed: https://review.openstack.org/467926 Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=92d8103a196c25157295789100117946dcf67304 Submitter: Jenkins Branch: master
commit 92d8103a196c25157295789100117946dcf67304 Author: Huan Xie <[email protected]> Date: Thu May 25 01:21:31 2017 -0700 XenAPI: nova-compute cannot restart after manually delete VM When a VM is deleted accidentally (e.g. hardware problem or manually through the hypervisor rather than Nova), there is a mis-match of information where the VM is still in nova DB but not the hypervisor. If we start nova-compute service in this setup, it will fail due to an untrapped exception when plugging VIFs. Return an expected exception when Nova cannot find VM via xapi. Closes-bug: #1693147 Change-Id: I937f5c202c9a4892e8aa56f74fad125791809f8c ** Changed in: nova Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1693147 Title: XenAPI: nova-compute cannot run when manually delete a VM Status in OpenStack Compute (nova): Fix Released Bug description: I have deployed a development DevStack OS environment with XenServer7.0 and then I did the below steps: 1. Create an instance via Horizon or OpenStack CLI 2. Delete the instance manually (using XenCenter or xe command) 3. Stop nova-compute service and start nova-compute service again Then I cannot start nova-compute service anymore with errors: 2017-05-24 05:54:09.373 DEBUG nova.compute.manager [req-97c683b3-cec8-46e6-bb04-d8f17a0b0be8 None None] [instance: 9d5ee1f9-ad88-48d1-b07e-730154ae8cfd] Checking state from (pid=24627) _get_power_state /opt/stack/nova/nova/compute/manager.py:1169 2017-05-24 05:54:09.380 DEBUG oslo_messaging._drivers.amqpdriver [req-97c683b3-cec8-46e6-bb04-d8f17a0b0be8 None None] CAST unique_id: 499446a84a51459aad4314c2301e7d08 FANOUT topic 'scheduler' from (pid=24627) _send /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py:478 2017-05-24 05:54:09.383 ERROR oslo_service.service [req-97c683b3-cec8-46e6-bb04-d8f17a0b0be8 None None] Error starting thread. 2017-05-24 05:54:09.383 TRACE oslo_service.service Traceback (most recent call last): 2017-05-24 05:54:09.383 TRACE oslo_service.service File "/usr/local/lib/python2.7/dist-packages/oslo_service/service.py", line 721, in run_service 2017-05-24 05:54:09.383 TRACE oslo_service.service service.start() 2017-05-24 05:54:09.383 TRACE oslo_service.service File "/opt/stack/nova/nova/service.py", line 143, in start 2017-05-24 05:54:09.383 TRACE oslo_service.service self.manager.init_host() 2017-05-24 05:54:09.383 TRACE oslo_service.service File "/opt/stack/nova/nova/compute/manager.py", line 1148, in init_host 2017-05-24 05:54:09.383 TRACE oslo_service.service self._init_instance(context, instance) 2017-05-24 05:54:09.383 TRACE oslo_service.service File "/opt/stack/nova/nova/compute/manager.py", line 945, in _init_instance 2017-05-24 05:54:09.383 TRACE oslo_service.service self.driver.plug_vifs(instance, net_info) 2017-05-24 05:54:09.383 TRACE oslo_service.service File "/opt/stack/nova/nova/virt/xenapi/driver.py", line 309, in plug_vifs 2017-05-24 05:54:09.383 TRACE oslo_service.service self._vmops.plug_vifs(instance, network_info) 2017-05-24 05:54:09.383 TRACE oslo_service.service File "/opt/stack/nova/nova/virt/xenapi/vmops.py", line 1961, in plug_vifs 2017-05-24 05:54:09.383 TRACE oslo_service.service self.vif_driver.plug(instance, vif, device=device) 2017-05-24 05:54:09.383 TRACE oslo_service.service File "/opt/stack/nova/nova/virt/xenapi/vif.py", line 251, in plug 2017-05-24 05:54:09.383 TRACE oslo_service.service vif_ref = self._get_vif_ref(vif, vm_ref) 2017-05-24 05:54:09.383 TRACE oslo_service.service File "/opt/stack/nova/nova/virt/xenapi/vif.py", line 43, in _get_vif_ref 2017-05-24 05:54:09.383 TRACE oslo_service.service vif_refs = self._session.call_xenapi("VM.get_VIFs", vm_ref) 2017-05-24 05:54:09.383 TRACE oslo_service.service File "/opt/stack/os-xenapi/os_xenapi/client/session.py", line 200, in call_xenapi 2017-05-24 05:54:09.383 TRACE oslo_service.service return session.xenapi_request(method, args) 2017-05-24 05:54:09.383 TRACE oslo_service.service File "/opt/stack/os-xenapi/os_xenapi/client/XenAPI.py", line 130, in xenapi_request 2017-05-24 05:54:09.383 TRACE oslo_service.service result = _parse_result(getattr(self, methodname)(*full_params)) 2017-05-24 05:54:09.383 TRACE oslo_service.service File "/usr/lib/python2.7/xmlrpclib.py", line 1243, in _call_ 2017-05-24 05:54:09.383 TRACE oslo_service.service return self._send(self._name, args) 2017-05-24 05:54:09.383 TRACE oslo_service.service File "/usr/lib/python2.7/xmlrpclib.py", line 1596, in __request 2017-05-24 05:54:09.383 TRACE oslo_service.service allow_none=self.__allow_none) 2017-05-24 05:54:09.383 TRACE oslo_service.service File "/usr/lib/python2.7/xmlrpclib.py", line 1094, in dumps 2017-05-24 05:54:09.383 TRACE oslo_service.service data = m.dumps(params) 2017-05-24 05:54:09.383 TRACE oslo_service.service File "/usr/lib/python2.7/xmlrpclib.py", line 638, in dumps 2017-05-24 05:54:09.383 TRACE oslo_service.service dump(v, write) 2017-05-24 05:54:09.383 TRACE oslo_service.service File "/usr/lib/python2.7/xmlrpclib.py", line 660, in __dump 2017-05-24 05:54:09.383 TRACE oslo_service.service f(self, value, write) 2017-05-24 05:54:09.383 TRACE oslo_service.service File "/usr/lib/python2.7/xmlrpclib.py", line 664, in dump_nil 2017-05-24 05:54:09.383 TRACE oslo_service.service raise TypeError, "cannot marshal None unless allow_none is enabled" 2017-05-24 05:54:09.383 TRACE oslo_service.service TypeError: cannot marshal None unless allow_none is enabled 2017-05-24 05:54:09.383 TRACE oslo_service.service 2017-05-24 05:54:09.495 DEBUG oslo_concurrency.lockutils [req-774ec28a-fdb4-4742-bcbe-f97a4769652a None None] Acquired semaphore "singleton_lock" from (pid=24627) lock /usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:212 Note: When I delete the VM manually, OpenStack didn't notice it which is fine, but this should not block nova-compute service, nova-compute service should be able to run and provide services although there are VMs which cannot initialized during its startup process To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1693147/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
