Looks like one or more of your VMs get stuck and libvirt having problems to
stop it now:
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher if ret ==
-1: raise libvirtError ('virDomainDestroy() failed', dom=self)
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher libvirtError:
Failed to terminate process 8756 with SIGKILL: Device or resource busy
I would recommend you to check what exactly is PID (8756) in the error? I
assume is a qemu process and what's going on with this process?
Does it have some issue with accessing storage or something else? You can also
check the libvirtd logs, there should be something more.
====================================================
Radosław Śmigielski
ALCATEL-LUCENT - Cloudband/Node Team
Office: 238 646 13 or +353 (0)188 646 13
Mobile: +353 86 820 74 23
________________________________________
From: cosmos cosmos [[email protected]]
Sent: 22 May 2014 12:30
To: [email protected]
Subject: [openstack-dev] [sahara] Deleting cluster in sahara horizon, 500
Error occured in nova.
Hello My name is inhye park from samsung SDS.
I am a developer about nova and sahara.
Recently Our system updated to icehouse.
After deleting cluster in sahara, VM occurred 500 error.
But this problem not occur in nova cli command.
Only in sahara this error occurred.
This is error log.
-----------------------------------------------------------------------------------------
2014-05-22 20:06:42.631 25640 ERROR oslo.messaging.rpc.dispatcher [-] Exception
during message handling: Failed to terminate process 8756 with SIGKILL: Device
or resource busy
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher Traceback
(most recent call last):
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 133,
in _dispatch_and_reply
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher
incoming.message))
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 176,
in _dispatch
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher return
self._do_dispatch(endpoint, method, ctxt, args)
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 122,
in _do_dispatch
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher result =
getattr(endpoint, method)(ctxt, **new_args)
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/exception.py", line 88, in wrapped
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher payload)
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68,
in __exit__
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher
six.reraise(self.type_, self.value, self.tb)
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/exception.py", line 71, in wrapped
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher return
f(self, context, *args, **kw)
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 333, in
decorated_function
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher
function(self, context, *args, **kwargs)
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 309, in
decorated_function
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher e,
sys.exc_info())
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68,
in __exit__
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher
six.reraise(self.type_, self.value, self.tb)
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 296, in
decorated_function
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher return
function(self, context, *args, **kwargs)
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2262, in
terminate_instance
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher
do_terminate_instance(instance, bdms)
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/openstack/common/lockutils.py", line
249, in inner
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher return
f(*args, **kwargs)
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2260, in
do_terminate_instance
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher
self._set_instance_error_state(context, instance['uuid'])
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68,
in __exit__
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher
six.reraise(self.type_, self.value, self.tb)
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2250, in
do_terminate_instance
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher
reservations=reservations)
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/hooks.py", line 103, in inner
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher rv =
f(*args, **kwargs)
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2220, in
_delete_instance
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher
user_id=user_id)
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68,
in __exit__
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher
six.reraise(self.type_, self.value, self.tb)
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2190, in
_delete_instance
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher
self._shutdown_instance(context, db_inst, bdms)
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2125, in
_shutdown_instance
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher
requested_networks)
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68,
in __exit__
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher
six.reraise(self.type_, self.value, self.tb)
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2115, in
_shutdown_instance
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher
block_device_info)
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 951, in
destroy
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher
self._destroy(instance)
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 908, in
_destroy
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher
instance=instance)
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68,
in __exit__
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher
six.reraise(self.type_, self.value, self.tb)
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 880, in
_destroy
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher
virt_dom.destroy()
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 179, in doit
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher result =
proxy_call(self._autowrap, f, *args, **kwargs)
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 139, in proxy_call
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher rv =
execute(f,*args,**kwargs)
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 77, in tworker
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher rv =
meth(*args,**kwargs)
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/libvirt.py", line 918, in destroy
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher if ret ==
-1: raise libvirtError ('virDomainDestroy() failed', dom=self)
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher libvirtError:
Failed to terminate process 8756 with SIGKILL: Device or resource busy
2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher
2014-05-22 20:06:42.633 25640 ERROR oslo.messaging._drivers.common [-]
Returning exception Failed to terminate process 8756 with SIGKILL: Device or
resource busy to caller
Could you please help about this case?
Thanks.
_______________________________________________
OpenStack-dev mailing list
[email protected]
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
