** Also affects: nova/kilo
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1453666
Title:
libvirt: guestfs api makes nova-compute hang
Status in OpenStack Compute (nova):
Fix Released
Status in OpenStack Compute (nova) kilo series:
New
Bug description:
Latest Kilo code.
In inspect_capabilities() of nova/virt/disk/vfs/guestfs.py, guestfs
api, which is C-extension, will hang nova-compute process when it is
invoked. This problem will result in message queue time out error and
instance booting failure.
And example of this problem is:
2015-05-09 17:07:08.393 4449 DEBUG nova.virt.disk.vfs.api
[req-1f7c1104-2679-43a5-bbcb-f73114ce9103 - - - - -] Using primary VFSGuestFS
instance_for_image /usr/lib/python2.7/site-packages/nova/virt/disk/vfs/api.py:50
2015-05-09 17:08:35.443 4449 DEBUG nova.virt.disk.vfs.guestfs
[req-1f7c1104-2679-43a5-bbcb-f73114ce9103 - - - - -] Setting up appliance for
/var/lib/nova/instances/0517e2a9-469c-43f4-a129-f489fc1c8356/disk qcow2 setup
/usr/lib/python2.7/site-packages/nova/virt/disk/vfs/guestfs.py:169
2015-05-09 17:08:35.457 4449 DEBUG nova.openstack.common.periodic_task
[req-bb78b74b-bed7-450f-bd40-19686aab2c3e - - - - -] Running periodic task
ComputeManager._instance_usage_audit run_periodic_tasks
/usr/lib/python2.7/site-packages/nova/openstack/common/periodic_task.py:219
2015-05-09 17:08:35.461 4449 INFO oslo_messaging._drivers.impl_rabbit
[req-bb78b74b-bed7-450f-bd40-19686aab2c3e - - - - -] Connecting to AMQP server
on 127.0.0.1:5671
2015-05-09 17:08:35.472 4449 ERROR nova.compute.manager [-] Instance failed
network setup after 1 attempt(s)
2015-05-09 17:08:35.472 4449 TRACE nova.compute.manager Traceback (most
recent call last):
2015-05-09 17:08:35.472 4449 TRACE nova.compute.manager File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1783, in
_allocate_network_async
2015-05-09 17:08:35.472 4449 TRACE nova.compute.manager
system_metadata=sys_meta)
2015-05-09 17:08:35.472 4449 TRACE nova.compute.manager File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 739, in
_instance_update
2015-05-09 17:08:35.472 4449 TRACE nova.compute.manager **kwargs)
2015-05-09 17:08:35.472 4449 TRACE nova.compute.manager File
"/usr/lib/python2.7/site-packages/nova/conductor/api.py", line 308, in
instance_update
2015-05-09 17:08:35.472 4449 TRACE nova.compute.manager updates,
'conductor')
2015-05-09 17:08:35.472 4449 TRACE nova.compute.manager File
"/usr/lib/python2.7/site-packages/nova/conductor/rpcapi.py", line 194, in
instance_update
2015-05-09 17:08:35.472 4449 TRACE nova.compute.manager service=service)
2015-05-09 17:08:35.472 4449 TRACE nova.compute.manager File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", line 156, in
call
2015-05-09 17:08:35.472 4449 TRACE nova.compute.manager retry=self.retry)
2015-05-09 17:08:35.472 4449 TRACE nova.compute.manager File
"/usr/lib/python2.7/site-packages/oslo_messaging/transport.py", line 90, in
_send
2015-05-09 17:08:35.472 4449 TRACE nova.compute.manager timeout=timeout,
retry=retry)
2015-05-09 17:08:35.472 4449 TRACE nova.compute.manager File
"/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line
350, in send
2015-05-09 17:08:35.472 4449 TRACE nova.compute.manager retry=retry)
2015-05-09 17:08:35.472 4449 TRACE nova.compute.manager File
"/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line
339, in _send
2015-05-09 17:08:35.472 4449 TRACE nova.compute.manager result =
self._waiter.wait(msg_id, timeout)
2015-05-09 17:08:35.472 4449 TRACE nova.compute.manager File
"/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line
243, in wait
2015-05-09 17:08:35.472 4449 TRACE nova.compute.manager message =
self.waiters.get(msg_id, timeout=timeout)
2015-05-09 17:08:35.472 4449 TRACE nova.compute.manager File
"/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line
149, in get
2015-05-09 17:08:35.472 4449 TRACE nova.compute.manager 'to message ID
%s' % msg_id)
2015-05-09 17:08:35.472 4449 TRACE nova.compute.manager MessagingTimeout:
Timed out waiting for a reply to message ID 8ff07520ea8743c997b5017f6638a0df
2015-05-09 17:08:35.472 4449 TRACE nova.compute.manager
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1453666/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : [email protected]
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
