** Changed in: libvirt (Ubuntu)
Status: Confirmed => Fix Released
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1295876
Title:
libvirtError: internal error unable to add domain xxx to cgroup: No
space left on device
Status in OpenStack Compute (Nova):
Invalid
Status in libvirt package in Ubuntu:
Fix Released
Bug description:
logstash query: message:"cgroup\: No space left on device" AND
filename:logs*screen-n-cpu.txt
http://logs.openstack.org/12/80412/8/check/check-tempest-dsvm-
postgres-
full/f9f6158/logs/screen-n-cpu.txt.gz?level=TRACE#_2014-03-21_17_45_12_490
ERROR nova.compute.manager [req-630b71d6-0fbe-4e9e-99fe-019da7d29a3a
FixedIPsNegativeTestJson-475659359 FixedIPsNegativeTestJson-265680949]
[instance: 3f281136-ed69-4bfb-bf36-a7d4aa1c0640] Error: internal error unable
to add domain instance-00000002 task 3057 to cgroup: No space left on device
26028 TRACE nova.compute.manager [instance:
3f281136-ed69-4bfb-bf36-a7d4aa1c0640] Traceback (most recent call last):
26028 TRACE nova.compute.manager [instance:
3f281136-ed69-4bfb-bf36-a7d4aa1c0640] File
"/opt/stack/new/nova/nova/compute/manager.py", line 1304, in _build_instance
26028 TRACE nova.compute.manager [instance:
3f281136-ed69-4bfb-bf36-a7d4aa1c0640] set_access_ip=set_access_ip)
26028 TRACE nova.compute.manager [instance:
3f281136-ed69-4bfb-bf36-a7d4aa1c0640] File
"/opt/stack/new/nova/nova/compute/manager.py", line 394, in decorated_function
26028 TRACE nova.compute.manager [instance:
3f281136-ed69-4bfb-bf36-a7d4aa1c0640] return function(self, context, *args,
**kwargs)
26028 TRACE nova.compute.manager [instance:
3f281136-ed69-4bfb-bf36-a7d4aa1c0640] File
"/opt/stack/new/nova/nova/compute/manager.py", line 1716, in _spawn
26028 TRACE nova.compute.manager [instance:
3f281136-ed69-4bfb-bf36-a7d4aa1c0640] LOG.exception(_('Instance failed to
spawn'), instance=instance)
26028 TRACE nova.compute.manager [instance:
3f281136-ed69-4bfb-bf36-a7d4aa1c0640] File
"/opt/stack/new/nova/nova/openstack/common/excutils.py", line 68, in __exit__
26028 TRACE nova.compute.manager [instance:
3f281136-ed69-4bfb-bf36-a7d4aa1c0640] six.reraise(self.type_, self.value,
self.tb)
26028 TRACE nova.compute.manager [instance:
3f281136-ed69-4bfb-bf36-a7d4aa1c0640] File
"/opt/stack/new/nova/nova/compute/manager.py", line 1713, in _spawn
26028 TRACE nova.compute.manager [instance:
3f281136-ed69-4bfb-bf36-a7d4aa1c0640] block_device_info)
26028 TRACE nova.compute.manager [instance:
3f281136-ed69-4bfb-bf36-a7d4aa1c0640] File
"/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 2241, in spawn
26028 TRACE nova.compute.manager [instance:
3f281136-ed69-4bfb-bf36-a7d4aa1c0640] block_device_info)
26028 TRACE nova.compute.manager [instance:
3f281136-ed69-4bfb-bf36-a7d4aa1c0640] File
"/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 3621, in
_create_domain_and_network
26028 TRACE nova.compute.manager [instance:
3f281136-ed69-4bfb-bf36-a7d4aa1c0640] power_on=power_on)
26028 TRACE nova.compute.manager [instance:
3f281136-ed69-4bfb-bf36-a7d4aa1c0640] File
"/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 3531, in _create_domain
26028 TRACE nova.compute.manager [instance:
3f281136-ed69-4bfb-bf36-a7d4aa1c0640] domain.XMLDesc(0))
26028 TRACE nova.compute.manager [instance:
3f281136-ed69-4bfb-bf36-a7d4aa1c0640] File
"/opt/stack/new/nova/nova/openstack/common/excutils.py", line 68, in __exit__
26028 TRACE nova.compute.manager [instance:
3f281136-ed69-4bfb-bf36-a7d4aa1c0640] six.reraise(self.type_, self.value,
self.tb)
26028 TRACE nova.compute.manager [instance:
3f281136-ed69-4bfb-bf36-a7d4aa1c0640] File
"/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 3526, in _create_domain
26028 TRACE nova.compute.manager [instance:
3f281136-ed69-4bfb-bf36-a7d4aa1c0640] domain.createWithFlags(launch_flags)
26028 TRACE nova.compute.manager [instance:
3f281136-ed69-4bfb-bf36-a7d4aa1c0640] File
"/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 179, in doit
26028 TRACE nova.compute.manager [instance:
3f281136-ed69-4bfb-bf36-a7d4aa1c0640] result = proxy_call(self._autowrap,
f, *args, **kwargs)
26028 TRACE nova.compute.manager [instance:
3f281136-ed69-4bfb-bf36-a7d4aa1c0640] File
"/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 139, in
proxy_call
26028 TRACE nova.compute.manager [instance:
3f281136-ed69-4bfb-bf36-a7d4aa1c0640] rv = execute(f,*args,**kwargs)
26028 TRACE nova.compute.manager [instance:
3f281136-ed69-4bfb-bf36-a7d4aa1c0640] File
"/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 77, in tworker
26028 TRACE nova.compute.manager [instance:
3f281136-ed69-4bfb-bf36-a7d4aa1c0640] rv = meth(*args,**kwargs)
26028 TRACE nova.compute.manager [instance:
3f281136-ed69-4bfb-bf36-a7d4aa1c0640] File
"/usr/lib/python2.7/dist-packages/libvirt.py", line 581, in createWithFlags
26028 TRACE nova.compute.manager [instance:
3f281136-ed69-4bfb-bf36-a7d4aa1c0640] if ret == -1: raise libvirtError
('virDomainCreateWithFlags() failed', dom=self)
26028 TRACE nova.compute.manager [instance:
3f281136-ed69-4bfb-bf36-a7d4aa1c0640] libvirtError: internal error unable to
add domain instance-00000002 task 3057 to cgroup: No space left on device
26028 TRACE nova.compute.manager [instance:
3f281136-ed69-4bfb-bf36-a7d4aa1c0640]
26028 TRACE nova.compute.utils [instance:
3f281136-ed69-4bfb-bf36-a7d4aa1c0640] Traceback (most recent call last):
26028 TRACE nova.compute.utils [instance:
3f281136-ed69-4bfb-bf36-a7d4aa1c0640] File
"/opt/stack/new/nova/nova/compute/manager.py", line 1184, in _run_instance
26028 TRACE nova.compute.utils [instance:
3f281136-ed69-4bfb-bf36-a7d4aa1c0640] instance, image_meta,
legacy_bdm_in_spec)
26028 TRACE nova.compute.utils [instance:
3f281136-ed69-4bfb-bf36-a7d4aa1c0640] File
"/opt/stack/new/nova/nova/compute/manager.py", line 1354, in _build_instance
26028 TRACE nova.compute.utils [instance:
3f281136-ed69-4bfb-bf36-a7d4aa1c0640] reason=unicode(exc_info[1]))
26028 TRACE nova.compute.utils [instance:
3f281136-ed69-4bfb-bf36-a7d4aa1c0640] RescheduledException: Build of instance
3f281136-ed69-4bfb-bf36-a7d4aa1c0640 was re-scheduled: internal error unable to
add domain instance-00000002 task 3057 to cgroup: No space left on device
26028 TRACE nova.compute.utils [instance:
3f281136-ed69-4bfb-bf36-a7d4aa1c0640]
This started right around the time we started using:
https://review.openstack.org/#/c/79816/ (serge's libvirt fix for
https://bugs.launchpad.net/nova/+bug/1254872)
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1295876/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : [email protected]
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
