Ever since we started picking up the 3.13.0-65 kernel in the openstack CI system on 9/29 we've been seeing lots of random failures across 3 branches (juno/kilo/master) with ebtables, details are in:
https://bugs.launchpad.net/nova/+bug/1501558 An example failure: http://logs.openstack.org/62/229962/1/check/gate-tempest-dsvm-full/28bb6b6/logs/screen-n-cpu.txt.gz?level=TRACE#_2015-10-01_17_39_59_1872015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] Traceback (most recent call last): 2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] File "/opt/stack/new/nova/nova/compute/manager.py", line 2155, in _build_resources 2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] yield resources 2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] File "/opt/stack/new/nova/nova/compute/manager.py", line 2009, in _build_and_run_instance 2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] block_device_info=block_device_info) 2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 2444, in spawn 2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] block_device_info=block_device_info) 2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 4516, in _create_domain_and_network 2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] xml, pause=pause, power_on=power_on) 2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 4446, in _create_domain 2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] guest.launch(pause=pause) 2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] File "/opt/stack/new/nova/nova/virt/libvirt/guest.py", line 141, in launch 2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] self._encoded_xml, errors='ignore') 2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 195, in __exit__ 2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] six.reraise(self.type_, self.value, self.tb) 2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] File "/opt/stack/new/nova/nova/virt/libvirt/guest.py", line 136, in launch 2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] return self._domain.createWithFlags(flags) 2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 183, in doit 2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] result = proxy_call(self._autowrap, f, *args, **kwargs) 2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 141, in proxy_call 2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] rv = execute(f, *args, **kwargs) 2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 122, in execute 2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] six.reraise(c, e, tb) 2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 80, in tworker 2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] rv = meth(*args, **kwargs) 2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] File "/usr/local/lib/python2.7/dist-packages/libvirt.py", line 900, in createWithFlags 2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self) 2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] libvirtError: Error while building firewall: Some rules could not be created for interface vnet7: Failure to execute command '$EBT -t nat -A libvirt-J-vnet7 -j J-vnet7-mac' : 'Unable to update the kernel. Two possible causes: 2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] 1. Multiple ebtables programs were executing simultaneously. The ebtables 2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] userspace tool doesn't by default support multiple ebtables programs running 2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] concurrently. The ebtables option --concurrent or a tool like flock can be 2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] used to support concurrent scripts that update the ebtables kernel tables. 2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] 2. The kernel doesn't support a certain ebtables extension, consider 2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] recompiling your kernel or insmod the extension. 2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] .'. 2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] 2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1497184 Title: Kernel-panic with 3.13.0-64.104 generic kernel (BUG at net/core/skbuff:1290) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1497184/+subscriptions -- ubuntu-bugs mailing list [email protected] https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
