* Salvatore Orlando (sorla...@nicira.com) wrote: > I've found out that several jobs are exhibiting failures like bug 1254890 > [1] and bug 1253896 [2] because openvswitch seem to be crashing the kernel. > The kernel trace reports as offending process usually either > neutron-ns-metadata-proxy or dnsmasq, but [3] seem to clearly point to > ovs-vsctl.
Hmm, that actually shows dnsmasq is the running/exiting process. The ovs-vsctl was run nearly a half-second earlier. Looks like ovs-vsctl successfuly added the tap device (assuming it's for dnsmasq?). And dnsmasq is exiting upon receiving a signal. Shot in the dark, has the neutron path that would end up killing dnsmasq (Dnsmasq::reload_allocations()) changed recently? I didn't see much. > 254 events observed in the previous 6 days show a similar trace in the logs > [4]. That kernel (3.2.0) is over a year old. And there have been some network namespace fixes since then (IIRC, refcounting related). > This means that while this alone won't explain all the failures observed, > it is however potentially one of the prominent root causes. > > >From the logs I have little hints about the kernel running. It seems there > has been no update in the past 7 days, but I can't be sure. > Openvswitch builds are updated periodically. The last build I found not to > trigger failures was the one generated on 2014/01/16 at 01:58:18. > Unfortunately version-wise I always have only 1.4.0, no build number. > > I don't know if this will require getting in touch with ubuntu, or if we > can just prep a different image which an OVS build known to work without > problems. > > Salvatore > > [1] https://bugs.launchpad.net/neutron/+bug/1254890 > [2] https://bugs.launchpad.net/neutron/+bug/1253896 > [3] http://paste.openstack.org/show/61869/ > [4] "kernel BUG at /build/buildd/linux-3.2.0/fs/buffer.c:2917" and > filename:syslog.txt _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev