[Bug 1336555] Re: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size()
After running with the unoptimized packages for a bit, we hit: https://github.com/openvswitch/ovs/commit/417d7a008b9f280f0e0c603cb6f2871ab75b8d49 some further discussion with upstream would indicate that this is probably the source of the original problem, but the code optimizer was concealing the actual root cause in the code. -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to openvswitch in Ubuntu. https://bugs.launchpad.net/bugs/1336555 Title: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size() To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/openvswitch/+bug/1336555/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1336555] Re: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size()
After running with the unoptimized packages for a bit, we hit: https://github.com/openvswitch/ovs/commit/417d7a008b9f280f0e0c603cb6f2871ab75b8d49 some further discussion with upstream would indicate that this is probably the source of the original problem, but the code optimizer was concealing the actual root cause in the code. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1336555 Title: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size() To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/openvswitch/+bug/1336555/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1336555] Re: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size()
proxied from Joe Stringer@VMware I wonder if this patch fixes the issue: https://github.com/openvswitch/ovs/commit/546953509095cec6fad42663b659171618b765d2 Note that the last three lines within the bad_key_len / bad_mask_len conditional statement are the following: format_generic_odp_key(ma, ds); ds_put_char(ds, ')'); return; This is the same logic as the end of the function, where the backtrace is reporting the callstack to be. Jarno pointed out that the compiler could optimize out the first copy of this code to turn into a jump instruction which jumps inside the if (!is_exact) statement. Hence the backtrace shows this confusing callstack. Note that this problem would only present itself if there is: A) A mismatch between a newer kernel version and an older userspace (OVS2.3), where B) The kernel has a new flow match field available which ovs-vswitchd doesn't understand, and C) A flow_del command fails for some reason. It would be great if we could confirm by getting the existing build of OVS and applying the patch above. ** Description changed: + [Impact] + Open vSwitch daemon crashes, causing flow data to be lost and in an OpenStack cloud, instance connectivity to be lost. + + [Test Case] + trivialized step Install and OpenStack cloud using Neutron + ML2 plugin and OpenvSwitch + Run cloud for some time - ovs-vswitchd will crash causing loss of instance connectivity. + + + [Regression Potential] + Minimal - this code is in versions 2.0.2 for some time. + + [Original Bug Report] Hi I find that every 2 days or so I lose part of my cluster. It seems that openvswitch is crashing... The only message left on syslog is as follows: syslog:Jul 1 22:52:32 blue-compute kernel: [530482.190688] ovs- vswitchd[1935]: segfault at 0 ip 00459110 sp 7fff85804758 error 4 in ovs-vswitchd[40+133000] And this is the last message. I'm unable to reboot gracefully. I have to reset. (This can be because ceph not giving up also). And I can see a lot of traffic going around in the network. There so much traffic that some lowend routers/switches fail. Can be because another problem (machines stalled because the ovs fault and others trying to connect. Maybe it fails because much traffic). But I tell this for completeness. Now some info: Linux version 3.13.0-30-generic (buildd@allspice) (gcc version 4.8.2 (Ubuntu 4.8.2-19ubuntu1) ) #54-Ubu ntu SMP Mon Jun 9 22:45:01 UTC 2014 (Ubuntu 3.13.0-30.54-generic 3.13.11.2) vendor_id : AuthenticAMD cpu family: 16 model : 4 model name: AMD Phenom(tm) II X4 810 Processor Ubuntu 14.04 LTS (server). ovs-vsctl --version ovs-vsctl (Open vSwitch) 2.0.1 Compiled Feb 23 2014 14:42:32 I can attach full logs but I think there's nothing useful because only one line referring the problem. NOTE: restarting ovs does not solve the problem. -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to openvswitch in Ubuntu. https://bugs.launchpad.net/bugs/1336555 Title: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size() To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/openvswitch/+bug/1336555/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1336555] Re: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size()
proxied from Joe Stringer@VMware I wonder if this patch fixes the issue: https://github.com/openvswitch/ovs/commit/546953509095cec6fad42663b659171618b765d2 Note that the last three lines within the bad_key_len / bad_mask_len conditional statement are the following: format_generic_odp_key(ma, ds); ds_put_char(ds, ')'); return; This is the same logic as the end of the function, where the backtrace is reporting the callstack to be. Jarno pointed out that the compiler could optimize out the first copy of this code to turn into a jump instruction which jumps inside the if (!is_exact) statement. Hence the backtrace shows this confusing callstack. Note that this problem would only present itself if there is: A) A mismatch between a newer kernel version and an older userspace (OVS2.3), where B) The kernel has a new flow match field available which ovs-vswitchd doesn't understand, and C) A flow_del command fails for some reason. It would be great if we could confirm by getting the existing build of OVS and applying the patch above. ** Description changed: + [Impact] + Open vSwitch daemon crashes, causing flow data to be lost and in an OpenStack cloud, instance connectivity to be lost. + + [Test Case] + trivialized step Install and OpenStack cloud using Neutron + ML2 plugin and OpenvSwitch + Run cloud for some time - ovs-vswitchd will crash causing loss of instance connectivity. + + + [Regression Potential] + Minimal - this code is in versions 2.0.2 for some time. + + [Original Bug Report] Hi I find that every 2 days or so I lose part of my cluster. It seems that openvswitch is crashing... The only message left on syslog is as follows: syslog:Jul 1 22:52:32 blue-compute kernel: [530482.190688] ovs- vswitchd[1935]: segfault at 0 ip 00459110 sp 7fff85804758 error 4 in ovs-vswitchd[40+133000] And this is the last message. I'm unable to reboot gracefully. I have to reset. (This can be because ceph not giving up also). And I can see a lot of traffic going around in the network. There so much traffic that some lowend routers/switches fail. Can be because another problem (machines stalled because the ovs fault and others trying to connect. Maybe it fails because much traffic). But I tell this for completeness. Now some info: Linux version 3.13.0-30-generic (buildd@allspice) (gcc version 4.8.2 (Ubuntu 4.8.2-19ubuntu1) ) #54-Ubu ntu SMP Mon Jun 9 22:45:01 UTC 2014 (Ubuntu 3.13.0-30.54-generic 3.13.11.2) vendor_id : AuthenticAMD cpu family: 16 model : 4 model name: AMD Phenom(tm) II X4 810 Processor Ubuntu 14.04 LTS (server). ovs-vsctl --version ovs-vsctl (Open vSwitch) 2.0.1 Compiled Feb 23 2014 14:42:32 I can attach full logs but I think there's nothing useful because only one line referring the problem. NOTE: restarting ovs does not solve the problem. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1336555 Title: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size() To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/openvswitch/+bug/1336555/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1336555] Re: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size()
The testing PPA: https://launchpad.net/~james-page/+archive/ubuntu/openvswitch has held this patch for a few weeks now - we're running this in our test cloud and have not seen a recurrence of the problem. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1336555 Title: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size() To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/openvswitch/+bug/1336555/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1336555] Re: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size()
The testing PPA: https://launchpad.net/~james-page/+archive/ubuntu/openvswitch has held this patch for a few weeks now - we're running this in our test cloud and have not seen a recurrence of the problem. -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to openvswitch in Ubuntu. https://bugs.launchpad.net/bugs/1336555 Title: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size() To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/openvswitch/+bug/1336555/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1336555] Re: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size()
OK - so I managed to not disable the optimization in January - so the PPA packages where the same as in the archive. I've now hit the packaging a bit harder with the noopt flag, and the PPA has updates which have unoptimized ovs binaries. Apologies for any confusion caused. -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to openvswitch in Ubuntu. https://bugs.launchpad.net/bugs/1336555 Title: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size() To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/openvswitch/+bug/1336555/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1336555] Re: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size()
OK - so I managed to not disable the optimization in January - so the PPA packages where the same as in the archive. I've now hit the packaging a bit harder with the noopt flag, and the PPA has updates which have unoptimized ovs binaries. Apologies for any confusion caused. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1336555 Title: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size() To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/openvswitch/+bug/1336555/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1336555] Re: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size()
I am testing the unoptimized version of these and will update this bug report with any of our findings. -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to openvswitch in Ubuntu. https://bugs.launchpad.net/bugs/1336555 Title: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size() To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/openvswitch/+bug/1336555/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1336555] Re: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size()
I am testing the unoptimized version of these and will update this bug report with any of our findings. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1336555 Title: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size() To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/openvswitch/+bug/1336555/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1336555] Re: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size()
I also see alot of: 2015-01-08T13:36:07.822Z|80336|dpif|WARN|system@ovs-system: failed to flow_del (No such file or directory) skb_priority(0),in_port(4),skb_mark(0),eth(src=ae:c4:93:ba:85:3e,dst=33:33:ff:5f:c4:32),eth_type(0x86dd),ipv6(src=::,dst=ff02::1:ff5f:c432,label=0,proto=58,tclass=0,hlimit=255,frag=no),icmpv6(type=135,code=0),nd(target=fe80::8412:d1ff:fe5f:c432) 2015-01-08T13:36:08.823Z|80337|dpif|WARN|system@ovs-system: failed to flow_del (No such file or directory) skb_priority(0),in_port(4),skb_mark(0),eth(src=ae:c4:93:ba:85:3e,dst=33:33:ff:5f:c4:32),eth_type(0x86dd),ipv6(src=::,dst=ff02::1:ff5f:c432,label=0,proto=58,tclass=0,hlimit=255,frag=no),icmpv6(type=135,code=0),nd(target=fe80::8412:d1ff:fe5f:c432) 2015-01-08T13:36:09.822Z|80338|dpif|WARN|system@ovs-system: failed to flow_del (No such file or directory) skb_priority(0),in_port(4),skb_mark(0),eth(src=ae:c4:93:ba:85:3e,dst=33:33:ff:5f:c4:32),eth_type(0x86dd),ipv6(src=::,dst=ff02::1:ff5f:c432,label=0,proto=58,tclass=0,hlimit=255,frag=no),icmpv6(type=135,code=0),nd(target=fe80::8412:d1ff:fe5f:c432) 2015-01-08T13:36:10.823Z|80339|dpif|WARN|system@ovs-system: failed to flow_del (No such file or directory) skb_priority(0),in_port(4),skb_mark(0),eth(src=ae:c4:93:ba:85:3e,dst=33:33:ff:5f:c4:32),eth_type(0x86dd),ipv6(src=::,dst=ff02::1:ff5f:c432,label=0,proto=58,tclass=0,hlimit=255,frag=no),icmpv6(type=135,code=0),nd(target=fe80::8412:d1ff:fe5f:c432) 2015-01-08T13:36:11.824Z|80340|dpif|WARN|system@ovs-system: failed to flow_del (No such file or directory) skb_priority(0),in_port(4),skb_mark(0),eth(src=ae:c4:93:ba:85:3e,dst=33:33:ff:5f:c4:32),eth_type(0x86dd),ipv6(src=::,dst=ff02::1:ff5f:c432,label=0,proto=58,tclass=0,hlimit=255,frag=no),icmpv6(type=135,code=0),nd(target=fe80::8412:d1ff:fe5f:c432) which is related to https://github.com/openvswitch/ovs/commit/3601bd879 this is not present in the 3.13 kernel in trusty. Might be 2+2 =5 but who knows. -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to openvswitch in Ubuntu. https://bugs.launchpad.net/bugs/1336555 Title: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size() To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/openvswitch/+bug/1336555/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1336555] Re: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size()
Ubuntu kernel source for reference: http://kernel.ubuntu.com/git?p=ubuntu/ubuntu- trusty.git;a=blob;f=net/openvswitch/datapath.c;h=654e058a6e2775f203c785583343cf6180baaacb;hb=HEAD -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to openvswitch in Ubuntu. https://bugs.launchpad.net/bugs/1336555 Title: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size() To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/openvswitch/+bug/1336555/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1336555] Re: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size()
Ubuntu kernel source for reference: http://kernel.ubuntu.com/git?p=ubuntu/ubuntu- trusty.git;a=blob;f=net/openvswitch/datapath.c;h=654e058a6e2775f203c785583343cf6180baaacb;hb=HEAD -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1336555 Title: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size() To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/openvswitch/+bug/1336555/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1336555] Re: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size()
I also see alot of: 2015-01-08T13:36:07.822Z|80336|dpif|WARN|system@ovs-system: failed to flow_del (No such file or directory) skb_priority(0),in_port(4),skb_mark(0),eth(src=ae:c4:93:ba:85:3e,dst=33:33:ff:5f:c4:32),eth_type(0x86dd),ipv6(src=::,dst=ff02::1:ff5f:c432,label=0,proto=58,tclass=0,hlimit=255,frag=no),icmpv6(type=135,code=0),nd(target=fe80::8412:d1ff:fe5f:c432) 2015-01-08T13:36:08.823Z|80337|dpif|WARN|system@ovs-system: failed to flow_del (No such file or directory) skb_priority(0),in_port(4),skb_mark(0),eth(src=ae:c4:93:ba:85:3e,dst=33:33:ff:5f:c4:32),eth_type(0x86dd),ipv6(src=::,dst=ff02::1:ff5f:c432,label=0,proto=58,tclass=0,hlimit=255,frag=no),icmpv6(type=135,code=0),nd(target=fe80::8412:d1ff:fe5f:c432) 2015-01-08T13:36:09.822Z|80338|dpif|WARN|system@ovs-system: failed to flow_del (No such file or directory) skb_priority(0),in_port(4),skb_mark(0),eth(src=ae:c4:93:ba:85:3e,dst=33:33:ff:5f:c4:32),eth_type(0x86dd),ipv6(src=::,dst=ff02::1:ff5f:c432,label=0,proto=58,tclass=0,hlimit=255,frag=no),icmpv6(type=135,code=0),nd(target=fe80::8412:d1ff:fe5f:c432) 2015-01-08T13:36:10.823Z|80339|dpif|WARN|system@ovs-system: failed to flow_del (No such file or directory) skb_priority(0),in_port(4),skb_mark(0),eth(src=ae:c4:93:ba:85:3e,dst=33:33:ff:5f:c4:32),eth_type(0x86dd),ipv6(src=::,dst=ff02::1:ff5f:c432,label=0,proto=58,tclass=0,hlimit=255,frag=no),icmpv6(type=135,code=0),nd(target=fe80::8412:d1ff:fe5f:c432) 2015-01-08T13:36:11.824Z|80340|dpif|WARN|system@ovs-system: failed to flow_del (No such file or directory) skb_priority(0),in_port(4),skb_mark(0),eth(src=ae:c4:93:ba:85:3e,dst=33:33:ff:5f:c4:32),eth_type(0x86dd),ipv6(src=::,dst=ff02::1:ff5f:c432,label=0,proto=58,tclass=0,hlimit=255,frag=no),icmpv6(type=135,code=0),nd(target=fe80::8412:d1ff:fe5f:c432) which is related to https://github.com/openvswitch/ovs/commit/3601bd879 this is not present in the 3.13 kernel in trusty. Might be 2+2 =5 but who knows. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1336555 Title: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size() To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/openvswitch/+bug/1336555/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1336555] Re: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size()
Yes I use openvswitch for everything. I tried to mix br-utils with openvswitch and gave me no goood results. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1336555 Title: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size() To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/openvswitch/+bug/1336555/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1336555] Re: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size()
Yes I use openvswitch for everything. I tried to mix br-utils with openvswitch and gave me no goood results. -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to openvswitch in Ubuntu. https://bugs.launchpad.net/bugs/1336555 Title: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size() To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/openvswitch/+bug/1336555/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1336555] Re: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size()
Upstream references:
http://openvswitch.org/pipermail/discuss/2014-December/015931.html
http://openvswitch.org/pipermail/discuss/2014-October/015360.html
Plus comment from Ben (upstream developer):
This backtrace doesn't quite add up.
We can see from frames 4 and 3 that we've got a nonnull 'key', which
becomes a nonnull nlattr 'a' in frame 2. Along the same chain, we
have a null 'mask' that becomes a null 'ma'. I often don't trust GDB
to give me correct arguments in backtraces but all of that adds up
nicely so I tend to believe it.
Take a look at the code for format_odp_key_attr(). It always
dereferences 'a' to get its type 'attr':
enum ovs_key_attr attr = nl_attr_type(a);
A few lines later we can see 'is_exact' getting set to true (since
'ma' is NULL):
bool is_exact;
is_exact = ma ? odp_mask_attr_is_exact(ma) : true;
We're evidently hitting the default case in the switch statement given
the line number cited in the backtrace, which runs this code:
case OVS_KEY_ATTR_UNSPEC:
case __OVS_KEY_ATTR_MAX:
default:
format_generic_odp_key(a, ds);
if (!is_exact) {
ds_put_char(ds, '/');
format_generic_odp_key(ma, ds); line 1332
}
break;
but that doesn't make sense--we should never get there, because
is_exact is true. So--WTF?
--
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to openvswitch in Ubuntu.
https://bugs.launchpad.net/bugs/1336555
Title:
ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size()
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/openvswitch/+bug/1336555/+subscriptions
--
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at:
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1336555] Re: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size()
Upstream references:
http://openvswitch.org/pipermail/discuss/2014-December/015931.html
http://openvswitch.org/pipermail/discuss/2014-October/015360.html
Plus comment from Ben (upstream developer):
This backtrace doesn't quite add up.
We can see from frames 4 and 3 that we've got a nonnull 'key', which
becomes a nonnull nlattr 'a' in frame 2. Along the same chain, we
have a null 'mask' that becomes a null 'ma'. I often don't trust GDB
to give me correct arguments in backtraces but all of that adds up
nicely so I tend to believe it.
Take a look at the code for format_odp_key_attr(). It always
dereferences 'a' to get its type 'attr':
enum ovs_key_attr attr = nl_attr_type(a);
A few lines later we can see 'is_exact' getting set to true (since
'ma' is NULL):
bool is_exact;
is_exact = ma ? odp_mask_attr_is_exact(ma) : true;
We're evidently hitting the default case in the switch statement given
the line number cited in the backtrace, which runs this code:
case OVS_KEY_ATTR_UNSPEC:
case __OVS_KEY_ATTR_MAX:
default:
format_generic_odp_key(a, ds);
if (!is_exact) {
ds_put_char(ds, '/');
format_generic_odp_key(ma, ds); line 1332
}
break;
but that doesn't make sense--we should never get there, because
is_exact is true. So--WTF?
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1336555
Title:
ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size()
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/openvswitch/+bug/1336555/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1336555] Re: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size()
@gad Does your installation rely on openvswitch management network connectivity for Ceph RBD access? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1336555 Title: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size() To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/openvswitch/+bug/1336555/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1336555] Re: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size()
@gad Does your installation rely on openvswitch management network connectivity for Ceph RBD access? -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to openvswitch in Ubuntu. https://bugs.launchpad.net/bugs/1336555 Title: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size() To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/openvswitch/+bug/1336555/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1336555] Re: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size()
Its probably that the fix for bug 1352570 improved the situation (i.e. occasional rather than regular failures), but it still appears that this is the same/similar backtrace. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1336555 Title: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size() To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/openvswitch/+bug/1336555/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1336555] Re: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size()
Its probably that the fix for bug 1352570 improved the situation (i.e. occasional rather than regular failures), but it still appears that this is the same/similar backtrace. -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to openvswitch in Ubuntu. https://bugs.launchpad.net/bugs/1336555 Title: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size() To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/openvswitch/+bug/1336555/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1336555] Re: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size()
I've uploaded ovs 2.0.2 packages with the optimizer disabled to: https://launchpad.net/~james-page/+archive/ubuntu/openvswitch It would be helpful is someone who see's this regularly could try with these packages; right now we really need to get a reliable reproducer for this problem, so that we can debug it more effectively. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1336555 Title: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size() To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/openvswitch/+bug/1336555/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1336555] Re: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size()
I've uploaded ovs 2.0.2 packages with the optimizer disabled to: https://launchpad.net/~james-page/+archive/ubuntu/openvswitch It would be helpful is someone who see's this regularly could try with these packages; right now we really need to get a reliable reproducer for this problem, so that we can debug it more effectively. -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to openvswitch in Ubuntu. https://bugs.launchpad.net/bugs/1336555 Title: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size() To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/openvswitch/+bug/1336555/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1336555] Re: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size()
** Tags added: openstack uosci -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1336555 Title: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size() To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/openvswitch/+bug/1336555/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1336555] Re: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size()
** Tags added: cts -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1336555 Title: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size() To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/openvswitch/+bug/1336555/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1336555] Re: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size()
** Tags added: openstack uosci -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to openvswitch in Ubuntu. https://bugs.launchpad.net/bugs/1336555 Title: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size() To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/openvswitch/+bug/1336555/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1336555] Re: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size()
** Tags added: cts -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to openvswitch in Ubuntu. https://bugs.launchpad.net/bugs/1336555 Title: ovs-vswitchd crashed with SIGSEGV in nl_attr_get_size() To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/openvswitch/+bug/1336555/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
