Pravin Shelar <[email protected]> wrote:
>On Fri, Nov 7, 2014 at 2:29 PM, Jesse Gross <[email protected]> wrote:
>> On Fri, Nov 7, 2014 at 1:13 PM, Jay Vosburgh <[email protected]>
>> wrote:
>>> Jesse Gross <[email protected]> wrote:
>>>
>>>>On Fri, Nov 7, 2014 at 10:34 AM, Jesse Gross <[email protected]> wrote:
>>>>> On Fri, Nov 7, 2014 at 9:40 AM, Pravin Shelar <[email protected]> wrote:
>>>>>> On Thu, Nov 6, 2014 at 5:58 PM, Jay Vosburgh
>>>>>> <[email protected]> wrote:
>>> [...]
>>>>>>> I'm not sure if this is an error on the part of the RX / GRO
>>>>>>> processing in assembling the GRO skb, or in how OVS calls skb_segment.
>>>>>>>
>>>>>>
>>>>>> I think this is related skb_segment() issue where it is not able to
>>>>>> handle this type of skb geometry. We need to fix skb-segmentation. I
>>>>>> will investigate it more.
>>>>>
>>>>> One problem that I see is that vxlan_gro_complete() doesn't add
>>>>> SKB_GSO_UDP_TUNNEL to gso_type. This causes us to attempt
>>>>> fragmentation as UDP rather than continuing down to do TCP
>>>>> segmentation. That probably screws up the skb geometry.
>>>>
>>>>I sent out a patch to fix this issue. I'm pretty sure that it is the
>>>>root cause of the originally reported case but I don't have a good way
>>>>to reproduce it so it would be great if you could test it Jay.
>>>
>>> I'm having an issue there; when I set up my recreation on
>>> current net-next (3.18-rc2) without your new patch, I get the following
>>> oops when my ovs script does "ovs-vsctl --if-exists del-br br-ex":
>>
>> Hmm, that looks like a totally different problem. Pravin - any ideas?
>
>I am not able to reproduce with above command. Jay, Can you send me
>steps to reproduce this issue?
Well, at the moment a 3.18.0-rc2 kernel panics in
ovs_flow_tbl_insert as soon as ovs-vswitchd starts up. Booting to an
earlier kernel (3.17, for example) with no other changes doesn't panic.
I moved /etc/openvswitch/conf.db away and rebooted the system
(which I think will eliminate the stored configuration), and the kernel
still hits this oops when ovs-vswitchd starts up.
A bit of poking with crash
[ 22.180002] RIP: 0010:[<ffffffffa01a55ec>] [<ffffffffa01a55ec>]
ovs_flow_tbl_insert+0xdc/0x1f0 [openvswitch]
[ 22.180002] RSP: 0018:ffff8801391a77a8 EFLAGS: 00010203
[ 22.180002] RAX: 00000000076cc6f1 RBX: ffff8800b35c2020 RCX: 00000000fb994f19
[ 22.180002] RDX: 000000009cc907e5 RSI: 00000000a490ff19 RDI: ffff8800b0c1c19c
[ 22.180002] RBP: ffff8801391a77f8 R08: 000000006867223a R09: 00000000819d92a7
[ 22.180002] R10: ffffffffa01a5696 R11: 0000000000000004 R12: ffff8800b35c2028
[ 22.180002] R13: ffff8801391a7850 R14: ffff8800b35c2028 R15: ffff880134827800
[ 22.180002] FS: 00007f0ef491a980(0000) GS:ffff88013fc80000(0000)
knlGS:0000000000000000
0xffffffffa01a55e7 <ovs_flow_tbl_insert+0xd7>: callq 0xffffffff813a75c0
<__jhash2>
0xffffffffa01a55ec <ovs_flow_tbl_insert+0xdc>: mov %eax,0x30(%r8)
0xffffffffa01a55f0 <ovs_flow_tbl_insert+0xe0>: mov (%rbx),%r13
0xffffffffa01a55f3 <ovs_flow_tbl_insert+0xe3>: mov %r8,%rsi
0xffffffffa01a55f6 <ovs_flow_tbl_insert+0xe6>: mov %r13,%rdi
0xffffffffa01a55f9 <ovs_flow_tbl_insert+0xe9>: callq 0xffffffffa01a4ba0
<table_instance_insert>
So it panics on return from __jhash2 because %r8 is invalid;
this is presumably
int ovs_flow_tbl_insert(struct flow_table *table, struct sw_flow *flow,
struct sw_flow_mask *mask)
{
[...]
flow->hash = flow_hash(&flow->key, flow->mask->range.start,
flow->mask->range.end);
ti = ovsl_dereference(table->ti);
table_instance_insert(ti, flow);
Looking at __jash2, it uses %r8 internally, but %r8 doesn't
appear to be saved and restored either around the call to, or within,
__jhash2. In ovs_flow_tbl_insert %r8 appears to hold the "flow"
variable, so this panic might have nothing to do with ovs itself.
I'll have to look at this further on Monday.
-J
---
-Jay Vosburgh, [email protected]
_______________________________________________
discuss mailing list
[email protected]
http://openvswitch.org/mailman/listinfo/discuss
