Hi,
I have observed the following crash below using the in-kernel packet generator
with skb_clone set to a non-zero value.
The problem manifests when using GRE or VXLAN but not when
no tunneling is configured.
I am using the Ubuntu 2.6.38-8-server kernel.
Where the relevant portion of skbuff.c looks like this:
768 int pskb_expand_head(struct sk_buff *skb, int nhead, int ntail,
769 gfp_t gfp_mask)
770 {
771 int i;
772 u8 *data;
773 int size = nhead + (skb_end_pointer(skb) - skb->head) + ntail;
774 long off;
775 bool fastpath;
776
777 BUG_ON(nhead < 0);
778
779 if (skb_shared(skb))
780 BUG();
781
782 size = SKB_DATA_ALIGN(size);
I am using the following revision of the OVS master branch
commit 279320b8276011df3667478de6750d5a40d0c6eb
Author: Tyler Coumbes <[email protected]>
Date: Mon Nov 14 09:08:03 2011 -0800
rhel: Add ability to enable bridge compatibility mode in /etc/sysconfig/open
With the following patches from the mailing list applied
commit 2588b90b7a10b6450bfc1d30a1149f1b58774212
Author: Justin Pettit <[email protected]>
Date: Tue Oct 18 17:01:58 2011 -0700
[RFC v2] vxlan: Add support for VXLAN-over-IPsec.
commit a971be698dc179803784e726feb2b253abec3592
Author: Ben Pfaff <[email protected]>
Date: Tue Oct 18 17:01:57 2011 -0700
[RFC v2] Implement draft VXLAN L2-over-L3 tunneling protocol.
--- crash log ---
[ 621.746623] ------------[ cut here ]------------
[ 621.748027] kernel BUG at /build/buildd/linux-2.6.38/net/core/skbuff.c:780!
[ 621.791069] invalid opcode: 0000 [#1] SMP
[ 621.834351] last sysfs file:
/sys/devices/pci0000:00/0000:00:03.0/0000:02:00.0/net/eth3/carrier
[ 621.923975] CPU 0
[ 621.924567] Modules linked in: pktgen netconsole configfs ipmi_devintf
ipmi_si ipmi_msghandler openvswitch_mod ip6table_filter ip6_tables btrfs
libcrc32c ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4
xt_state nf_conntrack ipt_REJECT xt_CHECKSUM iptable_mangle xt_tcpudp
iptable_filter ip_tables x_tables kvm_intel bridge stp kvm vesafb deflate
zlib_deflate ctr twofish_generic twofish_x86_64 twofish_common camellia serpent
blowfish cast5 des_generic aesni_intel cryptd aes_x86_64 aes_generic xcbc
rmd160 sha512_generic sha256_generic sha1_generic crypto_null af_key joydev
psmouse ixgbe ahci libahci usbhid igb e1000e hid i7core_edac ghes serio_raw hed
edac_core dca mdio xfs exportfs shpchp mptsas mptscsih mptbase
scsi_transport_sas
[ 622.470544]
[ 622.521074] Pid: 2082, comm: kpktgend_0 Not tainted 2.6.38-8-server
#42-Ubuntu Quanta QSSC-98J_C2 /QSSC-98J_C2
[ 622.627753] RIP: 0010:[<ffffffff814d850e>] [<ffffffff814d850e>]
pskb_expand_head+0x29e/0x2b0
[ 622.735904] RSP: 0018:ffff88040fe85ab0 EFLAGS: 00010202
[ 622.791295] RAX: 0000000000000002 RBX: ffff880410509600 RCX: 0000000000000020
[ 622.847066] RDX: 0000000000000000 RSI: 0000000000000040 RDI: ffff88041582f800
[ 622.902029] RBP: ffff88040fe85b00 R08: 0000000000000000 R09: 00000000a60b2c22
[ 622.956455] R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000040
[ 623.009861] R13: 0000000000000100 R14: 0000000000000000 R15: ffff8804161d6110
[ 623.062033] FS: 0000000000000000(0000) GS:ffff8800bf200000(0000)
knlGS:0000000000000000
[ 623.163174] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 623.214353] CR2: 00007f8a5e8438e0 CR3: 0000000001a03000 CR4: 00000000000006f0
[ 623.265831] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 623.316234] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 623.365129] Process kpktgend_0 (pid: 2082, threadinfo ffff88040fe84000, task
ffff880415c444a0)
[ 623.461998] Stack:
[ 623.509308] ffff88040fe85ac0 00ffffff8105f702 ffff880416a20000
0000000010509600
[ 623.604621] ffff880415e11500 ffff880410509600 ffff880415e11500
ffff88041dc5fcc0
[ 623.699719] 0000000000000000 ffff8804161d6110 ffff88040fe85bd0
ffffffffa0496dc9
[ 623.794975] Call Trace:
[ 623.840650] [<ffffffffa0496dc9>] tnl_send+0x649/0xc80 [openvswitch_mod]
[ 623.887345] [<ffffffff8115469c>] ? __slab_alloc+0xbc/0x390
[ 623.932963] [<ffffffff81038c79>] ? default_spin_lock_flags+0x9/0x10
[ 623.978985] [<ffffffff814d673d>] ? skb_queue_tail+0x4d/0x60
[ 624.024204] [<ffffffffa0497e6e>] vport_send+0x1e/0x60 [openvswitch_mod]
[ 624.069126] [<ffffffffa048e173>] do_output+0x23/0x40 [openvswitch_mod]
[ 624.113323] [<ffffffffa048e85e>] do_execute_actions+0x56e/0x810
[openvswitch_mod]
[ 624.198921] [<ffffffffa048f751>] ? queue_userspace_packet+0x321/0x340
[openvswitch_mod]
[ 624.287175] [<ffffffff815d7cce>] ? common_interrupt+0xe/0x13
[ 624.332881] [<ffffffffa0492ff2>] ? flow_extract+0x202/0xb80
[openvswitch_mod]
[ 624.425541] [<ffffffffa0499a30>] ? internal_dev_xmit+0x0/0x30
[openvswitch_mod]
[ 624.520802] [<ffffffffa048eb6c>] execute_actions+0x6c/0xc0 [openvswitch_mod]
[ 624.570075] [<ffffffffa0492974>] ? flow_used+0x74/0xb0 [openvswitch_mod]
[ 624.619052] [<ffffffffa0491541>] dp_process_received_packet+0x71/0x150
[openvswitch_mod]
[ 624.718285] [<ffffffffa04d65a6>] ? spin+0xb6/0x200 [pktgen]
[ 624.769008] [<ffffffffa0497e47>] vport_receive+0x67/0x70 [openvswitch_mod]
[ 624.819568] [<ffffffffa0499a53>] internal_dev_xmit+0x23/0x30
[openvswitch_mod]
[ 624.920491] [<ffffffffa04d8a4a>] pktgen_xmit+0x12a/0x3a0 [pktgen]
[ 624.971535] [<ffffffffa04d8e12>] pktgen_thread_worker+0x152/0x430 [pktgen]
[ 625.021907] [<ffffffff81056131>] ? finish_task_switch+0x41/0xe0
[ 625.037324] br3: no IPv6 routers present
[ 625.119558] [<ffffffff81087940>] ? autoremove_wake_function+0x0/0x40
[ 625.168269] [<ffffffff81087940>] ? autoremove_wake_function+0x0/0x40
[ 625.216467] [<ffffffffa04d8cc0>] ? pktgen_thread_worker+0x0/0x430 [pktgen]
[ 625.264370] [<ffffffff810871f6>] kthread+0x96/0xa0
[ 625.311788] [<ffffffff8100cde4>] kernel_thread_helper+0x4/0x10
[ 625.359373] [<ffffffff81087160>] ? kthread+0x0/0xa0
[ 625.406603] [<ffffffff8100cde0>] ? kernel_thread_helper+0x0/0x10
[ 625.453778] Code: 01 e9 e7 fe ff ff 0f 1f 80 00 00 00 00 48 8b bb d8 00 00
00 e8 a4 d4 c7 ff eb b1 48 8b 40 10 8b 48 08 f0 ff 40 08 e9 64 ff ff ff <0f> 0b
0f 0b 66 66 66 66 66 2e 0f 1f 84 00 00 00 00 00 55 48 89
[ 625.599468] RIP [<ffffffff814d850e>] pskb_expand_head+0x29e/0x2b0
[ 625.646942] RSP <ffff88040fe85ab0>
[ 625.757575] ---[ end trace 17f1ece3d541e4d0 ]---
[ 625.803494] Kernel panic - not syncing: Fatal exception in interrupt
[ 625.850041] Pid: 2082, comm: kpktgend_0 Tainted: G D
2.6.38-8-server #42-Ubuntu
[ 625.940989] Call Trace:
[ 625.984474] [<ffffffff815d4a0a>] ? panic+0x91/0x19c
[ 626.027856] [<ffffffff815d8c8a>] ? oops_end+0xea/0xf0
[ 626.070646] [<ffffffff8101007b>] ? die+0x5b/0x90
[ 626.112624] [<ffffffff815d8334>] ? do_trap+0xc4/0x170
[ 626.153462] [<ffffffff8100d395>] ? do_invalid_op+0x95/0xb0
[ 626.193668] [<ffffffff814d850e>] ? pskb_expand_head+0x29e/0x2b0
[ 626.233655] [<ffffffffa0294245>] ? ixgbe_xmit_frame_ring+0x2f5/0x400 [ixgbe]
[ 626.273980] [<ffffffff815d7cce>] ? common_interrupt+0xe/0x13
[ 626.313200] [<ffffffff8100cc5b>] ? invalid_op+0x1b/0x20
[ 626.351475] [<ffffffff814d850e>] ? pskb_expand_head+0x29e/0x2b0
[ 626.389919] [<ffffffffa0496dc9>] ? tnl_send+0x649/0xc80 [openvswitch_mod]
[ 626.428602] [<ffffffff8115469c>] ? __slab_alloc+0xbc/0x390
[ 626.466776] [<ffffffff81038c79>] ? default_spin_lock_flags+0x9/0x10
[ 626.504653] [<ffffffff814d673d>] ? skb_queue_tail+0x4d/0x60
[ 626.543343] [<ffffffffa0497e6e>] ? vport_send+0x1e/0x60 [openvswitch_mod]
[ 626.582452] [<ffffffffa048e173>] ? do_output+0x23/0x40 [openvswitch_mod]
[ 626.621405] [<ffffffffa048e85e>] ? do_execute_actions+0x56e/0x810
[openvswitch_mod]
[ 626.698721] [<ffffffffa048f751>] ? queue_userspace_packet+0x321/0x340
[openvswitch_mod]
[ 626.779475] [<ffffffff815d7cce>] ? common_interrupt+0xe/0x13
[ 626.820180] [<ffffffffa0492ff2>] ? flow_extract+0x202/0xb80
[openvswitch_mod]
[ 626.900729] [<ffffffffa0499a30>] ? internal_dev_xmit+0x0/0x30
[openvswitch_mod]
[ 626.983496] [<ffffffffa048eb6c>] ? execute_actions+0x6c/0xc0
[openvswitch_mod]
[ 627.068278] [<ffffffffa0492974>] ? flow_used+0x74/0xb0 [openvswitch_mod]
[ 627.111957] [<ffffffffa0491541>] ? dp_process_received_packet+0x71/0x150
[openvswitch_mod]
[ 627.199850] [<ffffffffa04d65a6>] ? spin+0xb6/0x200 [pktgen]
[ 627.245457] [<ffffffffa0497e47>] ? vport_receive+0x67/0x70 [openvswitch_mod]
[ 627.292142] [<ffffffffa0499a53>] ? internal_dev_xmit+0x23/0x30
[openvswitch_mod]
[ 627.385725] [<ffffffffa04d8a4a>] ? pktgen_xmit+0x12a/0x3a0 [pktgen]
[ 627.434722] [<ffffffffa04d8e12>] ? pktgen_thread_worker+0x152/0x430 [pktgen]
[ 627.483683] [<ffffffff81056131>] ? finish_task_switch+0x41/0xe0
[ 627.532296] [<ffffffff81087940>] ? autoremove_wake_function+0x0/0x40
[ 627.581288] [<ffffffff81087940>] ? autoremove_wake_function+0x0/0x40
[ 627.629341] [<ffffffffa04d8cc0>] ? pktgen_thread_worker+0x0/0x430 [pktgen]
[ 627.677524] [<ffffffff810871f6>] ? kthread+0x96/0xa0
[ 627.725063] [<ffffffff8100cde4>] ? kernel_thread_helper+0x4/0x10
[ 627.772788] [<ffffffff81087160>] ? kthread+0x0/0xa0
[ 627.820006] [<ffffffff8100cde0>] ? kernel_thread_helper+0x0/0x10
_______________________________________________
dev mailing list
[email protected]
http://openvswitch.org/mailman/listinfo/dev
