We've been seeing kernel panics caused by list_add corruption in
pvfs2_devreq_read(). Looking at the code in devpvfs2-req.c there is
a list_del inside of a list_for_each_entry. So, I think it should be
using the _safe variant of list_for_each_entry. I did a little testing
and it seems to behave correctly but I'm not sure if it remedied the
problem.
The kernel panic is below and the patch attached.
Michael
kernel: list_add corruption. prev->next should be ffff81042af26a20, but was
0000000000100100
kernel: ----------- [cut here ] --------- [please bite here ] ---------
kernel: Kernel BUG at lib/list_debug.c:31
kernel: invalid opcode: 0000 [1] SMP
kernel: last sysfs file: /devices/pci0000:00/0000:00:00.0/irq
kernel: CPU 7
kernel: Modules linked in: pvfs2(U) ipv6 xfrm_nalgo crypto_api
mx_driver(PU) mx_mcp(PU) ipmi_devintf ipmi_si ipmi_msghandler nfs fscache
nfs_acl lockd sunrpc ipt_REJECT xt_state ip_conntrack nfnetlink xt_tcpudp
iptable_filter ip_tables x_tables dm_mirror dm_multipath scsi_dh video hwmon
backlight sbs i2c_ec button battery asus_acpi acpi_memhotplug ac lp sg
i5000_edac i2c_i801 shpchp parport_pc e1000e i2c_core edac_mc parport serio_raw
pcspkr dm_raid45 dm_message dm_region_hash dm_log dm_mod dm_mem_cache ahci
libata sd_mod scsi_mod ext3 jbd uhci_hcd ohci_hcd ehci_hcd
kernel: Pid: 28211, comm: pvfs2-client-co Tainted: P 2.6.18-164.15.1.el5 #1
kernel: RIP: 0010:[<ffffffff80151ff0>] [<ffffffff80151ff0>]
__list_add+0x48/0x68
kernel: RSP: 0018:ffff810429809ea8 EFLAGS: 00010286
kernel: RAX: 0000000000000058 RBX: ffff81042af26a20 RCX: ffffffff8030ac28
kernel: RDX: ffffffff8030ac28 RSI: 0000000000000000 RDI: ffffffff8030ac20
kernel: RBP: ffff81028f75e980 R08: ffffffff8030ac28 R09: 0000000000000038
kernel: R10: 0000000000000100 R11: 0000000000000000 R12: ffff8103c0a3ed40
kernel: R13: 000000001b5f7c90 R14: 0000000000002160 R15: ffff8101dff03bc0
kernel: FS: 00002ab06e2c4890(0000) GS:ffff81043fc29340(0000)
knlGS:0000000000000000
kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
kernel: CR2: 00002ab3be581000 CR3: 0000000234f1f000 CR4: 00000000000006e0
kernel: Process pvfs2-client-co (pid: 28211, threadinfo ffff810429808000,
kernel: Stack: ffff81042aed0f80 ffff8103c0a3ab78 00000000000000a2
ffffffff88551ae4
kernel: 0000000000000000 0000000000000000 0000514100001923 ffff8101dff03bc0
kernel: 0000000000002160 000000001b5f7c90 ffff810429809f50 0000000000000000
kernel: Call Trace:
kernel: [<ffffffff88551ae4>] :pvfs2:pvfs2_devreq_read+0x1d3/0x327
kernel: [<ffffffff8000b695>] vfs_read+0xcb/0x171
kernel: [<ffffffff80011b60>] sys_read+0x45/0x6e
kernel: [<ffffffff8005d116>] system_call+0x7e/0x83
kernel:
kernel:
kernel: Code: 0f 0b 68 e9 64 2b 80 c2 1f 00 4c 89 63 08 49 89 1c 24 4c 89
kernel: RIP [<ffffffff80151ff0>] __list_add+0x48/0x68
kernel: RSP <ffff810429809ea8>
kernel: <0>Kernel panic - not syncing: Fatal exceptio
Index: pvfs2/src/kernel/linux-2.6/devpvfs2-req.c
===================================================================
RCS file: /projects/cvsroot/pvfs2/src/kernel/linux-2.6/devpvfs2-req.c,v
retrieving revision 1.78
diff -a -u -p -r1.78 devpvfs2-req.c
--- pvfs2/src/kernel/linux-2.6/devpvfs2-req.c 3 Feb 2010 18:14:55 -0000
1.78
+++ pvfs2/src/kernel/linux-2.6/devpvfs2-req.c 13 May 2010 17:45:12 -0000
@@ -112,10 +112,10 @@ static ssize_t pvfs2_devreq_read(
}
else
{
- pvfs2_kernel_op_t *op = NULL;
+ pvfs2_kernel_op_t *op = NULL, *temp = NULL;
/* get next op (if any) from top of list */
spin_lock(&pvfs2_request_list_lock);
- list_for_each_entry (op, &pvfs2_request_list, list)
+ list_for_each_entry_safe (op, temp, &pvfs2_request_list, list)
{
PVFS_fs_id fsid = fsid_of_op(op);
/* Check if this op's fsid is known and needs remounting */
_______________________________________________
Pvfs2-developers mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
