On Wed, 12 Sep 2012 07:56:48 +0400 Andrew Savchenko wrote: > Hello, Elaine > > On Tue, 11 Sep 2012 11:59:55 -0400 Elaine Quarles wrote: > > A fix for this problem has been checked into the stable branch. Attached is > > a patch. This affects any system based on a 2.6.38 or later kernel. > > I tried the latest stable branch (rev. 9483) with the 3.2.28 > vanilla kernel. Result is bad, very bad... On some I/O operations, but > not on all (e.g. this happens after a series of fast directory > changes) kernel instantly hangs. There is no kernel panic, no error > messages on console (iLO) or netconsole, nothing in the logs. System > just dies until it is rebooted by a hardware watchdog. > > I'll try to recompile OrangeFS in debug mode to obtain some useful > data. My PVFS2 configuration was not changed otherwise compared to my > previous reports.
It crashed again... Works fine on single simple operations: like file creation, copying, reading, but fails on anything intense enough, like untarring of any middle-sized tarball or sizable directory tree copying. Debugging gave nothing useful: [E 08:13:50.256329] PVFS Client Daemon Started. Version 2.8.6-orangefs-2012-09-12-040311 [D 08:13:50.281437] [INFO]: Mapping pointer 0x7fa247589000 for I/O. [D 08:13:50.282263] [INFO]: Mapping pointer 0x10cc000 for I/O. That's all before a crash... But I got something on simple file operations (system continues to work, pvfs2 module is unloaded): pvfs2: module version 2.8.6-orangefs-2012-09-12-040311 loaded PVFS: kernel debug mask has been modified to "none" (0x00000000) PVFS: client debug mask has been modified to "none" (0x00000000) ------------[ cut here ]------------ WARNING: at fs/dcache.c:1287 d_set_d_op+0x74/0xb0() Hardware name: ProLiant BL2x220c G5 Modules linked in: pvfs2(O) nfsd 8021q garp stp llc xt_NOTRACK iptable_raw iptable_nat nf_nat iptable_mangle ipt_REJECT ipt_LOG xt_pkttype xt_limit xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_hashlimit xt_recent xt_conntrack iptable_filter ip_tables x_tables knem(O) nf_conntrack_ftp nf_conntrack [last unloaded: pvfs2] Pid: 3060, comm: touch Tainted: G O 3.2.28-unicluster #2 Call Trace: [<ffffffff81033d5b>] ? warn_slowpath_common+0x7b/0xc0 [<ffffffff810d5494>] ? d_set_d_op+0x74/0xb0 [<ffffffffa01185d6>] ? pvfs2_create_entry+0x1f6/0x860 [pvfs2] [<ffffffffa011f1a0>] ? pvfs2_get_custom_inode_common+0x10d0/0x1a50 [pvfs2] [<ffffffff810cc976>] ? vfs_create+0x86/0xc0 [<ffffffff810cf77d>] ? do_last.isra.48+0x58d/0x7f0 [<ffffffff810cfbbb>] ? path_openat+0xcb/0x380 [<ffffffff810a8722>] ? handle_mm_fault+0x332/0x350 [<ffffffff810cffa5>] ? do_filp_open+0x45/0xb0 [<ffffffff810db2e6>] ? alloc_fd+0x46/0x130 [<ffffffff810c0cd7>] ? do_sys_open+0xf7/0x1e0 [<ffffffff8140ecbb>] ? system_call_fastpath+0x16/0x1b ---[ end trace fe572beee59c1e73 ]--- ------------[ cut here ]------------ WARNING: at fs/dcache.c:1291 d_set_d_op+0x97/0xb0() Hardware name: ProLiant BL2x220c G5 Modules linked in: pvfs2(O) nfsd 8021q garp stp llc xt_NOTRACK iptable_raw iptable_nat nf_nat iptable_mangle ipt_REJECT ipt_LOG xt_pkttype xt_limit xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_hashlimit xt_recent xt_conntrack iptable_filter ip_tables x_tables knem(O) nf_conntrack_ftp nf_conntrack [last unloaded: pvfs2] Pid: 3060, comm: touch Tainted: G W O 3.2.28-unicluster #2 Call Trace: [<ffffffff81033d5b>] ? warn_slowpath_common+0x7b/0xc0 [<ffffffff810d54b7>] ? d_set_d_op+0x97/0xb0 [<ffffffffa01185d6>] ? pvfs2_create_entry+0x1f6/0x860 [pvfs2] [<ffffffffa011f1a0>] ? pvfs2_get_custom_inode_common+0x10d0/0x1a50 [pvfs2] [<ffffffff810cc976>] ? vfs_create+0x86/0xc0 [<ffffffff810cf77d>] ? do_last.isra.48+0x58d/0x7f0 [<ffffffff810cfbbb>] ? path_openat+0xcb/0x380 [<ffffffff810a8722>] ? handle_mm_fault+0x332/0x350 [<ffffffff810cffa5>] ? do_filp_open+0x45/0xb0 [<ffffffff810db2e6>] ? alloc_fd+0x46/0x130 [<ffffffff810c0cd7>] ? do_sys_open+0xf7/0x1e0 [<ffffffff8140ecbb>] ? system_call_fastpath+0x16/0x1b ---[ end trace fe572beee59c1e74 ]--- pvfs2: module version 2.8.6-orangefs-2012-09-12-040311 unloaded Best regards, Andrew Savchenko
pgpnBlVqJxqs2.pgp
Description: PGP signature
_______________________________________________ Pvfs2-users mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
