> I took the liberty to paste the interesting parts to
> http://pastebin.com/m53578cd5. Notice the bottom, which was the original
> bottom as well. Mark, you've been asked to look at dmesg before this, so
I
> suppose this didn't happen before you tried this call-tracing?
> Besides, it would be interesting if an upgrade to 1.4.10 makes the
problem
> go away. Can you try that?
We upgraded to 1.4.10. We got the same errors. Here is the cmdebug
output:
=> cmdebug HOSTNAME
** Cache entry @ 0x172d1880 for 2.536870937.28.1820
[afs.dev.infoprint.com]
locks: (none_waiting, write_locked(pid:-246839824 at:681))
3 bytes DV 0 refcnt 2
callback 22174d40 expires 1242158668
0 opens 0 writers
mount point
states (0x5), stat'd, read-only
The volume # 536870937 above just happens to be root.cell but it has been
different based on which dir I do the ls in.
> Finally, it looks like ls and sshd got locked up trying to determine
your
> client machine's home cell. I can see that happening (only) if no cell
has
> been set at that point. The output of fs wscell would be interesting in
> this situation, but I'm not sure wether that would lock up as well (and
if
> it is at all helpful).
We tried the fs wscell command. It worked fine if the fs command was
local and hung if the fs was being retrieved from afs.
Also, here is a bit of interesting output from dmesg when the system is
hung:
AssertProcessEntry: pohm_main, pid=6518
openafs: Can't open inode 95550
------------[ cut here ]------------
kernel BUG at
/compile/openafs-1.4.10/src/libafs/MODLOAD-2.6.22.5-31-default-MP/osi_file.c:87!
invalid opcode: 0000 [1] SMP
last sysfs file: /class/scsi_host/host0/model
CPU 6
Modules linked in: tun iptable_filter ip_tables x_tables amk ipv6
libafs(P) microcode firmware_class usbhid hid ff_memless tp_dd af_packet
apparmor ext2 loop dm_mod parport_pc parport bnx2 rtc_cmos rtc_core
i2c_i801 rtc_lib ide_cd i2c_core cdrom shpchp tg3 pci_hotplug container
button sg ehci_hcd uhci_hcd usbcore sd_mod ata_piix libata edd ext3
mbcache jbd fan aacraid scsi_mod piix ide_core thermal processor
Pid: 6519, comm: ASCIIMast Tainted: P N 2.6.22.5-31-default #1
RIP: 0010:[<ffffffff8834dde9>] [<ffffffff8834dde9>]
:libafs:osi_UFSOpen+0x155/0x1f2
RSP: 0000:ffff81031de299c8 EFLAGS: 00010296
RAX: 0000000000000023 RBX: ffff81031a92b418 RCX: 0000000000000001
RDX: ffffffff804bdfe8 RSI: 0000000000000096 RDI: ffffffff804bdfe0
RBP: ffff81042145f000 R08: 0000000000000001 R09: ffff810001086bc0
R10: 0000000000000046 R11: ffff81042fa74ec0 R12: ffff81042af37000
R13: 000000000001753e R14: 000000000001753e R15: 0000000000000003
FS: 0000000000000000(0000) GS:ffff81042ee9f1c0(0000)
knlGS:0000000000000000
CS: 0010 DS: 002b ES: 002b CR0: 000000008005003b
CR2: 00000000f5d6050c CR3: 000000039e0b9000 CR4: 00000000000006e0
Process ASCIIMast (pid: 6519, threadinfo ffff81031de28000, task
ffff81031ad770c0)
Stack: ffffc2000770eb90 ffffc2000770caa8 ffff81042a631000
ffff8104172d1880
ffff81039800839c ffffffff8832f678 0000000300000000 0000000000000003
0000000000000000 ffffc2000770eb90 ffff8104172d1880 0000000000000000
Call Trace:
[<ffffffff8832f678>] :libafs:afs_UFSHandleLink+0xf7/0x1bd
[<ffffffff8832aadf>] :libafs:afs_lookup+0xbb2/0x115f
[<ffffffff88352713>] :libafs:afs_linux_dentry_revalidate+0x422/0x434
[<ffffffff883521ac>] :libafs:afs_linux_lookup+0x85/0x1ca
[<ffffffff883188c7>] :libafs:PagInCred+0x30/0xa9
[<ffffffff8028f972>] do_lookup+0xc4/0x1ae
[<ffffffff8029179a>] __link_path_walk+0x36c/0xd8b
[<ffffffff80299115>] dput+0x26/0x115
[<ffffffff80292066>] __link_path_walk+0xc38/0xd8b
[<ffffffff80292211>] link_path_walk+0x58/0xe0
[<ffffffff802877e9>] do_filp_open+0x1c/0x3d
[<ffffffff80292589>] do_path_lookup+0x1ab/0x227
[<ffffffff80292fb9>] __path_lookup_intent_open+0x56/0x97
[<ffffffff80293148>] open_namei+0x7a/0x674
[<ffffffff802877e9>] do_filp_open+0x1c/0x3d
[<ffffffff8028784e>] do_sys_open+0x44/0xc1
[<ffffffff80220bb2>] ia32_sysret+0x0/0xa
We still can't seem to get this system to stop hanging in the /afs area.
Mark Henry
_____________________________________________________________________________
"This message and any attachments are solely for the intended recipient and may
contain confidential or privileged information. If you are not the intended
recipient, any disclosure, copying, use, or distribution of the information
included in this message and any attachments is prohibited. If you have
received this communication in error, please notify us by reply e-mail and
immediately and permanently delete this message and any attachments. Thank
you."
_____________________________________________________________________________
