On Mon, May 14, 2018 at 01:41:09PM -0400, Michael H Lambert wrote: > I decided to try something more useful for debugging and built OpenAFS 1.8.0 > with "--enable-debug-kernel --disable-optimize-kernel > --disable-strip-binaries --enable-debug --disable-optimize". This is on > FreeBSD 11.1 with the fix for afs_vcache.c. The kernel panic is definitely > not limited to stopping afsd. I have seen the panic when unpacking a tar > file in an afs directory. Below is some kgdb output which might help. I can > provide more if someone tells me what to look for. I can also try rebuilding > the system with an actual partition for /var/openafs/cache, rather than using > a ZFS block device, if anyone thinks that might make a difference.
This looks awfully similar to what 54e84a98f9747bb5bb2ad4b8031115ad7684c914 is trying to fix, so I'd ask you to double-check that you are in fact running with an installed kernel module that includes the cherry-pick, as the first step. Barring that... > Thanks, > > Michael > > ----- > # kgdb /boot/kernel/kernel vmcore.last > GNU gdb 6.1.1 [FreeBSD] > Copyright 2004 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and you are > welcome to change it and/or distribute copies of it under certain conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for details. > This GDB was configured as "amd64-marcel-freebsd"... > > Unread portion of the kernel message buffer: > processor eflags = interrupt enabled, resume, IOPL = 0 > current process = 56194 (bsdtar) > trap number = 12 > panic: page fault > cpuid = 3 > KDB: stack backtrace: > #0 0xffffffff80aaf087 at kdb_backtrace+0x67 > #1 0xffffffff80a6d166 at vpanic+0x186 > #2 0xffffffff80a6cfd3 at panic+0x43 > #3 0xffffffff80eef192 at trap_fatal+0x322 > #4 0xffffffff80eef1eb at trap_pfault+0x4b > #5 0xffffffff80eee948 at trap+0x2a8 > #6 0xffffffff80ed03e0 at calltrap+0x8 > #7 0xffffffff826a685c at afs_FlushVCache+0x38c > #8 0xffffffff82733818 at afs_vop_reclaim+0x138 > #9 0xffffffff8105e219 at VOP_RECLAIM_APV+0x89 > #10 0xffffffff80b2b7fc at vgonel+0x21c > #11 0xffffffff80b2bd70 at vgone+0x40 > #12 0xffffffff82730fc4 at osi_TryEvictVCache+0x1b4 > #13 0xffffffff826a86fe at afs_ShakeLooseVCaches+0x23e > #14 0xffffffff826a8a39 at afs_NewVCache_int+0x49 > #15 0xffffffff826a89e4 at afs_NewVCache+0x24 > #16 0xffffffff826aa11d at afs_GetVCache+0x24d > #17 0xffffffff826b6514 at afs_mkdir+0x12f4 > Uptime: 1h30m33s > Dumping 1534 out of 32727 MB:..2%..11%..21%..31%..41%..51%..61%..71%..81%..91% > > Reading symbols from /boot/kernel/zfs.ko...Reading symbols from > /usr/lib/debug// > boot/kernel/zfs.ko.debug...done. > done. > Loaded symbols for /boot/kernel/zfs.ko > Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from > /usr/lib > /debug//boot/kernel/opensolaris.ko.debug...done. > done. > Loaded symbols for /boot/kernel/opensolaris.ko > Reading symbols from /boot/kernel/uhid.ko...Reading symbols from > /usr/lib/debug/ > /boot/kernel/uhid.ko.debug...done. > done. > Loaded symbols for /boot/kernel/uhid.ko > Reading symbols from /boot/kernel/pflog.ko...Reading symbols from > /usr/lib/debug > //boot/kernel/pflog.ko.debug...done. > done. > Loaded symbols for /boot/kernel/pflog.ko > Reading symbols from /boot/kernel/pf.ko...Reading symbols from > /usr/lib/debug//b > oot/kernel/pf.ko.debug...done. > done. > Loaded symbols for /boot/kernel/pf.ko > Reading symbols from /boot/modules/libafs.ko...Reading symbols from > /usr/lib/deb > ug//boot/modules/libafs.ko.debug...done. > done. > Loaded symbols for /boot/modules/libafs.ko > #0 doadump (textdump=<value optimized out>) at pcpu.h:229 > 229 pcpu.h: No such file or directory. > in pcpu.h > > > (kgdb) where > #0 doadump (textdump=<value optimized out>) at pcpu.h:229 > #1 0xffffffff80a6cce1 in kern_reboot (howto=260) > at /usr/src/sys/kern/kern_shutdown.c:366 > #2 0xffffffff80a6d1a0 in vpanic (fmt=<value optimized out>, > ap=<value optimized out>) at /usr/src/sys/kern/kern_shutdown.c:759 > #3 0xffffffff80a6cfd3 in panic (fmt=<value optimized out>) > at /usr/src/sys/kern/kern_shutdown.c:690 > #4 0xffffffff80eef192 in trap_fatal (frame=0xfffffe085e480f10, eva=468) > at /usr/src/sys/amd64/amd64/trap.c:878 > #5 0xffffffff80eef1eb in trap_pfault (frame=0xfffffe085e480f10, usermode=0) > at pcpu.h:229 > #6 0xffffffff80eee948 in trap (frame=0xfffffe085e480f10) > at /usr/src/sys/amd64/amd64/trap.c:422 > #7 0xffffffff80ed03e0 in calltrap () > at /usr/src/sys/amd64/amd64/exception.S:231 > #8 0xffffffff826a6b93 in afs_StaleVCacheFlags (avc=0xfffffe003e0161d0, > flags=2, cflags=4097) ... in kgdb, you could do 'frame 8' and then: p avc p *avc p avc->v -Ben > at /home/pscnoc/openafs/openafs-1.8.0/src/afs/afs_vcache.c:3256 > #9 0xffffffff826a685c in afs_FlushVCache (avc=0xfffffe003e0161d0, > slept=0xfffffe085e4810c0) > at /home/pscnoc/openafs/openafs-1.8.0/src/afs/afs_vcache.c:251 > #10 0xffffffff82733818 in afs_vop_reclaim (ap=0xfffffe085e481118) > at /home/pscnoc/openafs/openafs-1.8.0/src/afs/FBSD/osi_vnodeops.c:1505 > #11 0xffffffff8105e219 in VOP_RECLAIM_APV (vop=<value optimized out>, > a=0xfffffe085e481118) at vnode_if.c:2021 > #12 0xffffffff80b2b7fc in vgonel (vp=0xfffff80449c0b588) at vnode_if.h:830 > #13 0xffffffff80b2bd70 in vgone (vp=0xfffff80449c0b588) > at /usr/src/sys/kern/vfs_subr.c:3134 > #14 0xffffffff82730fc4 in osi_TryEvictVCache (avc=0xfffffe003e0161d0, > slept=0xfffffe085e481264, defersleep=0) > at /home/pscnoc/openafs/openafs-1.8.0/src/afs/FBSD/osi_vcache.c:47 > #15 0xffffffff826a86fe in afs_ShakeLooseVCaches (anumber=5) > at /home/pscnoc/openafs/openafs-1.8.0/src/afs/afs_vcache.c:776 > #16 0xffffffff826a8a39 in afs_NewVCache_int (afid=0xfffffe085e481748, > serverp=0x0, seq=0) > at /home/pscnoc/openafs/openafs-1.8.0/src/afs/afs_vcache.c:955 > #17 0xffffffff826a89e4 in afs_NewVCache (afid=0xfffffe085e481748, serverp=0x0) > at /home/pscnoc/openafs/openafs-1.8.0/src/afs/afs_vcache.c:1028 > #18 0xffffffff826aa11d in afs_GetVCache (afid=0xfffffe085e481748, > areq=0xfffff8002f944800, cached=0x0, avc=0x0) > at /home/pscnoc/openafs/openafs-1.8.0/src/afs/afs_vcache.c:1707 > #19 0xffffffff826b6514 in afs_mkdir (adp=0xfffffe003e148500, > aname=0xfffff805280be2a0 "fe", attrs=0xfffffe085e481860, > avcp=0xfffffe085e4817f0, acred=0xfffff8002fb74c00) > at /home/pscnoc/openafs/openafs-1.8.0/src/afs/VNOPS/afs_vnop_dirops.c:231 > #20 0xffffffff827330b3 in afs_vop_mkdir (ap=0xfffffe085e4819f0) > at /home/pscnoc/openafs/openafs-1.8.0/src/afs/FBSD/osi_vnodeops.c:1329 > #21 0xffffffff8105db79 in VOP_MKDIR_APV (vop=<value optimized out>, > a=0xfffffe085e4819f0) at vnode_if.c:1610 > #22 0xffffffff80b36a12 in kern_mkdirat (td=0xfffff80024b1b560, fd=-100, > path=0x80221b100 <Address 0x80221b100 out of bounds>, > segflg=UIO_USERSPACE, mode=<value optimized out>) at vnode_if.h:665 > #23 0xffffffff80eefd74 in amd64_syscall (td=0xfffff80024b1b560, traced=0) > at subr_syscall.c:135 > #24 0xffffffff80ed0c12 in fast_syscall_common () > at /usr/src/sys/amd64/amd64/exception.S:464 > #25 0x000000080221b100 in ?? () > #26 0x00000000000001ed in ?? () > #27 0x8080808080808080 in ?? () > #28 0x00000008008d4218 in ?? () > #29 0xfefefefefefefeff in ?? () > #30 0x8080808080808080 in ?? () > #31 0x0000000000000088 in ?? () > #32 0x00000000000001ed in ?? () > #33 0x00007fffffffe7a0 in ?? () > #34 0x0000000802274040 in ?? () > #35 0x0000000802222000 in ?? () > #36 0x00000000000001ed in ?? () > #37 0x000000000060f800 in ?? () > #38 0x0000000802222000 in ?? () > #39 0x0000000802216500 in ?? () > #40 0x001b001300000000 in ?? () > #41 0x003b003b00000001 in ?? () > #42 0x003b003b00000001 in ?? () > #43 0x0000000000000002 in ?? () > #44 0x0000000800c42d9a in ?? () > #45 0x0000000000000043 in ?? () > #46 0x0000000000000206 in ?? () > #47 0x00007fffffffe6c8 in ?? () > #48 0x000000000000003b in ?? () > #49 0xffffffff81d1e948 in sleepq_chains () > #50 0x0000000000000003 in ?? () > #51 0xfffff80024b1b560 in ?? () > #52 0xffffffff81d1e948 in sleepq_chains () > #53 0xfffffe085e4810a0 in ?? () > #54 0xfffffe085e481048 in ?? () > #55 0xfffff8000a3a0560 in ?? () > #56 0xffffffff80a98dfa in sched_switch (td=0x60f800, newtd=0x1ed, > flags=<value optimized out>) at /usr/src/sys/kern/sched_ule.c:1982 > Previous frame inner to this frame (corrupt stack?) > Current language: auto; currently minimal > > _______________________________________________ > OpenAFS-info mailing list > [email protected] > https://lists.openafs.org/mailman/listinfo/openafs-info _______________________________________________ OpenAFS-info mailing list [email protected] https://lists.openafs.org/mailman/listinfo/openafs-info
