On 20/03/2021 05:01, Yoshihiro Ota wrote: > On Tue, 9 Mar 2021 11:24:53 +0200 > Andriy Gapon <a...@freebsd.org> wrote: > >> On 08/03/2021 05:24, Yoshihiro Ota wrote: >>> On Sun, 7 Mar 2021 00:09:33 +0200 >>> Andriy Gapon <a...@freebsd.org> wrote: >>> >>>> On 06/03/2021 20:09, Yoshihiro Ota wrote: >>>>> Hi all, >>>>> >>>>> I'm upgrading fron 12.2-RELEASE to 13-BETA/RC one by one. >>>>> >>>>> After upgrading one in VMWare, 'zfs mount -a' hangs the system. >>>>> I don't have boottime zfs mount on nor don't have zfsroot. >>>>> I just simply ran install world/kernel and mergemaster. >>>> >>>> Please use procstat -kk to capture a kernel stack trace of the hung >>>> process. >>> >>> Actually, spining was 'kldload zfs'. >>> Console doesn't response but ping and sshd sessions still work. >>> procstat output is below. >>> In addition, this doesn't happen to systems that I've been following >>> 13-CURRENT >>> but rather happen only wiht a system upgraded from 12.2-RELEASE to 13-RC. >>> >>> >>> # procstat -kk 1049 >>> PID TID COMM TDNAME KSTACK >>> >>> 1049 100215 kldload - spa_init+0xc6 >>> zfs_kmod_init+0x1a >>> zfs_modevent+0x34 module_register_init+0x8c linker_load_module+0xaab >>> kern_kldload+0xc1 >>> sys_kldload+0x50 syscall+0x17d g_ctx+0xe280bf29 >>> >> >> If you could use kgdb to find out what source code line spa_init+0xc6 >> corresponds to that may help to see what's going on. >> > > It look me a while to get kgdb working properly. > At last, I got the output. > It looks it is spining on a mutex. > > I have few other machines run the same kernel but they can load zfs.ko. > It is only vmware VM that spins with 'kldload zfs'. > > vmware# kgdb101 /usr/usr/lib/debug/boot/kernel/zfs.ko.debug > GNU gdb (GDB) 10.1 [GDB v10.1 for FreeBSD] > Copyright (C) 2020 Free Software Foundation, Inc. > License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> > This is free software: you are free to change and redistribute it. > There is NO WARRANTY, to the extent permitted by law. > Type "show copying" and "show warranty" for details. > This GDB was configured as "i386-portbld-freebsd13.0". > Type "show configuration" for configuration details. > For bug reporting instructions, please see: > <https://www.gnu.org/software/gdb/bugs/>. > Find the GDB manual and other documentation resources online at: > <http://www.gnu.org/software/gdb/documentation/>. > > For help, type "helpType "apropos word" to search for commands related to > "word"... > Reading symbols from zfs.ko.debug... > (kgdb) info line *spa_init+0xc6 > Line 2345 of "/usr/src/sys/contrib/openzfs/module/zfs/spa_misc.c" > starts at address 0x2b0461 <spa_init+193> > and ends at 0x2b0467 <spa_init+199>. > (kgdb) > > void > spa_init(spa_mode_t mode) > { > mutex_init(&spa_namespace_lock, NULL, MUTEX_DEFAULT, NULL); > mutex_init(&spa_spare_lock, NULL, MUTEX_DEFAULT, NULL); // <- line > 2345 >
It would highly unusual (and unexplainable) for a thread to get stuck here. Are you sure that your source tree matches the binary? Can you disassemble the function to check instructions at and near 0xc6 offset? -- Andriy Gapon _______________________________________________ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
