Re: panic: LK_RETRY set with incompatible flags
On 5 February 2013 04:13, Rick Macklem wrote:
> Sergey Kandaurov wrote:
>> On 4 February 2013 06:32, Rick Macklem wrote:
>> > Konstantin Belousov wrote:
>> >> On Sat, Feb 02, 2013 at 09:30:39PM -0500, Rick Macklem wrote:
>> >> > Andriy Gapon wrote:
>> >> > > on 31/01/2013 15:29 Sergey Kandaurov said the following:
>> >> > > > Hi.
>> >> > > >
>> >> > > > Got this assertion on idle NFS server while `ls -la
>> >> > > > /.zfs/shares/'
>> >> > > > issued on NFS client.
>> > Ok, here's a snippet of zfs_dirlook() from zfs_dir.c:
>> > 388 } else if (name[0] == '.' && name[1] == '.' && name[2] == 0) {
>> > 389 zfsvfs_t *zfsvfs = dzp->z_zfsvfs;
>> > 390
>> > 391 /*
>> > 392 * If we are a snapshot mounted under .zfs, return
>> > 393 * the vp for the snapshot directory.
>> > 394 */
>> > 395 if ((error = sa_lookup(dzp->z_sa_hdl,
>> > 396 SA_ZPL_PARENT(zfsvfs), &parent, sizeof (parent))) != 0)
>> > 397 return (error);
>> > 398 if (parent == dzp->z_id && zfsvfs->z_parent != zfsvfs) {
>> > 399 error = zfsctl_root_lookup(zfsvfs->z_parent->z_ctldir,
>> > 400 "snapshot", vpp, NULL, 0, NULL, kcred,
>> > 401 NULL, NULL, NULL);
>> > 402 return (error);
>> > 403 }
>> >
>> > Just reading the comment, I don't know what it is referring to by
>> > "snapshot directory". Would that be "shares" for Sergey's case?
>> >
>> > It seems too obvious, but maybe the lookup of ".." is returning the
>> > vnode for /.zfs/shares for this case?
>> >
>> > I don't know why this wouldn't have shown up before, but maybe it
>> > usually
>> > replies from its cache (I think zfs_lookup() calls it a "fast
>> > path").
>> >
>> > It might still be interesting to replace zfs_vnops.c line# 1451
>> > with:
>> > if ((cnp->cn_flags & ISDOTDOT) && *vpp != dvp)
>> > and see what happens?
>> >
>>
>> With this change `ls /home/user1001/.zfs/shares/' lists empty
>> directory
>> (although the relevant dataset has snapshot, but that's a different
>> story :)).
>> Great!
>> Nothing panics/asserts/etc, just seemingly unrelated LOR
>>
> Yes, I think the patch is relatively safe, since lookup() checks for
> same vnode and does a vrele() instead of a vput() when they are the same,
> at least for a plain lookup without wantparent.
>
> So, since I've never used ZFS, what does a "ls -la
> /home/user1001/.zfs/shares/"
> give you when done locally one the server?
On server (with unmodified kernel):
# ls -la /pool1/user1001/.zfs/share
total 2
dr-xr-xr-x 2 root wheel 2 Feb 2 20:06 .
dr-xr-xr-x 4 root wheel 4 Feb 2 20:06 ..
It crashes only when .zfs/share is accessed via NFS
(with and without snapshots), and not when accessed locally.
--
wbr,
pluknet
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: panic: LK_RETRY set with incompatible flags
Sergey Kandaurov wrote:
> On 4 February 2013 06:32, Rick Macklem wrote:
> > Konstantin Belousov wrote:
> >> On Sat, Feb 02, 2013 at 09:30:39PM -0500, Rick Macklem wrote:
> >> > Andriy Gapon wrote:
> >> > > on 31/01/2013 15:29 Sergey Kandaurov said the following:
> >> > > > Hi.
> >> > > >
> >> > > > Got this assertion on idle NFS server while `ls -la
> >> > > > /.zfs/shares/'
> >> > > > issued on NFS client.
> > Ok, here's a snippet of zfs_dirlook() from zfs_dir.c:
> > 388 } else if (name[0] == '.' && name[1] == '.' && name[2] == 0) {
> > 389 zfsvfs_t *zfsvfs = dzp->z_zfsvfs;
> > 390
> > 391 /*
> > 392 * If we are a snapshot mounted under .zfs, return
> > 393 * the vp for the snapshot directory.
> > 394 */
> > 395 if ((error = sa_lookup(dzp->z_sa_hdl,
> > 396 SA_ZPL_PARENT(zfsvfs), &parent, sizeof (parent))) != 0)
> > 397 return (error);
> > 398 if (parent == dzp->z_id && zfsvfs->z_parent != zfsvfs) {
> > 399 error = zfsctl_root_lookup(zfsvfs->z_parent->z_ctldir,
> > 400 "snapshot", vpp, NULL, 0, NULL, kcred,
> > 401 NULL, NULL, NULL);
> > 402 return (error);
> > 403 }
> >
> > Just reading the comment, I don't know what it is referring to by
> > "snapshot directory". Would that be "shares" for Sergey's case?
> >
> > It seems too obvious, but maybe the lookup of ".." is returning the
> > vnode for /.zfs/shares for this case?
> >
> > I don't know why this wouldn't have shown up before, but maybe it
> > usually
> > replies from its cache (I think zfs_lookup() calls it a "fast
> > path").
> >
> > It might still be interesting to replace zfs_vnops.c line# 1451
> > with:
> > if ((cnp->cn_flags & ISDOTDOT) && *vpp != dvp)
> > and see what happens?
> >
>
> With this change `ls /home/user1001/.zfs/shares/' lists empty
> directory
> (although the relevant dataset has snapshot, but that's a different
> story :)).
> Great!
> Nothing panics/asserts/etc, just seemingly unrelated LOR
>
Yes, I think the patch is relatively safe, since lookup() checks for
same vnode and does a vrele() instead of a vput() when they are the same,
at least for a plain lookup without wantparent.
So, since I've never used ZFS, what does a "ls -la /home/user1001/.zfs/shares/"
give you when done locally one the server?
> 1st 0xfe00b9569d50 zfs (zfs) @
> /usr/src/sys/fs/nfsserver/nfs_nfsdsocket.c:884
> 2nd 0xfe001dfd89a0 ufs (ufs) @ /usr/src/sys/kern/vfs_vnops.c:461
> KDB: stack backtrace:
> db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame
> 0xff848e709d00
> kdb_backtrace() at kdb_backtrace+0x39/frame 0xff848e709db0
> witness_checkorder() at witness_checkorder+0xc47/frame
> 0xff848e709e30
> __lockmgr_args() at __lockmgr_args+0x6c9/frame 0xff848e709f60
> ffs_lock() at ffs_lock+0x84/frame 0xff848e709fb0
> VOP_LOCK1_APV() at VOP_LOCK1_APV+0xd0/frame 0xff848e709fd0
> _vn_lock() at _vn_lock+0xab/frame 0xff848e70a040
> vn_rdwr() at vn_rdwr+0x1f1/frame 0xff848e70a100
> nfsrv_writestable() at nfsrv_writestable+0xbe/frame 0xff848e70a170
> nfsrv_openupdate() at nfsrv_openupdate+0x489/frame 0xff848e70a5d0
> nfsrvd_openconfirm() at nfsrvd_openconfirm+0x123/frame
> 0xff848e70a6b0
> nfsrvd_dorpc() at nfsrvd_dorpc+0xf9a/frame 0xff848e70a8a0
> nfssvc_program() at nfssvc_program+0x482/frame 0xff848e70aa00
> svc_run_internal() at svc_run_internal+0x1e9/frame 0xff848e70aba0
> svc_thread_start() at svc_thread_start+0xb/frame 0xff848e70abb0
> fork_exit() at fork_exit+0x84/frame 0xff848e70abf0
> fork_trampoline() at fork_trampoline+0xe/frame 0xff848e70abf0
> --- trap 0xc, rip = 0x800883b7a, rsp = 0x7fffd6c8, rbp =
> 0x7fffd970 ---
>
I don't think this LOR is an issue. It definitely isn't related to
the panic/crash.
Thanks for collecting this information and passing it along, rick
>
> --
> wbr,
> pluknet
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: panic: LK_RETRY set with incompatible flags
Sergey Kandaurov wrote: > On 4 February 2013 16:30, Sergey Kandaurov wrote: > > On 4 February 2013 16:13, Sergey Kandaurov > > wrote: > >> On 4 February 2013 16:06, Andriy Gapon wrote: > >>> on 04/02/2013 13:49 Sergey Kandaurov said the following: > Hi, Rick! Here is the requested info regarding witness, and a bit > more. > The triggered KASSERT is now different though. > >>> > >>> It's exactly the same problem though :-) > >>> Do you have a crashdump? > >>> If yes, please print **vpp. > >> > >> Yep, but it's in a bad form :( > >> It has many bits optimized out (compiled with clang). > >> I'll rebuild the kernel with -O or so and will try again. > >> > >> #8 0x808bc4ce in kdb_enter (why=0x80e7ed99 "panic", > >> msg=) at cpufunc.h:63 > >> #9 0x80888fb7 in vpanic (fmt=, > >> ap=) at > >> /usr/src/sys/kern/kern_shutdown.c:746 > >> #10 0x80888e66 in kassert_panic (fmt=) > >> at /usr/src/sys/kern/kern_shutdown.c:641 > >> #11 0x808d2259 in witness_checkorder > >> (lock=0xfe00a1b44240, > >> ---Type to continue, or q to quit--- > >> flags=1, > >> file=0x81b2bb36 > >> "/usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c", > >> line=1452, > >> interlock=) at > >> /usr/src/sys/kern/subr_witness.c:1122 > >> #12 0x8086c11e in __lockmgr_args (lk=0xfe00a1b44240, > >> flags=, ilk=0xfe00a1b44270, > >> wmesg=0x81b2808d "zfs", pri=96, timo=51, > >> file=0x80e8e407 "/usr/src/sys/kern/vfs_default.c", > >> line=0) > >> at /usr/src/sys/kern/kern_lock.c:511 > >> #13 0x8091439c in vop_stdlock (ap=) > >> at lockmgr.h:97 > >> #14 0x80cb70c0 in VOP_LOCK1_APV (vop=, > >> a=) at vnode_if.c:2022 > >> #15 0x80932fbb in _vn_lock (vp=0xfe00a1b441d8, > >> flags=, > >> file=0x81b2bb36 > >> "/usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c", > >> line=1452) at vnode_if.h:859 > >> #16 0x81abd902 in zfs_lookup () > >> at > >> > >> /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:1452 > >> #17 0x81abdc1d in zfs_freebsd_lookup > >> (ap=0xff848e6c0270) > >> at > >> > >> /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:5864 > >> ---Type to continue, or q to quit--- > >> #18 0x80cb51b2 in VOP_CACHEDLOOKUP_APV (vop= >> optimized out>, > >> a=) at vnode_if.c:191 > >> #19 0x8091177f in vfs_cache_lookup (ap= >> out>) > >> at vnode_if.h:80 > >> #20 0x80cb50a2 in VOP_LOOKUP_APV (vop= >> out>, > >> a=) at vnode_if.c:125 > >> #21 0x80919658 in lookup (ndp=0xff848e6c05d8) at > >> vnode_if.h:54 > >> #22 0x807dffe5 in nfsvno_namei (nd=0xff848e6c08c8, > >> ndp=0xff848e6c05d8, dp=, > >> islocked=, exp=, > >> p=, retdirp=) > >> at /usr/src/sys/fs/nfsserver/nfs_nfsdport.c:413 > >> #23 0x807d8ffa in nfsrvd_lookup (nd=0xff848e6c08c8, > >> isdgram=, dp=0xfe00a1b441d8, > >> vpp=0xff848e6c0810, fhp=0xff848e6c07f4, > >> p=0xfe00a1198000, > >> exp=0xff848e6c07a0) at > >> /usr/src/sys/fs/nfsserver/nfs_nfsdserv.c:517 > >> #24 0x807cb845 in nfsrvd_dorpc (nd=0xff848e6c08c8, > >> isdgram=0, > >> p=0xfe00a1198000) at > >> /usr/src/sys/fs/nfsserver/nfs_nfsdsocket.c:823 > >> #25 0x807d7af2 in nfssvc_program (rqst=0xfe00a17bb000, > >> xprt=0xfe00a124b200) at > >> /usr/src/sys/fs/nfsserver/nfs_nfsdkrpc.c:347 > >> #26 0x80a70659 in svc_run_internal > >> (pool=0xfe00067db600, > >> ismaster=0) at /usr/src/sys/rpc/svc.c:895 > >> #27 0x80a7155b in svc_thread_start (arg=0xff848e6bfc50) > >> ---Type to continue, or q to quit--- > >> at /usr/src/sys/rpc/svc.c:1200 > >> #28 0x80858944 in fork_exit ( > >> callout=0x80a71550 , > >> arg=0xfe00067db600, > >> frame=0xff848e6c0c00) at /usr/src/sys/kern/kern_fork.c:991 > >> #29 0x80bfa86e in fork_trampoline () at exception.S:602 > >> #30 0x0080 in ?? () > >> #31 0x7fffd820 in ?? () > >> #32 0x0001 in ?? () > >> #33 0x in ?? () > >> Current language: auto; currently minimal > >> > >> (kgdb) p vpp > >> Cannot access memory at address 0x0 > >> (kgdb) p dvp > >> Cannot access memory at address 0x0 > >> > > > > That was from f 16. And as Andriy suggested in private (thanks > > Andriy!) > > those pointers could be reached from ap in f 17. > > On the other hand, zfs_lookup() is large, and *vpp might be > > rewritten several times there. Here it is: > > > > (kgdb) f 17 > > #17 0x81abdc1d in zfs_freebsd_lookup (ap=0xff848e6c0270) > > at > > > > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:5864 > > 5864 return
Dtrace status - material for someone's TODO list.
Hello; Last year I did an attempt to merge some of the changes from illumos' Dtrace into FreeBSD. I don't use Dtrace a lot, so this was done mainly for fun. In general, merging changes from Illumos was pretty straightforward; just a matter of readapting the paths to match our layout. I was able to merge some basic stuff (semilog plots) and fixes but we are at a point now where they are building upon their own developments and we need to merge their stuff in the same order they did. Some of their changes (KVM and zones) we can just ignore. As of lately, I got stuck because our userland support is different from the Solaris stuff. The conflicts I am seeing now are not easy for my (non-)level of inner Dtrace-foo and I am not working more on this so I will post the links to the patches in the hope that someone will pick up where I left: - 1368 enablings on defunct providers prevent providers from unregistering http://people.freebsd.org/~pfg/patches/dtrace/illumos-gate-8e6add739e38.diff - 1455 DTrace tracemem() should take an optional size argument http://people.freebsd.org/~pfg/patches/dtrace/illumos-gate-571b0355c2e3.diff Once done with those we can continue bringing the nicer features: http://dtrace.org/blogs/eschrock/2011/10/26/your-mdb-fell-into-my-dtrace/ Cheers, Pedro. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: CURRENT: Broken on > r246222?
Am 02/04/13 20:50, schrieb John Baldwin: > On Sunday, February 03, 2013 11:26:53 am O. Hartmann wrote: >> CURRENT, as of recent Revision 246285 (see below) fails and >> crashes/reboots on an Ivy-Bridge platform (see below). Since the box >> does not have debugging switched on (not yet, will do eventually >> Monday), Last thing I see on screen is >> >> p4tcc3: on cpu3 >> >> then the system run dark and reboots without any notice (kernel doesn't >> have debugging switched on so far, will do this after Monday). > > I don't think there are any known issues. Can you narrow it down to a > specific commit? > No, I can't. But I can now confirm, that with r246326 the problem has gone. Oliver signature.asc Description: OpenPGP digital signature
Re: CURRENT: Broken on > r246222?
On Sunday, February 03, 2013 11:26:53 am O. Hartmann wrote: > CURRENT, as of recent Revision 246285 (see below) fails and > crashes/reboots on an Ivy-Bridge platform (see below). Since the box > does not have debugging switched on (not yet, will do eventually > Monday), Last thing I see on screen is > > p4tcc3: on cpu3 > > then the system run dark and reboots without any notice (kernel doesn't > have debugging switched on so far, will do this after Monday). I don't think there are any known issues. Can you narrow it down to a specific commit? -- John Baldwin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: r246057: buildworld fails with: /usr/obj/usr/src/tmp/usr/lib/libc++.so: undefined reference to `std::bad_alloc::~bad_alloc()'
Am 02/03/13 21:48, schrieb Dimitry Andric: > On 2013-01-29 11:06, O. Hartmann wrote: >> I receive this error since yesterday building world and it is still >> sticky on most recent sources (r246057) and I was wondering why the >> tinderboxes do not pick this up on the 10.0-CURRENT builds ... just for >> a notice for the development folks ... > ... >> /usr/obj/usr/src/tmp/usr/lib/libc++.so: undefined reference to >> `std::bad_alloc::~bad_alloc()' > > I have committed a fix for libcxxrt's symbol version map in r246297. > Please update to that revision, clean out /usr/obj, and try buildworld > again. > > -Dimitry Thank you, works. Oliver signature.asc Description: OpenPGP digital signature
Re: wifi and wpa_supplicant
On 02/04/2013 10:58 AM, CeDeROM wrote: > Thank you Nikolai! Shouldn't this be a default when > ifconfig_wlan0="WPA DHCP" in rc.conf? :-) > > Best regards :-) > Tomek > > On Mon, Feb 4, 2013 at 4:47 PM, Nikolai Lifanov > wrote: >> On 02/04/2013 10:37 AM, CeDeROM wrote: >>> Hello :-) >>> >>> I am wondering how exactly the wifi interface and wpa_supplicant is >>> organized - is there any script at wlan0 interface up that starts >>> wpa_supplicant for that interface? Do I have to start wpa_supplicant >>> by hand any time I bring interface down and up (it looks yes for me)? >>> I have ifconfig_wlan0="WPA DHCP" set in rc.conf so I guess after/when >>> interface is up both wpa_supplicant and dhclient should be started..? >>> What should I make to wpa_supplicant starts automatically when I bring >>> wlan0 up? I am using iwn intel wifi driver. >>> >>> Any hints appreciated :-) >>> Tomek >>> >> >> If you want to do this automatically outside of rc.conf, take a look at >> devd(8). >> >> - Nikolai Lifanov >> >> ___ >> freebsd-current@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-current >> To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org" > > > Well, ifconfig_wlan0="WPA DHCP" should connect you at boot if you have your wpa_supplicant.conf(5) configured for it. There is a good writeup for it in the handbook: http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/network-wireless.html Also, this should probably have gone to freebsd-users@ and not freebsd-current@ - Nikolai Lifanov ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: wifi and wpa_supplicant
On 02/04/2013 10:37 AM, CeDeROM wrote: > Hello :-) > > I am wondering how exactly the wifi interface and wpa_supplicant is > organized - is there any script at wlan0 interface up that starts > wpa_supplicant for that interface? Do I have to start wpa_supplicant > by hand any time I bring interface down and up (it looks yes for me)? > I have ifconfig_wlan0="WPA DHCP" set in rc.conf so I guess after/when > interface is up both wpa_supplicant and dhclient should be started..? > What should I make to wpa_supplicant starts automatically when I bring > wlan0 up? I am using iwn intel wifi driver. > > Any hints appreciated :-) > Tomek > If you want to do this automatically outside of rc.conf, take a look at devd(8). - Nikolai Lifanov ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
wifi and wpa_supplicant
Hello :-) I am wondering how exactly the wifi interface and wpa_supplicant is organized - is there any script at wlan0 interface up that starts wpa_supplicant for that interface? Do I have to start wpa_supplicant by hand any time I bring interface down and up (it looks yes for me)? I have ifconfig_wlan0="WPA DHCP" set in rc.conf so I guess after/when interface is up both wpa_supplicant and dhclient should be started..? What should I make to wpa_supplicant starts automatically when I bring wlan0 up? I am using iwn intel wifi driver. Any hints appreciated :-) Tomek -- CeDeROM, SQ7MHZ, http://www.tomek.cedro.info ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: panic: LK_RETRY set with incompatible flags
On 4 February 2013 06:32, Rick Macklem wrote:
> Konstantin Belousov wrote:
>> On Sat, Feb 02, 2013 at 09:30:39PM -0500, Rick Macklem wrote:
>> > Andriy Gapon wrote:
>> > > on 31/01/2013 15:29 Sergey Kandaurov said the following:
>> > > > Hi.
>> > > >
>> > > > Got this assertion on idle NFS server while `ls -la
>> > > > /.zfs/shares/'
>> > > > issued on NFS client.
> Ok, here's a snippet of zfs_dirlook() from zfs_dir.c:
> 388 } else if (name[0] == '.' && name[1] == '.' && name[2] == 0) {
> 389 zfsvfs_t *zfsvfs = dzp->z_zfsvfs;
> 390
> 391 /*
> 392* If we are a snapshot mounted under .zfs, return
> 393* the vp for the snapshot directory.
> 394*/
> 395 if ((error = sa_lookup(dzp->z_sa_hdl,
> 396SA_ZPL_PARENT(zfsvfs), &parent, sizeof (parent))) != 0)
> 397 return (error);
> 398 if (parent == dzp->z_id && zfsvfs->z_parent != zfsvfs) {
> 399 error = zfsctl_root_lookup(zfsvfs->z_parent->z_ctldir,
> 400 "snapshot", vpp, NULL, 0, NULL, kcred,
> 401 NULL, NULL, NULL);
> 402 return (error);
> 403 }
>
> Just reading the comment, I don't know what it is referring to by
> "snapshot directory". Would that be "shares" for Sergey's case?
>
> It seems too obvious, but maybe the lookup of ".." is returning the
> vnode for /.zfs/shares for this case?
>
> I don't know why this wouldn't have shown up before, but maybe it usually
> replies from its cache (I think zfs_lookup() calls it a "fast path").
>
> It might still be interesting to replace zfs_vnops.c line# 1451 with:
> if ((cnp->cn_flags & ISDOTDOT) && *vpp != dvp)
> and see what happens?
>
With this change `ls /home/user1001/.zfs/shares/' lists empty directory
(although the relevant dataset has snapshot, but that's a different story :)).
Great!
Nothing panics/asserts/etc, just seemingly unrelated LOR
1st 0xfe00b9569d50 zfs (zfs) @
/usr/src/sys/fs/nfsserver/nfs_nfsdsocket.c:884
2nd 0xfe001dfd89a0 ufs (ufs) @ /usr/src/sys/kern/vfs_vnops.c:461
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xff848e709d00
kdb_backtrace() at kdb_backtrace+0x39/frame 0xff848e709db0
witness_checkorder() at witness_checkorder+0xc47/frame 0xff848e709e30
__lockmgr_args() at __lockmgr_args+0x6c9/frame 0xff848e709f60
ffs_lock() at ffs_lock+0x84/frame 0xff848e709fb0
VOP_LOCK1_APV() at VOP_LOCK1_APV+0xd0/frame 0xff848e709fd0
_vn_lock() at _vn_lock+0xab/frame 0xff848e70a040
vn_rdwr() at vn_rdwr+0x1f1/frame 0xff848e70a100
nfsrv_writestable() at nfsrv_writestable+0xbe/frame 0xff848e70a170
nfsrv_openupdate() at nfsrv_openupdate+0x489/frame 0xff848e70a5d0
nfsrvd_openconfirm() at nfsrvd_openconfirm+0x123/frame 0xff848e70a6b0
nfsrvd_dorpc() at nfsrvd_dorpc+0xf9a/frame 0xff848e70a8a0
nfssvc_program() at nfssvc_program+0x482/frame 0xff848e70aa00
svc_run_internal() at svc_run_internal+0x1e9/frame 0xff848e70aba0
svc_thread_start() at svc_thread_start+0xb/frame 0xff848e70abb0
fork_exit() at fork_exit+0x84/frame 0xff848e70abf0
fork_trampoline() at fork_trampoline+0xe/frame 0xff848e70abf0
--- trap 0xc, rip = 0x800883b7a, rsp = 0x7fffd6c8, rbp = 0x7fffd970 ---
--
wbr,
pluknet
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: panic: LK_RETRY set with incompatible flags
On 4 February 2013 16:30, Sergey Kandaurov wrote:
> On 4 February 2013 16:13, Sergey Kandaurov wrote:
>> On 4 February 2013 16:06, Andriy Gapon wrote:
>>> on 04/02/2013 13:49 Sergey Kandaurov said the following:
Hi, Rick! Here is the requested info regarding witness, and a bit more.
The triggered KASSERT is now different though.
>>>
>>> It's exactly the same problem though :-)
>>> Do you have a crashdump?
>>> If yes, please print **vpp.
>>
>> Yep, but it's in a bad form :(
>> It has many bits optimized out (compiled with clang).
>> I'll rebuild the kernel with -O or so and will try again.
>>
>> #8 0x808bc4ce in kdb_enter (why=0x80e7ed99 "panic",
>> msg=) at cpufunc.h:63
>> #9 0x80888fb7 in vpanic (fmt=,
>> ap=) at /usr/src/sys/kern/kern_shutdown.c:746
>> #10 0x80888e66 in kassert_panic (fmt=)
>> at /usr/src/sys/kern/kern_shutdown.c:641
>> #11 0x808d2259 in witness_checkorder (lock=0xfe00a1b44240,
>> ---Type to continue, or q to quit---
>> flags=1,
>> file=0x81b2bb36
>> "/usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c",
>> line=1452,
>> interlock=) at /usr/src/sys/kern/subr_witness.c:1122
>> #12 0x8086c11e in __lockmgr_args (lk=0xfe00a1b44240,
>> flags=, ilk=0xfe00a1b44270,
>> wmesg=0x81b2808d "zfs", pri=96, timo=51,
>> file=0x80e8e407 "/usr/src/sys/kern/vfs_default.c", line=0)
>> at /usr/src/sys/kern/kern_lock.c:511
>> #13 0x8091439c in vop_stdlock (ap=)
>> at lockmgr.h:97
>> #14 0x80cb70c0 in VOP_LOCK1_APV (vop=,
>> a=) at vnode_if.c:2022
>> #15 0x80932fbb in _vn_lock (vp=0xfe00a1b441d8,
>> flags=,
>> file=0x81b2bb36
>> "/usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c",
>> line=1452) at vnode_if.h:859
>> #16 0x81abd902 in zfs_lookup ()
>> at
>> /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:1452
>> #17 0x81abdc1d in zfs_freebsd_lookup (ap=0xff848e6c0270)
>> at
>> /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:5864
>> ---Type to continue, or q to quit---
>> #18 0x80cb51b2 in VOP_CACHEDLOOKUP_APV (vop=,
>> a=) at vnode_if.c:191
>> #19 0x8091177f in vfs_cache_lookup (ap=)
>> at vnode_if.h:80
>> #20 0x80cb50a2 in VOP_LOOKUP_APV (vop=,
>> a=) at vnode_if.c:125
>> #21 0x80919658 in lookup (ndp=0xff848e6c05d8) at vnode_if.h:54
>> #22 0x807dffe5 in nfsvno_namei (nd=0xff848e6c08c8,
>> ndp=0xff848e6c05d8, dp=,
>> islocked=, exp=,
>> p=, retdirp=)
>> at /usr/src/sys/fs/nfsserver/nfs_nfsdport.c:413
>> #23 0x807d8ffa in nfsrvd_lookup (nd=0xff848e6c08c8,
>> isdgram=, dp=0xfe00a1b441d8,
>> vpp=0xff848e6c0810, fhp=0xff848e6c07f4, p=0xfe00a1198000,
>> exp=0xff848e6c07a0) at /usr/src/sys/fs/nfsserver/nfs_nfsdserv.c:517
>> #24 0x807cb845 in nfsrvd_dorpc (nd=0xff848e6c08c8, isdgram=0,
>> p=0xfe00a1198000) at /usr/src/sys/fs/nfsserver/nfs_nfsdsocket.c:823
>> #25 0x807d7af2 in nfssvc_program (rqst=0xfe00a17bb000,
>> xprt=0xfe00a124b200) at /usr/src/sys/fs/nfsserver/nfs_nfsdkrpc.c:347
>> #26 0x80a70659 in svc_run_internal (pool=0xfe00067db600,
>> ismaster=0) at /usr/src/sys/rpc/svc.c:895
>> #27 0x80a7155b in svc_thread_start (arg=0xff848e6bfc50)
>> ---Type to continue, or q to quit---
>> at /usr/src/sys/rpc/svc.c:1200
>> #28 0x80858944 in fork_exit (
>> callout=0x80a71550 , arg=0xfe00067db600,
>> frame=0xff848e6c0c00) at /usr/src/sys/kern/kern_fork.c:991
>> #29 0x80bfa86e in fork_trampoline () at exception.S:602
>> #30 0x0080 in ?? ()
>> #31 0x7fffd820 in ?? ()
>> #32 0x0001 in ?? ()
>> #33 0x in ?? ()
>> Current language: auto; currently minimal
>>
>> (kgdb) p vpp
>> Cannot access memory at address 0x0
>> (kgdb) p dvp
>> Cannot access memory at address 0x0
>>
>
> That was from f 16. And as Andriy suggested in private (thanks Andriy!)
> those pointers could be reached from ap in f 17.
> On the other hand, zfs_lookup() is large, and *vpp might be
> rewritten several times there. Here it is:
>
> (kgdb) f 17
> #17 0x81abdc1d in zfs_freebsd_lookup (ap=0xff848e6c0270)
> at
> /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:5864
> 5864return (zfs_lookup(ap->a_dvp, nm, ap->a_vpp, cnp,
> cnp->cn_nameiop,
> (kgdb) p ap->a_vpp
> $4 = (struct vnode **) 0xff848e6c0618
> (kgdb) p *ap->a_vpp
> $5 = (struct vnode *) 0xfe00a1b441d8
> (kgdb) p **ap->a_vpp
> $6 = {v_tag = 0x81b2808d "zfs", v_op = 0x81b35808,
> v_data = 0xfe00a188f000, v_mount = 0xfe0015a5f000, v_nmntvnodes = {
> tqe_next =
Re: panic: LK_RETRY set with incompatible flags
On 4 February 2013 16:13, Sergey Kandaurov wrote:
> On 4 February 2013 16:06, Andriy Gapon wrote:
>> on 04/02/2013 13:49 Sergey Kandaurov said the following:
>>> Hi, Rick! Here is the requested info regarding witness, and a bit more.
>>> The triggered KASSERT is now different though.
>>
>> It's exactly the same problem though :-)
>> Do you have a crashdump?
>> If yes, please print **vpp.
>
> Yep, but it's in a bad form :(
> It has many bits optimized out (compiled with clang).
> I'll rebuild the kernel with -O or so and will try again.
>
> #8 0x808bc4ce in kdb_enter (why=0x80e7ed99 "panic",
> msg=) at cpufunc.h:63
> #9 0x80888fb7 in vpanic (fmt=,
> ap=) at /usr/src/sys/kern/kern_shutdown.c:746
> #10 0x80888e66 in kassert_panic (fmt=)
> at /usr/src/sys/kern/kern_shutdown.c:641
> #11 0x808d2259 in witness_checkorder (lock=0xfe00a1b44240,
> ---Type to continue, or q to quit---
> flags=1,
> file=0x81b2bb36
> "/usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c",
> line=1452,
> interlock=) at /usr/src/sys/kern/subr_witness.c:1122
> #12 0x8086c11e in __lockmgr_args (lk=0xfe00a1b44240,
> flags=, ilk=0xfe00a1b44270,
> wmesg=0x81b2808d "zfs", pri=96, timo=51,
> file=0x80e8e407 "/usr/src/sys/kern/vfs_default.c", line=0)
> at /usr/src/sys/kern/kern_lock.c:511
> #13 0x8091439c in vop_stdlock (ap=)
> at lockmgr.h:97
> #14 0x80cb70c0 in VOP_LOCK1_APV (vop=,
> a=) at vnode_if.c:2022
> #15 0x80932fbb in _vn_lock (vp=0xfe00a1b441d8,
> flags=,
> file=0x81b2bb36
> "/usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c",
> line=1452) at vnode_if.h:859
> #16 0x81abd902 in zfs_lookup ()
> at
> /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:1452
> #17 0x81abdc1d in zfs_freebsd_lookup (ap=0xff848e6c0270)
> at
> /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:5864
> ---Type to continue, or q to quit---
> #18 0x80cb51b2 in VOP_CACHEDLOOKUP_APV (vop=,
> a=) at vnode_if.c:191
> #19 0x8091177f in vfs_cache_lookup (ap=)
> at vnode_if.h:80
> #20 0x80cb50a2 in VOP_LOOKUP_APV (vop=,
> a=) at vnode_if.c:125
> #21 0x80919658 in lookup (ndp=0xff848e6c05d8) at vnode_if.h:54
> #22 0x807dffe5 in nfsvno_namei (nd=0xff848e6c08c8,
> ndp=0xff848e6c05d8, dp=,
> islocked=, exp=,
> p=, retdirp=)
> at /usr/src/sys/fs/nfsserver/nfs_nfsdport.c:413
> #23 0x807d8ffa in nfsrvd_lookup (nd=0xff848e6c08c8,
> isdgram=, dp=0xfe00a1b441d8,
> vpp=0xff848e6c0810, fhp=0xff848e6c07f4, p=0xfe00a1198000,
> exp=0xff848e6c07a0) at /usr/src/sys/fs/nfsserver/nfs_nfsdserv.c:517
> #24 0x807cb845 in nfsrvd_dorpc (nd=0xff848e6c08c8, isdgram=0,
> p=0xfe00a1198000) at /usr/src/sys/fs/nfsserver/nfs_nfsdsocket.c:823
> #25 0x807d7af2 in nfssvc_program (rqst=0xfe00a17bb000,
> xprt=0xfe00a124b200) at /usr/src/sys/fs/nfsserver/nfs_nfsdkrpc.c:347
> #26 0x80a70659 in svc_run_internal (pool=0xfe00067db600,
> ismaster=0) at /usr/src/sys/rpc/svc.c:895
> #27 0x80a7155b in svc_thread_start (arg=0xff848e6bfc50)
> ---Type to continue, or q to quit---
> at /usr/src/sys/rpc/svc.c:1200
> #28 0x80858944 in fork_exit (
> callout=0x80a71550 , arg=0xfe00067db600,
> frame=0xff848e6c0c00) at /usr/src/sys/kern/kern_fork.c:991
> #29 0x80bfa86e in fork_trampoline () at exception.S:602
> #30 0x0080 in ?? ()
> #31 0x7fffd820 in ?? ()
> #32 0x0001 in ?? ()
> #33 0x in ?? ()
> Current language: auto; currently minimal
>
> (kgdb) p vpp
> Cannot access memory at address 0x0
> (kgdb) p dvp
> Cannot access memory at address 0x0
>
That was from f 16. And as Andriy suggested in private (thanks Andriy!)
those pointers could be reached from ap in f 17.
On the other hand, zfs_lookup() is large, and *vpp might be
rewritten several times there. Here it is:
(kgdb) f 17
#17 0x81abdc1d in zfs_freebsd_lookup (ap=0xff848e6c0270)
at
/usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:5864
5864return (zfs_lookup(ap->a_dvp, nm, ap->a_vpp, cnp,
cnp->cn_nameiop,
(kgdb) p ap->a_vpp
$4 = (struct vnode **) 0xff848e6c0618
(kgdb) p *ap->a_vpp
$5 = (struct vnode *) 0xfe00a1b441d8
(kgdb) p **ap->a_vpp
$6 = {v_tag = 0x81b2808d "zfs", v_op = 0x81b35808,
v_data = 0xfe00a188f000, v_mount = 0xfe0015a5f000, v_nmntvnodes = {
tqe_next = 0xfe00a1b44000, tqe_prev = 0xfe00a1b443d0}, v_un = {
vu_mount = 0x0, vu_socket = 0x0, vu_cdev = 0x0, vu_fifoinfo = 0x0},
v_hashlist = {le_next = 0x0, le_prev = 0x0}
Re: panic: LK_RETRY set with incompatible flags
On 4 February 2013 16:06, Andriy Gapon wrote:
> on 04/02/2013 13:49 Sergey Kandaurov said the following:
>> Hi, Rick! Here is the requested info regarding witness, and a bit more.
>> The triggered KASSERT is now different though.
>
> It's exactly the same problem though :-)
> Do you have a crashdump?
> If yes, please print **vpp.
Yep, but it's in a bad form :(
It has many bits optimized out (compiled with clang).
I'll rebuild the kernel with -O or so and will try again.
#8 0x808bc4ce in kdb_enter (why=0x80e7ed99 "panic",
msg=) at cpufunc.h:63
#9 0x80888fb7 in vpanic (fmt=,
ap=) at /usr/src/sys/kern/kern_shutdown.c:746
#10 0x80888e66 in kassert_panic (fmt=)
at /usr/src/sys/kern/kern_shutdown.c:641
#11 0x808d2259 in witness_checkorder (lock=0xfe00a1b44240,
---Type to continue, or q to quit---
flags=1,
file=0x81b2bb36
"/usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c",
line=1452,
interlock=) at /usr/src/sys/kern/subr_witness.c:1122
#12 0x8086c11e in __lockmgr_args (lk=0xfe00a1b44240,
flags=, ilk=0xfe00a1b44270,
wmesg=0x81b2808d "zfs", pri=96, timo=51,
file=0x80e8e407 "/usr/src/sys/kern/vfs_default.c", line=0)
at /usr/src/sys/kern/kern_lock.c:511
#13 0x8091439c in vop_stdlock (ap=)
at lockmgr.h:97
#14 0x80cb70c0 in VOP_LOCK1_APV (vop=,
a=) at vnode_if.c:2022
#15 0x80932fbb in _vn_lock (vp=0xfe00a1b441d8,
flags=,
file=0x81b2bb36
"/usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c",
line=1452) at vnode_if.h:859
#16 0x81abd902 in zfs_lookup ()
at
/usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:1452
#17 0x81abdc1d in zfs_freebsd_lookup (ap=0xff848e6c0270)
at
/usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:5864
---Type to continue, or q to quit---
#18 0x80cb51b2 in VOP_CACHEDLOOKUP_APV (vop=,
a=) at vnode_if.c:191
#19 0x8091177f in vfs_cache_lookup (ap=)
at vnode_if.h:80
#20 0x80cb50a2 in VOP_LOOKUP_APV (vop=,
a=) at vnode_if.c:125
#21 0x80919658 in lookup (ndp=0xff848e6c05d8) at vnode_if.h:54
#22 0x807dffe5 in nfsvno_namei (nd=0xff848e6c08c8,
ndp=0xff848e6c05d8, dp=,
islocked=, exp=,
p=, retdirp=)
at /usr/src/sys/fs/nfsserver/nfs_nfsdport.c:413
#23 0x807d8ffa in nfsrvd_lookup (nd=0xff848e6c08c8,
isdgram=, dp=0xfe00a1b441d8,
vpp=0xff848e6c0810, fhp=0xff848e6c07f4, p=0xfe00a1198000,
exp=0xff848e6c07a0) at /usr/src/sys/fs/nfsserver/nfs_nfsdserv.c:517
#24 0x807cb845 in nfsrvd_dorpc (nd=0xff848e6c08c8, isdgram=0,
p=0xfe00a1198000) at /usr/src/sys/fs/nfsserver/nfs_nfsdsocket.c:823
#25 0x807d7af2 in nfssvc_program (rqst=0xfe00a17bb000,
xprt=0xfe00a124b200) at /usr/src/sys/fs/nfsserver/nfs_nfsdkrpc.c:347
#26 0x80a70659 in svc_run_internal (pool=0xfe00067db600,
ismaster=0) at /usr/src/sys/rpc/svc.c:895
#27 0x80a7155b in svc_thread_start (arg=0xff848e6bfc50)
---Type to continue, or q to quit---
at /usr/src/sys/rpc/svc.c:1200
#28 0x80858944 in fork_exit (
callout=0x80a71550 , arg=0xfe00067db600,
frame=0xff848e6c0c00) at /usr/src/sys/kern/kern_fork.c:991
#29 0x80bfa86e in fork_trampoline () at exception.S:602
#30 0x0080 in ?? ()
#31 0x7fffd820 in ?? ()
#32 0x0001 in ?? ()
#33 0x in ?? ()
Current language: auto; currently minimal
(kgdb) p vpp
Cannot access memory at address 0x0
(kgdb) p dvp
Cannot access memory at address 0x0
>
>> Full witness is at
>> http://people.freebsd.org/~pluknet/witness-zfs-20130204.txt
>>
>> shared lock of (lockmgr) zfs @
>> /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:1452
>> while exclusively locked from
>> /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c:1747
>> panic: share->excl
>> cpuid = 2
>> KDB: enter: panic
>> [ thread pid 812 tid 100884 ]
>> Stopped at kdb_enter+0x3e: movq$0,kdb_why
>>
>> The 1st line is at zfs_lookup():
>> if (error == 0 && (nm[0] != '.' || nm[1] != '\0')) {
>> int ltype = 0;
>>
>> if (cnp->cn_flags & ISDOTDOT) {
>> ltype = VOP_ISLOCKED(dvp);
>> VOP_UNLOCK(dvp, 0);
>> }
>> ZFS_EXIT(zfsvfs);
>> error = zfs_vnode_lock(*vpp, cnp->cn_lkf
Re: panic: LK_RETRY set with incompatible flags
On 4 February 2013 05:50, Rick Macklem wrote:
> Konstantin Belousov wrote:
>> On Sat, Feb 02, 2013 at 09:30:39PM -0500, Rick Macklem wrote:
>> > Andriy Gapon wrote:
>> > > on 31/01/2013 15:29 Sergey Kandaurov said the following:
>> > > > Hi.
>> > > >
>> > > > Got this assertion on idle NFS server while `ls -la
>> > > > /.zfs/shares/'
>> > > > issued on NFS client.
>> > > > kern/vfs_vnops.c:_vn_lock()
>> > > > KASSERT((flags & LK_RETRY) == 0 || error == 0,
>> > > > ("LK_RETRY set with incompatible flags
>> > > > (0x%x) or
>> > > > an error occured (%d)",
>> > > >
>> > > > panic: LK_RETRY set with incompatible flags (0x200400) or an
>> > > > error
>> > > > occured (11)
>> > > >
>> > > > What does that mean and how is it possible? As you can see, both
>> > > > parts
>> > > > of assertion failed.
>> > > > 11 is EDEADLK
>> > > > 0x200400: LK_RETRY & LK_UPGRADE
>> > >
>> > > LK_SHARED, not LK_UPGRADE.
>> > > Apparently the thread already holds an exlusive lock on the vnode,
>> > > which you
>> > > confirm below.
>> > >
>> > >
>> > > > Tracing pid 2943 tid 101532 td 0xfe004f5b7000
>> > > > kdb_enter() at kdb_enter+0x3e/frame 0xff848e45ef50
>> > > > vpanic() at vpanic+0x147/frame 0xff848e45ef90
>> > > > kassert_panic() at kassert_panic+0x136/frame 0xff848e45f000
>> > > > _vn_lock() at _vn_lock+0x70/frame 0xff848e45f070
>> > > > zfs_lookup() at zfs_lookup+0x392/frame 0xff848e45f100
>> > > > zfs_freebsd_lookup() at zfs_freebsd_lookup+0x6d/frame
>> > > > 0xff848e45f240
>> > > > VOP_CACHEDLOOKUP_APV() at VOP_CACHEDLOOKUP_APV+0xc2/frame
>> > > > 0xff848e45f260
>> > > > vfs_cache_lookup() at vfs_cache_lookup+0xcf/frame
>> > > > 0xff848e45f2b0
>> > > > VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xc2/frame 0xff848e45f2d0
>> > > > lookup() at lookup+0x548/frame 0xff848e45f350
>> > > > nfsvno_namei() at nfsvno_namei+0x1a5/frame 0xff848e45f400
>> > > > nfsrvd_lookup() at nfsrvd_lookup+0x13a/frame 0xff848e45f6b0
>> > > > nfsrvd_dorpc() at nfsrvd_dorpc+0xca5/frame 0xff848e45f8a0
>> > > > nfssvc_program() at nfssvc_program+0x482/frame
>> > > > 0xff848e45fa00
>> > > > svc_run_internal() at svc_run_internal+0x1e9/frame
>> > > > 0xff848e45fba0
>> > > > svc_thread_start() at svc_thread_start+0xb/frame
>> > > > 0xff848e45fbb0
>> > > > fork_exit() at fork_exit+0x84/frame 0xff848e45fbf0
>> > > > fork_trampoline() at fork_trampoline+0xe/frame
>> > > > 0xff848e45fbf0
>> > > > --- trap 0xc, rip = 0x800883e9a, rsp = 0x7fffd488, rbp =
>> > > > 0x7fffd730 ---
>> > > >
>> > > > db> show lockedvnods
>> > > > Locked vnodes
>> > > >
>> > > > 0xfe02e21b11d8: tag zfs, type VDIR
>> > > > usecount 4, writecount 0, refcount 4 mountedhere 0
>> > > > flags (VI_ACTIVE)
>> > > > v_object 0xfe02d9f2eb40 ref 0 pages 0
>> > > > lock type zfs: EXCL by thread 0xfe004f5b7000 (pid 2943,
>> > > > nfsd,
>> > > > tid 101532)
>> > > >
>> > > >
>> > > >
>> > I just took a look at zfs_vnops.c and I am probably missing
>> > something,
>> > but I can't see how this ever worked for a lookup of ".." when at
>> > the
>> > root (unless ZFS doesn't do the ".." is the current directory when
>> > at
>> > the root).
>> >
>> > Here's the code snippet:
>> > 1442 if (error == 0 && (nm[0] != '.' || nm[1] != '\0')) {
>> > 1443 int ltype = 0;
>> > 1444
>> > 1445 if (cnp->cn_flags & ISDOTDOT) {
>> > 1446 ltype = VOP_ISLOCKED(dvp);
>> > 1447 VOP_UNLOCK(dvp, 0);
>> > 1448 }
>> > 1449 ZFS_EXIT(zfsvfs);
>> > 1450 error = zfs_vnode_lock(*vpp, cnp->cn_lkflags);
>> > 1451 if (cnp->cn_flags & ISDOTDOT)
>> > 1452 vn_lock(dvp, ltype | LK_RETRY);
>> > 1453 if (error != 0) {
>> > 1454 VN_RELE(*vpp);
>> > 1455 *vpp = NULL;
>> > 1456 return (error);
>> > 1457 }
>> >
>> > Maybe line# 1451 should be changed to:
>> > if ((cnp->cn_flags & ISDOTDOT) && *vpp != dvp)
>> >
>> > I'm not at all familiar with ZFS, so I've probably way
>> > off the mark on this, rick
>> > ps: I hope kib and jhb don't mind being added as cc's, since
>> > they are familiar with this stuff, although maybe not ZFS
>> > specifics.
>>
>> VFS (should) never call VOP_LOOKUP for the dotdot and root vnode.
>> The logic in the lookup() prevents it.
>>
> Correcting my previous posts, like usual. If you look at the above snippet of
> code, it seems that zfs_lock_vnode() must lock the same vnode as
> *dvp. Notice that vn_lock() is only called when ISDOTDOT is set and the
> code does a VOP_UNLOCK(dvp, 0); for this case, just before the
> zfs_vnode_lock().
>
> This assumes that the vn_lock() call at #1452 causes the panic.
> This is the only vn_lock() call in zfs_lookup(), so unless the compiler
> has done something weird, it seems the case.
>
> Sergey, do you have this kernel handy? If so, maybe you could check the
> line# for zfs_lookup+0x392. (If you haven't done this before, just email
> and I'll give you the steps. You jus
Re: panic: LK_RETRY set with incompatible flags
on 04/02/2013 13:49 Sergey Kandaurov said the following:
> Hi, Rick! Here is the requested info regarding witness, and a bit more.
> The triggered KASSERT is now different though.
It's exactly the same problem though :-)
Do you have a crashdump?
If yes, please print **vpp.
> Full witness is at http://people.freebsd.org/~pluknet/witness-zfs-20130204.txt
>
> shared lock of (lockmgr) zfs @
> /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:1452
> while exclusively locked from
> /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c:1747
> panic: share->excl
> cpuid = 2
> KDB: enter: panic
> [ thread pid 812 tid 100884 ]
> Stopped at kdb_enter+0x3e: movq$0,kdb_why
>
> The 1st line is at zfs_lookup():
> if (error == 0 && (nm[0] != '.' || nm[1] != '\0')) {
> int ltype = 0;
>
> if (cnp->cn_flags & ISDOTDOT) {
> ltype = VOP_ISLOCKED(dvp);
> VOP_UNLOCK(dvp, 0);
> }
> ZFS_EXIT(zfsvfs);
> error = zfs_vnode_lock(*vpp, cnp->cn_lkflags);
> if (cnp->cn_flags & ISDOTDOT)
> ==> vn_lock(dvp, ltype | LK_RETRY);
> if (error != 0) {
> VN_RELE(*vpp);
> *vpp = NULL;
> return (error);
> }
> } else {
> ZFS_EXIT(zfsvfs);
> }
>
> The 2nd line is at zfs_vnode_lock():
> int
> zfs_vnode_lock(vnode_t *vp, int flags)
> {
> int error;
>
> ASSERT(vp != NULL);
>
> error = vn_lock(vp, flags);
> return (error);
> }
>
> db> show locks
> exclusive lockmgr zfs (zfs) r = 0 (0xfe00a1b44240) locked @
> /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c:1747
> db> show alllocks
> Process 812 (nfsd) thread 0xfe00a1198000 (100884)
> exclusive lockmgr zfs (zfs) r = 0 (0xfe00a1b44240) locked @
> /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c:1747
> Process 750 (syslogd) thread 0xfe0015a4c480 (100706)
> exclusive lockmgr ufs (ufs) r = 0 (0xfe00a1962d50) locked @
> /usr/src/sys/kern/vfs_syscalls.c:3433
> Process 12 (intr) thread 0xfe0006813000 (100033)
> exclusive sleep mutex AAC I/O lock (AAC I/O lock) r = 0
> (0xff8001bfb210) locked @ /usr/src/sys/dev/aac/aac.c:827
>
> db> show lock 0xfe00a1b44240
> class: lockmgr
> name: zfs
> state: XLOCK: 0xfe00a1198000 (tid 100884, pid 812, "nfsd")
> waiters: none
> spinners: none
>
> As KASSERT is different:
>
> db> bt
> Tracing pid 812 tid 100884 td 0xfe00a1198000
> kdb_enter() at kdb_enter+0x3e/frame 0xff848e6bfd60
> vpanic() at vpanic+0x147/frame 0xff848e6bfda0
> kassert_panic() at kassert_panic+0x136/frame 0xff848e6bfe10
> witness_checkorder() at witness_checkorder+0x289/frame 0xff848e6bfe90
> __lockmgr_args() at __lockmgr_args+0x43e/frame 0xff848e6bffc0
> vop_stdlock() at vop_stdlock+0x3c/frame 0xff848e6bffe0
> VOP_LOCK1_APV() at VOP_LOCK1_APV+0xd0/frame 0xff848e6c
> _vn_lock() at _vn_lock+0xab/frame 0xff848e6c0070
> zfs_lookup() at zfs_lookup+0x392/frame 0xff848e6c0100
> zfs_freebsd_lookup() at zfs_freebsd_lookup+0x6d/frame 0xff848e6c0240
> VOP_CACHEDLOOKUP_APV() at VOP_CACHEDLOOKUP_APV+0xc2/frame 0xff848e6c0260
> vfs_cache_lookup() at vfs_cache_lookup+0xcf/frame 0xff848e6c02b0
> VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xc2/frame 0xff848e6c02d0
> lookup() at lookup+0x548/frame 0xff848e6c0350
> nfsvno_namei() at nfsvno_namei+0x1a5/frame 0xff848e6c0400
> nfsrvd_lookup() at nfsrvd_lookup+0x13a/frame 0xff848e6c06b0
> nfsrvd_dorpc() at nfsrvd_dorpc+0xca5/frame 0xff848e6c08a0
> nfssvc_program() at nfssvc_program+0x482/frame 0xff848e6c0a00
> svc_run_internal() at svc_run_internal+0x1e9/frame 0xff848e6c0ba0
> svc_thread_start() at svc_thread_start+0xb/frame 0xff848e6c0bb0
> fork_exit() at fork_exit+0x84/frame 0xff848e6c0bf0
> fork_trampoline() at fork_trampoline+0xe/frame 0xff848e6c0bf0
> --- trap 0xc, rip = 0x800883b7a, rsp = 0x7fffd6c8, rbp = 0x7fffd970
> ---
>
--
Andriy Gapon
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: panic: LK_RETRY set with incompatible flags
On 4 February 2013 05:07, Rick Macklem wrote:
> Andriy Gapon wrote:
>> on 03/02/2013 18:36 Rick Macklem said the following:
>> > I can think of two possibilities:
>> > 1 - ZFS isn't setting VV_ROOT on the root vnode under some
>> > condition.
>> > or
>> > 2 - The vnode was left locked from some previous operation that
>> > happened
>> > to be done by this thread. Doesn't seem likely, but???
>> >
>> > Maybe Sergey could try the change to line#1451 and see if the panic
>> > still happens. If not, that would suggest possibility #1, I think.
>>
>> If the kernel is configured with witness, then it should be easy to
>> check where
>> the exclusive lock was taken (file and line number).
>>
> Yep. If Sergey can reproduce this using a kernel with witness,
> doing "show witness" to see where the lock on the directory vnode
> was acquired, could be helpful.
Hi, Rick! Here is the requested info regarding witness, and a bit more.
The triggered KASSERT is now different though.
Full witness is at http://people.freebsd.org/~pluknet/witness-zfs-20130204.txt
shared lock of (lockmgr) zfs @
/usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:1452
while exclusively locked from
/usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c:1747
panic: share->excl
cpuid = 2
KDB: enter: panic
[ thread pid 812 tid 100884 ]
Stopped at kdb_enter+0x3e: movq$0,kdb_why
The 1st line is at zfs_lookup():
if (error == 0 && (nm[0] != '.' || nm[1] != '\0')) {
int ltype = 0;
if (cnp->cn_flags & ISDOTDOT) {
ltype = VOP_ISLOCKED(dvp);
VOP_UNLOCK(dvp, 0);
}
ZFS_EXIT(zfsvfs);
error = zfs_vnode_lock(*vpp, cnp->cn_lkflags);
if (cnp->cn_flags & ISDOTDOT)
==> vn_lock(dvp, ltype | LK_RETRY);
if (error != 0) {
VN_RELE(*vpp);
*vpp = NULL;
return (error);
}
} else {
ZFS_EXIT(zfsvfs);
}
The 2nd line is at zfs_vnode_lock():
int
zfs_vnode_lock(vnode_t *vp, int flags)
{
int error;
ASSERT(vp != NULL);
error = vn_lock(vp, flags);
return (error);
}
db> show locks
exclusive lockmgr zfs (zfs) r = 0 (0xfe00a1b44240) locked @
/usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c:1747
db> show alllocks
Process 812 (nfsd) thread 0xfe00a1198000 (100884)
exclusive lockmgr zfs (zfs) r = 0 (0xfe00a1b44240) locked @
/usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c:1747
Process 750 (syslogd) thread 0xfe0015a4c480 (100706)
exclusive lockmgr ufs (ufs) r = 0 (0xfe00a1962d50) locked @
/usr/src/sys/kern/vfs_syscalls.c:3433
Process 12 (intr) thread 0xfe0006813000 (100033)
exclusive sleep mutex AAC I/O lock (AAC I/O lock) r = 0
(0xff8001bfb210) locked @ /usr/src/sys/dev/aac/aac.c:827
db> show lock 0xfe00a1b44240
class: lockmgr
name: zfs
state: XLOCK: 0xfe00a1198000 (tid 100884, pid 812, "nfsd")
waiters: none
spinners: none
As KASSERT is different:
db> bt
Tracing pid 812 tid 100884 td 0xfe00a1198000
kdb_enter() at kdb_enter+0x3e/frame 0xff848e6bfd60
vpanic() at vpanic+0x147/frame 0xff848e6bfda0
kassert_panic() at kassert_panic+0x136/frame 0xff848e6bfe10
witness_checkorder() at witness_checkorder+0x289/frame 0xff848e6bfe90
__lockmgr_args() at __lockmgr_args+0x43e/frame 0xff848e6bffc0
vop_stdlock() at vop_stdlock+0x3c/frame 0xff848e6bffe0
VOP_LOCK1_APV() at VOP_LOCK1_APV+0xd0/frame 0xff848e6c
_vn_lock() at _vn_lock+0xab/frame 0xff848e6c0070
zfs_lookup() at zfs_lookup+0x392/frame 0xff848e6c0100
zfs_freebsd_lookup() at zfs_freebsd_lookup+0x6d/frame 0xff848e6c0240
VOP_CACHEDLOOKUP_APV() at VOP_CACHEDLOOKUP_APV+0xc2/frame 0xff848e6c0260
vfs_cache_lookup() at vfs_cache_lookup+0xcf/frame 0xff848e6c02b0
VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xc2/frame 0xff848e6c02d0
lookup() at lookup+0x548/frame 0xff848e6c0350
nfsvno_namei() at nfsvno_namei+0x1a5/frame 0xff848e6c0400
nfsrvd_lookup() at nfsrvd_lookup+0x13a/frame 0xff848e6c06b0
nfsrvd_dorpc() at nfsrvd_dorpc+0xca5/frame 0xff848e6c08a0
nfssvc_program() at nfssvc_program+0x482/frame 0xff848e6c0a00
svc_run_internal() at svc_run_internal+0x1e9/frame 0xff848e6c0ba0
svc_thread_start() at svc_thread_start+0xb/frame 0xff848e6c0bb0
fork_exit() at fork_exit+0x84/frame 0xff848e6c0bf0
fork_trampoline() at fork_trampoline+0xe/frame 0xff848e6c0bf0
--- trap 0xc, rip = 0x800883b7a, rsp = 0x7fffd6c8, rbp = 0x
