Dominique Martinet wrote on Tue, Jan 17, 2017 at 11:14:04AM +0100:
> Here's the backtrace I have:
>
> ganesha.nfsd: /export/nfs-ganesha/src/SAL/state_misc.c:917:
> dec_state_owner_ref: Assertion `refcount > 0' failed.
>
> Thread 46 "ganesha.nfsd" received signal SIGABRT, Aborted.
> [Switching to Thread 0x7fffd89fe700 (LWP 4321)]
> 0x00007ffff53346f5 in raise () from /lib64/libc.so.6
> #0 0x00007ffff53346f5 in raise () from /lib64/libc.so.6
> #1 0x00007ffff53362fa in abort () from /lib64/libc.so.6
> #2 0x00007ffff532cf97 in __assert_fail_base () from /lib64/libc.so.6
> #3 0x00007ffff532d042 in __assert_fail () from /lib64/libc.so.6
> #4 0x00000000004c0281 in dec_state_owner_ref (owner=0x7fffd5662620) at
> /export/nfs-ganesha/src/SAL/state_misc.c:917
> #5 0x00000000004c05eb in uncache_nfs4_owner (nfs4_owner=0x7fffd5662670)
> at /export/nfs-ganesha/src/SAL/state_misc.c:1009
> #6 0x0000000000455deb in reap_expired_open_owners () at
> /export/nfs-ganesha/src/MainNFSD/nfs_reaper_thread.c:183
> #7 0x0000000000456039 in reaper_run (ctx=0x7ffff03e6e80) at
> /export/nfs-ganesha/src/MainNFSD/nfs_reaper_thread.c:247
> #8 0x00000000005026c9 in fridgethr_start_routine (arg=0x7ffff03e6e80)
> at /export/nfs-ganesha/src/support/fridgethr.c:550
> #9 0x00007ffff79b55ca in start_thread () from /lib64/libpthread.so.0
> #10 0x00007ffff5402f6d in clone () from /lib64/libc.so.6
Here's the last few lines of logs, I kept the last ~100MB of full debug
logs if you want more (12MB compressed so won't send on the list)
17/01/2017 10:37:36 4526[reaper] /export/nfs-ganesha/src/SAL/state_misc.c:1001
:uncache_nfs4_owner :STATE :F_DBG :Uncache {STATE_OPEN_OWNER_NFSV4
0x7fffd633b200: clientid={0x7fffd53bed00 ClientID={Epoch=0x587df1f2
Counter=0x000000f1} CONFIRMED Client={0x7fffd5c562a0
name=(27:Environment.init_1484649396) refcount=1} t_delta=59 reservations=0
refcount=4} owner=(8:maketree) confirmed=0 seqid=0 cached(expires in 0 secs)
refcount=1}
17/01/2017 10:37:36 4526[reaper] /export/nfs-ganesha/src/SAL/nfs4_owner.c:277
:nfs4_owner_value_hash_func :STATE :F_DBG :value = 10
17/01/2017 10:37:36 4526[reaper] /export/nfs-ganesha/src/SAL/nfs4_owner.c:313
:nfs4_owner_rbt_hash_func :STATE :F_DBG :rbt = 6376518669110871114
17/01/2017 10:37:36 4526[reaper]
/export/nfs-ganesha/src/hashtable/hashtable.c:460 :hashtable_getlatch :RW LOCK
:F_DBG :Got write lock on 0x7ffff2ff8c98 (&(ht->partitions[index].lock)) at
/export/nfs-ganesha/src/hashtable/hashtable.c:460
17/01/2017 10:37:36 4526[reaper]
/export/nfs-ganesha/src/hashtable/hashtable.c:169 :key_locate :HT CACHE :F_DBG
:hash miss index 10 slot 4103
17/01/2017 10:37:36 4526[reaper] /export/nfs-ganesha/src/SAL/nfs4_owner.c:233
:compare_nfs4_owner_key :STATE :F_DBG :{STATE_OPEN_OWNER_NFSV4 0x7fffd633b200:
clientid={0x7fffd53bed00 ClientID={Epoch=0x587df1f2 Counter=0x000000f1}
CONFIRMED Client={0x7fffd5c562a0 name=(27:Environment.init_1484649396)
refcount=1} t_delta=59 reservations=0 refcount=4} owner=(8:maketree)
confirmed=0 seqid=0 refcount=0} vs {STATE_OPEN_OWNER_NFSV4 0x7fffd633b200:
clientid={0x7fffd53bed00 ClientID={Epoch=0x587df1f2 Counter=0x000000f1}
CONFIRMED Client={0x7fffd5c562a0 name=(27:Environment.init_1484649396)
refcount=1} t_delta=59 reservations=0 refcount=4} owner=(8:maketree)
confirmed=0 seqid=0 refcount=0}
17/01/2017 10:37:36 4526[reaper] /export/nfs-ganesha/src/SAL/nfs4_owner.c:184
:compare_nfs4_owner :STATE :F_DBG :{STATE_OPEN_OWNER_NFSV4 0x7fffd633b200:
clientid={0x7fffd53bed00 ClientID={Epoch=0x587df1f2 Counter=0x000000f1}
CONFIRMED Client={0x7fffd5c562a0 name=(27:Environment.init_1484649396)
refcount=1} t_delta=59 reservations=0 refcount=4} owner=(8:maketree)
confirmed=0 seqid=0 refcount=0} vs {STATE_OPEN_OWNER_NFSV4 0x7fffd633b200:
clientid={0x7fffd53bed00 ClientID={Epoch=0x587df1f2 Counter=0x000000f1}
CONFIRMED Client={0x7fffd5c562a0 name=(27:Environment.init_1484649396)
refcount=1} t_delta=59 reservations=0 refcount=4} owner=(8:maketree)
confirmed=0 seqid=0 refcount=0}
17/01/2017 10:37:36 4526[reaper]
/export/nfs-ganesha/src/hashtable/hashtable.c:486 :hashtable_getlatch : :F_DBG
:Get (null) returning Value=0x7fffd633b200 {STATE_OPEN_OWNER_NFSV4
0x7fffd633b200: clientid={0x7fffd53bed00 ClientID={Epoch=0x587df1f2
Counter=0x000000f1} CONFIRMED Client={0x7fffd5c562a0
name=(27:Environment.init_1484649396) refcount=1} t_delta=59 reservations=0
refcount=4} owner=(8:maketree) confirmed=0 seqid=0 refcount=0}
17/01/2017 10:37:36 4526[reaper]
/export/nfs-ganesha/src/hashtable/hashtable.c:735 :hashtable_deletelatched :
:F_DBG :Delete (null) Key=0x7fffd633b200 {STATE_OPEN_OWNER_NFSV4
0x7fffd633b200: clientid={0x7fffd53bed00 ClientID={Epoch=0x587df1f2
Counter=0x000000f1} CONFIRMED Client={0x7fffd5c562a0
name=(27:Environment.init_1484649396) refcount=1} t_delta=59 reservations=0
refcount=4} owner=(8:maketree) confirmed=0 seqid=0 refcount=0}
Value=0x7fffd633b200 {STATE_OPEN_OWNER_NFSV4 0x7fffd633b200:
clientid={0x7fffd53bed00 ClientID={Epoch=0x587df1f2 Counter=0x000000f1}
CONFIRMED Client={0x7fffd5c562a0 name=(27:Environment.init_1484649396)
refcount=1} t_delta=59 reservations=0 refcount=4} owner=(8:maketree)
confirmed=0 seqid=0 refcount=0} index=10 rbt_hash=6376518669110871114 was
removed
17/01/2017 10:37:36 4526[reaper]
/export/nfs-ganesha/src/hashtable/hashtable.c:764 :hashtable_deletelatched :HT
CACHE :F_DBG :hash clear slot 4103
17/01/2017 10:37:36 4526[reaper]
/export/nfs-ganesha/src/hashtable/hashtable.c:526 :hashtable_releaselatched :RW
LOCK :F_DBG :Unlocked 0x7ffff2ff8c98 (&ht->partitions[latch->index].lock) at
/export/nfs-ganesha/src/hashtable/hashtable.c:526
17/01/2017 10:37:36 4526[reaper] /export/nfs-ganesha/src/SAL/state_misc.c:966
:dec_state_owner_ref :STATE :F_DBG :Free {STATE_OPEN_OWNER_NFSV4
0x7fffd633b200: clientid={0x7fffd53bed00 ClientID={Epoch=0x587df1f2
Counter=0x000000f1} CONFIRMED Client={0x7fffd5c562a0
name=(27:Environment.init_1484649396) refcount=1} t_delta=59 reservations=0
refcount=4} owner=(8:maketree) confirmed=0 seqid=0 refcount=1}
17/01/2017 10:37:36 4526[reaper] /export/nfs-ganesha/src/SAL/nfs4_owner.c:335
:free_nfs4_owner :RW LOCK :F_DBG :Acquired mutex 0x7fffd53bed90
(&owner->so_owner.so_nfs4_owner.so_clientrec ->cid_mutex) at
/export/nfs-ganesha/src/SAL/nfs4_owner.c:335
17/01/2017 10:37:36 4526[reaper] /export/nfs-ganesha/src/SAL/nfs4_owner.c:340
:free_nfs4_owner :RW LOCK :F_DBG :Released mutex 0x7fffd53bed90
(&owner->so_owner.so_nfs4_owner.so_clientrec ->cid_mutex) at
/export/nfs-ganesha/src/SAL/nfs4_owner.c:340
17/01/2017 10:37:36 4526[reaper]
/export/nfs-ganesha/src/SAL/nfs4_clientid.c:375 :dec_client_id_ref :CLIENT ID
:F_DBG :Decrement refcount Clientid {0x7fffd53bed00 ClientID={Epoch=0x587df1f2
Counter=0x000000f1} CONFIRMED Client={0x7fffd5c562a0
name=(27:Environment.init_1484649396) refcount=1} t_delta=59 reservations=0
refcount=4} refcount to 3
17/01/2017 10:37:36 4526[reaper] /export/nfs-ganesha/src/SAL/state_misc.c:843
:free_state_owner :RW LOCK :F_DBG :Destroy mutex 0x7fffd633b218
(&owner->so_mutex) at /export/nfs-ganesha/src/SAL/state_misc.c:843
17/01/2017 10:37:36 4526[reaper] /export/nfs-ganesha/src/SAL/state_misc.c:1001
:uncache_nfs4_owner :STATE :F_DBG :Uncache {STATE_LOCK_OWNER_UNKNOWN
powner=0x7fffd7c30420: refcount=0}
17/01/2017 10:37:36 4526[reaper] /export/nfs-ganesha/src/SAL/state_misc.c:915
:dec_state_owner_ref :STATE :F_DBG :Decrement refcount now=-1
{STATE_LOCK_OWNER_UNKNOWN powner=0x7fffd7c30420: refcount=0}
Could print more stuff from powner if anything would help, I honestly
have no idea about states life cycle or even how to get if it's related
to a file in particular, sorry.
--
Dominique
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Nfs-ganesha-devel mailing list
Nfs-ganesha-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel
