Could you send the log directly to me?
Thanks,
Daniel
On 01/17/2017 05:55 AM, Dominique Martinet wrote:
> Dominique Martinet wrote on Tue, Jan 17, 2017 at 11:14:04AM +0100:
>> Here's the backtrace I have:
>>
>> ganesha.nfsd: /export/nfs-ganesha/src/SAL/state_misc.c:917:
>> dec_state_owner_ref: Assertion `refcount > 0' failed.
>>
>> Thread 46 "ganesha.nfsd" received signal SIGABRT, Aborted.
>> [Switching to Thread 0x7fffd89fe700 (LWP 4321)]
>> 0x00007ffff53346f5 in raise () from /lib64/libc.so.6
>> #0 0x00007ffff53346f5 in raise () from /lib64/libc.so.6
>> #1 0x00007ffff53362fa in abort () from /lib64/libc.so.6
>> #2 0x00007ffff532cf97 in __assert_fail_base () from /lib64/libc.so.6
>> #3 0x00007ffff532d042 in __assert_fail () from /lib64/libc.so.6
>> #4 0x00000000004c0281 in dec_state_owner_ref (owner=0x7fffd5662620) at
>> /export/nfs-ganesha/src/SAL/state_misc.c:917
>> #5 0x00000000004c05eb in uncache_nfs4_owner (nfs4_owner=0x7fffd5662670)
>> at /export/nfs-ganesha/src/SAL/state_misc.c:1009
>> #6 0x0000000000455deb in reap_expired_open_owners () at
>> /export/nfs-ganesha/src/MainNFSD/nfs_reaper_thread.c:183
>> #7 0x0000000000456039 in reaper_run (ctx=0x7ffff03e6e80) at
>> /export/nfs-ganesha/src/MainNFSD/nfs_reaper_thread.c:247
>> #8 0x00000000005026c9 in fridgethr_start_routine (arg=0x7ffff03e6e80)
>> at /export/nfs-ganesha/src/support/fridgethr.c:550
>> #9 0x00007ffff79b55ca in start_thread () from /lib64/libpthread.so.0
>> #10 0x00007ffff5402f6d in clone () from /lib64/libc.so.6
>
>
> Here's the last few lines of logs, I kept the last ~100MB of full debug
> logs if you want more (12MB compressed so won't send on the list)
>
> 17/01/2017 10:37:36 4526[reaper]
> /export/nfs-ganesha/src/SAL/state_misc.c:1001 :uncache_nfs4_owner :STATE
> :F_DBG :Uncache {STATE_OPEN_OWNER_NFSV4 0x7fffd633b200:
> clientid={0x7fffd53bed00 ClientID={Epoch=0x587df1f2 Counter=0x000000f1}
> CONFIRMED Client={0x7fffd5c562a0 name=(27:Environment.init_1484649396)
> refcount=1} t_delta=59 reservations=0 refcount=4} owner=(8:maketree)
> confirmed=0 seqid=0 cached(expires in 0 secs) refcount=1}
> 17/01/2017 10:37:36 4526[reaper] /export/nfs-ganesha/src/SAL/nfs4_owner.c:277
> :nfs4_owner_value_hash_func :STATE :F_DBG :value = 10
> 17/01/2017 10:37:36 4526[reaper] /export/nfs-ganesha/src/SAL/nfs4_owner.c:313
> :nfs4_owner_rbt_hash_func :STATE :F_DBG :rbt = 6376518669110871114
> 17/01/2017 10:37:36 4526[reaper]
> /export/nfs-ganesha/src/hashtable/hashtable.c:460 :hashtable_getlatch :RW
> LOCK :F_DBG :Got write lock on 0x7ffff2ff8c98 (&(ht->partitions[index].lock))
> at /export/nfs-ganesha/src/hashtable/hashtable.c:460
> 17/01/2017 10:37:36 4526[reaper]
> /export/nfs-ganesha/src/hashtable/hashtable.c:169 :key_locate :HT CACHE
> :F_DBG :hash miss index 10 slot 4103
> 17/01/2017 10:37:36 4526[reaper] /export/nfs-ganesha/src/SAL/nfs4_owner.c:233
> :compare_nfs4_owner_key :STATE :F_DBG :{STATE_OPEN_OWNER_NFSV4
> 0x7fffd633b200: clientid={0x7fffd53bed00 ClientID={Epoch=0x587df1f2
> Counter=0x000000f1} CONFIRMED Client={0x7fffd5c562a0
> name=(27:Environment.init_1484649396) refcount=1} t_delta=59 reservations=0
> refcount=4} owner=(8:maketree) confirmed=0 seqid=0 refcount=0} vs
> {STATE_OPEN_OWNER_NFSV4 0x7fffd633b200: clientid={0x7fffd53bed00
> ClientID={Epoch=0x587df1f2 Counter=0x000000f1} CONFIRMED
> Client={0x7fffd5c562a0 name=(27:Environment.init_1484649396) refcount=1}
> t_delta=59 reservations=0 refcount=4} owner=(8:maketree) confirmed=0 seqid=0
> refcount=0}
> 17/01/2017 10:37:36 4526[reaper] /export/nfs-ganesha/src/SAL/nfs4_owner.c:184
> :compare_nfs4_owner :STATE :F_DBG :{STATE_OPEN_OWNER_NFSV4 0x7fffd633b200:
> clientid={0x7fffd53bed00 ClientID={Epoch=0x587df1f2 Counter=0x000000f1}
> CONFIRMED Client={0x7fffd5c562a0 name=(27:Environment.init_1484649396)
> refcount=1} t_delta=59 reservations=0 refcount=4} owner=(8:maketree)
> confirmed=0 seqid=0 refcount=0} vs {STATE_OPEN_OWNER_NFSV4 0x7fffd633b200:
> clientid={0x7fffd53bed00 ClientID={Epoch=0x587df1f2 Counter=0x000000f1}
> CONFIRMED Client={0x7fffd5c562a0 name=(27:Environment.init_1484649396)
> refcount=1} t_delta=59 reservations=0 refcount=4} owner=(8:maketree)
> confirmed=0 seqid=0 refcount=0}
> 17/01/2017 10:37:36 4526[reaper]
> /export/nfs-ganesha/src/hashtable/hashtable.c:486 :hashtable_getlatch :
> :F_DBG :Get (null) returning Value=0x7fffd633b200 {STATE_OPEN_OWNER_NFSV4
> 0x7fffd633b200: clientid={0x7fffd53bed00 ClientID={Epoch=0x587df1f2
> Counter=0x000000f1} CONFIRMED Client={0x7fffd5c562a0
> name=(27:Environment.init_1484649396) refcount=1} t_delta=59 reservations=0
> refcount=4} owner=(8:maketree) confirmed=0 seqid=0 refcount=0}
> 17/01/2017 10:37:36 4526[reaper]
> /export/nfs-ganesha/src/hashtable/hashtable.c:735 :hashtable_deletelatched :
> :F_DBG :Delete (null) Key=0x7fffd633b200 {STATE_OPEN_OWNER_NFSV4
> 0x7fffd633b200: clientid={0x7fffd53bed00 ClientID={Epoch=0x587df1f2
> Counter=0x000000f1} CONFIRMED Client={0x7fffd5c562a0
> name=(27:Environment.init_1484649396) refcount=1} t_delta=59 reservations=0
> refcount=4} owner=(8:maketree) confirmed=0 seqid=0 refcount=0}
> Value=0x7fffd633b200 {STATE_OPEN_OWNER_NFSV4 0x7fffd633b200:
> clientid={0x7fffd53bed00 ClientID={Epoch=0x587df1f2 Counter=0x000000f1}
> CONFIRMED Client={0x7fffd5c562a0 name=(27:Environment.init_1484649396)
> refcount=1} t_delta=59 reservations=0 refcount=4} owner=(8:maketree)
> confirmed=0 seqid=0 refcount=0} index=10 rbt_hash=6376518669110871114 was
> removed
> 17/01/2017 10:37:36 4526[reaper]
> /export/nfs-ganesha/src/hashtable/hashtable.c:764 :hashtable_deletelatched
> :HT CACHE :F_DBG :hash clear slot 4103
> 17/01/2017 10:37:36 4526[reaper]
> /export/nfs-ganesha/src/hashtable/hashtable.c:526 :hashtable_releaselatched
> :RW LOCK :F_DBG :Unlocked 0x7ffff2ff8c98 (&ht->partitions[latch->index].lock)
> at /export/nfs-ganesha/src/hashtable/hashtable.c:526
> 17/01/2017 10:37:36 4526[reaper] /export/nfs-ganesha/src/SAL/state_misc.c:966
> :dec_state_owner_ref :STATE :F_DBG :Free {STATE_OPEN_OWNER_NFSV4
> 0x7fffd633b200: clientid={0x7fffd53bed00 ClientID={Epoch=0x587df1f2
> Counter=0x000000f1} CONFIRMED Client={0x7fffd5c562a0
> name=(27:Environment.init_1484649396) refcount=1} t_delta=59 reservations=0
> refcount=4} owner=(8:maketree) confirmed=0 seqid=0 refcount=1}
> 17/01/2017 10:37:36 4526[reaper] /export/nfs-ganesha/src/SAL/nfs4_owner.c:335
> :free_nfs4_owner :RW LOCK :F_DBG :Acquired mutex 0x7fffd53bed90
> (&owner->so_owner.so_nfs4_owner.so_clientrec ->cid_mutex) at
> /export/nfs-ganesha/src/SAL/nfs4_owner.c:335
> 17/01/2017 10:37:36 4526[reaper] /export/nfs-ganesha/src/SAL/nfs4_owner.c:340
> :free_nfs4_owner :RW LOCK :F_DBG :Released mutex 0x7fffd53bed90
> (&owner->so_owner.so_nfs4_owner.so_clientrec ->cid_mutex) at
> /export/nfs-ganesha/src/SAL/nfs4_owner.c:340
> 17/01/2017 10:37:36 4526[reaper]
> /export/nfs-ganesha/src/SAL/nfs4_clientid.c:375 :dec_client_id_ref :CLIENT ID
> :F_DBG :Decrement refcount Clientid {0x7fffd53bed00
> ClientID={Epoch=0x587df1f2 Counter=0x000000f1} CONFIRMED
> Client={0x7fffd5c562a0 name=(27:Environment.init_1484649396) refcount=1}
> t_delta=59 reservations=0 refcount=4} refcount to 3
> 17/01/2017 10:37:36 4526[reaper] /export/nfs-ganesha/src/SAL/state_misc.c:843
> :free_state_owner :RW LOCK :F_DBG :Destroy mutex 0x7fffd633b218
> (&owner->so_mutex) at /export/nfs-ganesha/src/SAL/state_misc.c:843
> 17/01/2017 10:37:36 4526[reaper]
> /export/nfs-ganesha/src/SAL/state_misc.c:1001 :uncache_nfs4_owner :STATE
> :F_DBG :Uncache {STATE_LOCK_OWNER_UNKNOWN powner=0x7fffd7c30420: refcount=0}
> 17/01/2017 10:37:36 4526[reaper] /export/nfs-ganesha/src/SAL/state_misc.c:915
> :dec_state_owner_ref :STATE :F_DBG :Decrement refcount now=-1
> {STATE_LOCK_OWNER_UNKNOWN powner=0x7fffd7c30420: refcount=0}
>
>
> Could print more stuff from powner if anything would help, I honestly
> have no idea about states life cycle or even how to get if it's related
> to a file in particular, sorry.
>
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Nfs-ganesha-devel mailing list
Nfs-ganesha-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel
