I was testing Ganesha NFS V2.4.0 using the POSIX FSAL on a CephFS file
system, using the Ceph FUSE interface, and I am seeing ganesha.nfsd
killed by an ABRT signal. I was using the same ganesha.conf file that
worked with V2.3.2 and CephFS. Here is the gdb backtrace:
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/opt/keeper/bin/ganesha.nfsd -F -L
/var/log/nfs-ganesha.log'.
Program terminated with signal SIGABRT, Aborted.
#0 0x00007f4fcf23ecc9 in __GI_raise (sig=sig@entry=6) at
../nptl/sysdeps/unix/sysv/linux/raise.c:56
56 ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0 0x00007f4fcf23ecc9 in __GI_raise (sig=sig@entry=6) at
../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1 0x00007f4fcf2420d8 in __GI_abort () at abort.c:89
#2 0x00007f4fcf237b86 in __assert_fail_base (fmt=0x7f4fcf388830
"%s%s%s:%u: %s%sAssertion `%s' failed.\n%n",
assertion=assertion@entry=0x575fef "refcount > 0",
file=file@entry=0x575878
"/home/keeper/work/ganesha/ganesha_2.4.0/nfs-ganesha/src/SAL/state_misc.c",
line=line@entry=917,
function=function@entry=0x5763c0 <__PRETTY_FUNCTION__.20389>
"dec_state_owner_ref") at assert.c:92
#3 0x00007f4fcf237c32 in __GI___assert_fail (assertion=0x575fef
"refcount > 0", file=0x575878
"/home/keeper/work/ganesha/ganesha_2.4.0/nfs-ganesha/src/SAL/state_misc.c",
line=917, function=0x5763c0 <__PRETTY_FUNCTION__.20389>
"dec_state_owner_ref") at assert.c:101
#4 0x00000000004c4bd2 in dec_state_owner_ref (owner=0x7f4f4085d5e8)
at /home/keeper/work/ganesha/ganesha_2.4.0/nfs-ganesha/src/SAL/state_misc.c:917
#5 0x00000000004c4fac in uncache_nfs4_owner
(nfs4_owner=0x7f4f4085d638) at
/home/keeper/work/ganesha/ganesha_2.4.0/nfs-ganesha/src/SAL/state_misc.c:1010
#6 0x000000000045693d in reap_expired_open_owners () at
/home/keeper/work/ganesha/ganesha_2.4.0/nfs-ganesha/src/MainNFSD/nfs_reaper_thread.c:183
#7 0x0000000000456bda in reaper_run (ctx=0x7f4fc9ffc180) at
/home/keeper/work/ganesha/ganesha_2.4.0/nfs-ganesha/src/MainNFSD/nfs_reaper_thread.c:247
#8 0x000000000050ab9d in fridgethr_start_routine (arg=0x7f4fc9ffc180)
at
/home/keeper/work/ganesha/ganesha_2.4.0/nfs-ganesha/src/support/fridgethr.c:550
#9 0x00007f4fcfa2c182 in start_thread (arg=0x7f4f43ffe700) at
pthread_create.c:312
#10 0x00007f4fcf30247d in clone () at
../sysdeps/unix/sysv/linux/x86_64/clone.S:111
The output for the backtrace for all the threads using the gdb command
"thread apply all bt" is at: http://pasted.co/18de09f1
and the output of "thread apply all bt full" is at: http://pasted.co/72ed25ef
I can upload the core file, binary and library files if needed to our
public ftp site.
Other info:
# /opt/keeper/bin/ganesha.nfsd -v
ganesha.nfsd compiled on Sep 22 2016 at 18:34:16
Release = V2.4.0
Release comment = GANESHA file server is 64 bits compliant and
supports NFS v3,4.0,4.1 (pNFS) and 9P
Git HEAD = 0c209a710292e24a867a6e9b9281a89563fdb148
Git Describe = V2.4.0-0-g0c209a7
# cat ganesha.conf
LOG {
components {
ALL = INFO;
#FSAL = DEBUG;
}
}
EXPORT_DEFAULTS {
SecType = none, sys;
Protocols = 3, 4;
Transports = TCP;
}
# define CephFS FUSE export
EXPORT {
Export_ID = 41;
Path = /cephfsFUSE/top;
Pseudo = /cephfsFUSE/top;
Access_Type = RW;
Squash = No_Root_Squash;
FSAL {
Name = VFS;
}
}
OS: Ubuntu 14.04
# uname -a
Linux ede-c2-gw01 4.8.0-rc7-4.8.0rc7ksafe #1 SMP Sun Sep 18 22:23:34
EDT 2016 x86_64 x86_64 x86_64 GNU/Linux
# ceph -v
ceph version 10.2.2 (45107e21c568dd033c2f0a3107dec8f0b0e58374)
# cat /proc/mounts | grep ceph
ceph-fuse /cephfsFUSE fuse.ceph-fuse
rw,noatime,user_id=0,group_id=0,allow_other 0 0
The test was 3 Ubuntu 14.04 NFS clients each having 6 processes,
writing 11,000 256k files in separate directory trees with 11 files
per lowest level node. On each Ubuntu client, 3 processes wrote to a
NFS 3 mount and 3 wrote to a NFS 4 mount.
The reason for not using the CEPHFS FSAL is with Ganesha V2.3.2 I had
some issues with file overwrites. After I finished my initial 2.4.0
tests with the POSIX interface I was going to retest with 2.4.0 with
the CEPHFS FSAL.
Best regards,
Eric
------------------------------------------------------------------------------
_______________________________________________
Nfs-ganesha-devel mailing list
Nfs-ganesha-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel
