On 03/07/2018 10:21 PM, Kinglong Mee wrote:
On 2018/3/7 21:10, Daniel Gryniewicz wrote:
On 03/06/2018 10:10 PM, Kinglong Mee wrote:
On 2018/3/7 10:59, Kinglong Mee wrote:
When using nfsv3 on glusterfs-3.13.1-1.el7.x86_64 and
nfs-ganesha-2.6.0-0.2rc3.el7.centos.x86_64,
I gets strange "Invalid argument" when writing file.
1. With quota disabled;
nfs client mount nfs-ganesha share, and do 'll' in the testing directory.
2. Enable quota;
# getfattr -d -m . -e hex /root/rpmbuild/gvtest/nfs-ganesha/testfile92
getfattr: Removing leading '/' from absolute path names
# file: root/rpmbuild/gvtest/nfs-ganesha/testfile92
trusted.gfid=0xe2edaac0eca8420ebbbcba7e56bbd240
trusted.gfid2path.b3250af8fa558e66=0x39663134343566662d653530332d343831352d396635312d3236633565366332633137642f7465737466696c653932
trusted.glusterfs.quota.9f1445ff-e503-4815-9f51-26c5e6c2c17d.contri.3=0x00000000000002000000000000000001
Notice: testfile92 without trusted.pgfid xattr.
The trusted.pgfid will be created by the next name lookup; nameless lookup
don't create it.
3. restart glusterfs volume by "gluster volume stop/start gvtest"
Restarting glusterfsd here cleanup all inode cache from memory;
after starting, inode of testfile92's parent is NULL.
4. echo somedata > testfile92
Because, nfs-ganesha and nfs client has cache for testfile92,
before write fops, no name lookup happens that trusted.pgfid is not created for
testfile92.
Quota_writev call quota_build_ancestry building the ancestry in
quota_check_limit,
but testfile92 doesn't contain trusted.pgfid, so that write fops failed with
Invalid argument.
I have no idea of fixing this problem, any comments are welcome.
I think, ideally, Gluster would send an invalidate upcall under the
circumstances, causing Ganesha do drop it's cached entry.
It doesn't work.
I try to restarting nfs-ganesha, and echo data to testfile92, the problem also
exists.
After nfs-ganesha restart,
1. A GETATTR is send from nfs-client for the testfile92, ganesha translates it
to nameless lookup.
2. A ACCESS gets attributes from nfs-ganesha's cache (cached by #1).
3. A SETATTR sets the testfile92's size to 0, ganesha translates it to setattr
fop.
4. A WRITE also get Invalid argument error.
If ganesha drops its cache, nfs client may write file by filehandle;
ganesha lookup it by nameless lookup from glusterfs,
so that, trusted.pgfid isn't created too.
I think, a name lookup is needed for testfile92 after quota enable.
thanks,
Kinglong Mee
If a name lookup is needed, then gluster cannot provide NFS semantics in
these circumstances. NFS *requires* that access by handle continue to
work once the client has a handle. In fact, there's no way to get a
name for this, since the name doesn't exist anywhere in the handle (by
design, since a name is an attribute of a dirent, not of a handle/inode).
So, this likely is a bug in Gluster, and needs to be fixed there. Would
it be possible to enable quota globally at the start as a workaround?
Daniel
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Nfs-ganesha-devel mailing list
Nfs-ganesha-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel
