On 2018/3/7 21:10, Daniel Gryniewicz wrote: > On 03/06/2018 10:10 PM, Kinglong Mee wrote: >> On 2018/3/7 10:59, Kinglong Mee wrote: >>> When using nfsv3 on glusterfs-3.13.1-1.el7.x86_64 and >>> nfs-ganesha-2.6.0-0.2rc3.el7.centos.x86_64, >>> I gets strange "Invalid argument" when writing file. >>> >>> 1. With quota disabled; >>> nfs client mount nfs-ganesha share, and do 'll' in the testing directory. >>> >>> 2. Enable quota; >>> # getfattr -d -m . -e hex /root/rpmbuild/gvtest/nfs-ganesha/testfile92 >>> getfattr: Removing leading '/' from absolute path names >>> # file: root/rpmbuild/gvtest/nfs-ganesha/testfile92 >>> trusted.gfid=0xe2edaac0eca8420ebbbcba7e56bbd240 >>> trusted.gfid2path.b3250af8fa558e66=0x39663134343566662d653530332d343831352d396635312d3236633565366332633137642f7465737466696c653932 >>> trusted.glusterfs.quota.9f1445ff-e503-4815-9f51-26c5e6c2c17d.contri.3=0x00000000000002000000000000000001 >>> >>> Notice: testfile92 without trusted.pgfid xattr. >> >> The trusted.pgfid will be created by the next name lookup; nameless lookup >> don't create it. >> >>> 3. restart glusterfs volume by "gluster volume stop/start gvtest" >> >> Restarting glusterfsd here cleanup all inode cache from memory; >> after starting, inode of testfile92's parent is NULL. >> >>> 4. echo somedata > testfile92 >> >> Because, nfs-ganesha and nfs client has cache for testfile92, >> before write fops, no name lookup happens that trusted.pgfid is not created >> for testfile92. >> >> Quota_writev call quota_build_ancestry building the ancestry in >> quota_check_limit, >> but testfile92 doesn't contain trusted.pgfid, so that write fops failed with >> Invalid argument. >> >> I have no idea of fixing this problem, any comments are welcome. >> > > I think, ideally, Gluster would send an invalidate upcall under the > circumstances, causing Ganesha do drop it's cached entry.
It doesn't work. I try to restarting nfs-ganesha, and echo data to testfile92, the problem also exists. After nfs-ganesha restart, 1. A GETATTR is send from nfs-client for the testfile92, ganesha translates it to nameless lookup. 2. A ACCESS gets attributes from nfs-ganesha's cache (cached by #1). 3. A SETATTR sets the testfile92's size to 0, ganesha translates it to setattr fop. 4. A WRITE also get Invalid argument error. If ganesha drops its cache, nfs client may write file by filehandle; ganesha lookup it by nameless lookup from glusterfs, so that, trusted.pgfid isn't created too. I think, a name lookup is needed for testfile92 after quota enable. thanks, Kinglong Mee _______________________________________________ Gluster-devel mailing list Glusterfirstname.lastname@example.org http://lists.gluster.org/mailman/listinfo/gluster-devel