Hi, Sorry if I missed it, but do you have pcap trace(s) showing the protocol operations when this reproduces?
Thanks Matt ----- Original Message ----- > From: "Tushar Shinde" <mtk.tus...@gmail.com> > To: "Tushar Shinde" <mtk.tus...@gmail.com>, > nfs-ganesha-devel@lists.sourceforge.net > Sent: Thursday, March 10, 2016 8:51:56 AM > Subject: Re: [Nfs-ganesha-devel] NFS Ganesha rm -rf randomly not working in > Solaris. > > Hello Malahal, > > Thanks for reply. > I tried that fix, But I am seeing following messages from solaris > client. This is bit random issue. > > rm: WARNING: A subdirectory of /mnt/vdbench2/bigfileset/00000003 was > moved or linked to another directory during the execution of rm > rm: Unable to remove directory > /mnt/vdbench2/bigfileset/00000003/00000004: No such file or directory > rm: WARNING: A subdirectory of /mnt/vdbench2/bigfileset/00000003 was > moved or linked to another directory during the execution of rm > rm: Unable to remove directory > /mnt/vdbench2/bigfileset/00000003/00000004: No such file or directory > rm: WARNING: A subdirectory of /mnt/vdbench2/bigfileset/00000003 was > moved or linked to another directory during the execution of rm > rm: Unable to remove directory > /mnt/vdbench2/bigfileset/00000003/00000004: No such file or directory > rm: WARNING: A subdirectory of /mnt/vdbench2/bigfileset/00000003 was > moved or linked to another directory during the execution of rm > > > The filesystem is not updated by any other than Ganesha server itself. > And this issue only comes on Solaris, I have not see it on Linux. > > Thank you, > Tushar > > > > On Wed, Mar 9, 2016 at 3:41 AM, Malahal Naineni <mala...@us.ibm.com> wrote: > > Does the following fix the issue? > > > > commit d669eb56d19f1d722e1919d0f39e7055caa88ddd > > Author: Krishna Harathi <khara...@exablox.com> > > Date: Fri Nov 6 00:03:58 2015 +0000 > > > > nfsv3 - fix malformed packet response in readdir when zero entries are > > returned. Also in cache_inode_readdir. > > > > Change-Id: If9489308b863d8a819511fbd8cb0ed7f633e283f > > Signed-off-by: Krishna Harathi <khara...@exablox.com> > > > > > > Tushar Shinde [mtk.tus...@gmail.com] wrote: > >> Hi All, > >> > >> Env: > >> Server: NFS Ganesha V2.2-stable x86_64, on RHEL 6.5 > >> Client: SunOS 5.10 Generic_150400-30 sun4v sparc > >> SUNW,SPARC-Enterprise-T5120 > >> bash-3.2# cat /etc/release > >> Oracle Solaris 10 1/13 s10s_u11wos_24a SPARC > >> Copyright (c) 1983, 2013, Oracle and/or its affiliates. All rights > >> reserved. > >> Assembled 17 January 2013 > >> > >> > >> Issue: > >> >From solaris client if we do rm -rf it randomly fails, Many time it > >> says eexists or directory is moved or linked while rm. > >> > >> I was debugging this issue and following are observations. Please > >> guide me to patch (if already fixed) or any clue to solve this. > >> > >> After debugging I found if unlink is in progress in same dir as of > >> reading, > >> while(ent = readdir()) { > >> remove() > >> } > >> > >> on solaris, getdents returns ENOENT on last call, rather it should get 0. > >> Client is sending cookie which deleted on server by last remove call. > >> Since there is no files left, > >> Ganesha gets no new node and prints error like following. > >> > >> On Client: > >> getdents(4, 0xFFFFFFFF7E700000, 8192) Err#2 ENOENT > >> > >> On Server > >> 2016-03-08 12:53:28 : ganesha.nfsd-30982[work-9] nfs3_readdir :F_DBG > >> :---> nfs3_readdir: count=8192 cookie=18411418809641280358 > >> estimated_num_entries=120 > >> 2016-03-08 12:53:28 : ganesha.nfsd-30982[work-9] cache_inode_readdir > >> :F_DBG :Enter.... > >> 2016-03-08 12:53:28 : ganesha.nfsd-30982[work-9] cache_inode_access_sw > >> :DEBUG :INODE: DEBUG: access_type=0X84000001 > >> 2016-03-08 12:53:28 : ganesha.nfsd-30982[work-9] > >> cache_inode_avl_lookup_k :DEBUG :found deleted supremum (nil) > >> 2016-03-08 12:53:28 : ganesha.nfsd-30982[work-9] cache_inode_readdir > >> :F_DBG :seek to cookie=18411418809641280358 fail > >> > >> Thing to note here is cookie 18411418809641280358 is no more alive and > >> hence lookup return nil. > >> > >> I tried sending CACHE_INODE_SUCCESS with 0 entries instead of > >> CACHE_INODE_BAD_COOKIE but that also did not worked. > >> > >> What should be correct return code for readdir request with last > >> cookie which is deleted on server? Any one know any relevant issues > >> like this. > >> > >> (I tried to subscribe the mailing list multiple times but I never got > >> confirmation email so sending message to > >> nfs-ganesha-devel@lists.sourceforge.net, Please route to correct alias > >> if not appropriate for this id.) > >> > >> Thank you, > >> Tushar Shinde. > >> > >> ------------------------------------------------------------------------------ > >> Transform Data into Opportunity. > >> Accelerate data analysis in your applications with > >> Intel Data Analytics Acceleration Library. > >> Click to learn more. > >> http://makebettercode.com/inteldaal-eval > >> _______________________________________________ > >> Nfs-ganesha-devel mailing list > >> Nfs-ganesha-devel@lists.sourceforge.net > >> https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel > >> > > > > ------------------------------------------------------------------------------ > Transform Data into Opportunity. > Accelerate data analysis in your applications with > Intel Data Analytics Acceleration Library. > Click to learn more. > http://pubads.g.doubleclick.net/gampad/clk?id=278785111&iu=/4140 > _______________________________________________ > Nfs-ganesha-devel mailing list > Nfs-ganesha-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel > -- Matt Benjamin Red Hat, Inc. 315 West Huron Street, Suite 140A Ann Arbor, Michigan 48103 http://www.redhat.com/en/technologies/storage tel. 734-707-0660 fax. 734-769-8938 cel. 734-216-5309 ------------------------------------------------------------------------------ Transform Data into Opportunity. Accelerate data analysis in your applications with Intel Data Analytics Acceleration Library. Click to learn more. http://pubads.g.doubleclick.net/gampad/clk?id=278785111&iu=/4140 _______________________________________________ Nfs-ganesha-devel mailing list Nfs-ganesha-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel