Hi Tushar,
IIRC, settng of notfound and eod are independent from what I was trying to do
in that patch, which is avoid returning badcookie when a client (mis)used a
cookie it had seen but was no longer valid--the Linux client was doing
something pretty questionable, in fact. I wish we could forget about cookies,
but I do think it may be safe to set notfound and eod in that case (there is no
reason for the client to continue iterating if we believe there are no entries).
Matt
----- Original Message -----
> From: "Tushar Shinde" <mtk.tus...@gmail.com>
> To: "Matt Benjamin" <mbenja...@redhat.com>
> Cc: nfs-ganesha-devel@lists.sourceforge.net
> Sent: Monday, March 14, 2016 2:51:06 AM
> Subject: Re: [Nfs-ganesha-devel] NFS Ganesha rm -rf randomly not working in
> Solaris.
>
> Hi Matt,
>
> I am not able to reproduce this rm issue issue, I will get back to you
> if I manage to repro it.
>
>
> I have query on the commit d669eb56d19f1d722e1919d0f39e7055caa88ddd
>
> In that fix, if flag is CACHE_INODE_FLAG_NONE, Then
> cache_inode_avl_lookup_k checks cookie tree,
> "&entry->object.dir.avl.c;"
>
> Consider following flow,
> - directory got modified outside current servers knowledge (because of
> cluster filesystem)
> - In cache_inode_readdir -> cache_inode_lock_trust_attrs ->
> cache_inode_invalidate_all_cached_dirent ->
> cache_inode_release_dirents(entry, CACHE_INODE_AVL_BOTH); Cookie tree
> entries got released.
> - Later cache_inode_readdir_populate will rebuild the 't' tree.
>
> In this case the cookie tree got cleaned because of "BOTH" flag, But
> if server had sent any cookie to client for which entry got deleted
> (and entry exists in 'c' tree) we will return bad cookie without
> setting following flags
> 690 *nbfound = 0;
> 691 *eod_met = true;
>
> Do you think instead of checking 'c' tree we can check size of 't'
> tree and if that tree is empty we can set above values?
> I did test with avltree_size('t') for above case and its working.
>
> Thank you,
> Tushar.
>
>
>
>
>
>
> On Thu, Mar 10, 2016 at 10:38 PM, Matt Benjamin <mbenja...@redhat.com> wrote:
> > Hi,
> >
> > Sorry if I missed it, but do you have pcap trace(s) showing the protocol
> > operations when this reproduces?
> >
> > Thanks
> >
> > Matt
> >
> > ----- Original Message -----
> >> From: "Tushar Shinde" <mtk.tus...@gmail.com>
> >> To: "Tushar Shinde" <mtk.tus...@gmail.com>,
> >> nfs-ganesha-devel@lists.sourceforge.net
> >> Sent: Thursday, March 10, 2016 8:51:56 AM
> >> Subject: Re: [Nfs-ganesha-devel] NFS Ganesha rm -rf randomly not working
> >> in Solaris.
> >>
> >> Hello Malahal,
> >>
> >> Thanks for reply.
> >> I tried that fix, But I am seeing following messages from solaris
> >> client. This is bit random issue.
> >>
> >> rm: WARNING: A subdirectory of /mnt/vdbench2/bigfileset/00000003 was
> >> moved or linked to another directory during the execution of rm
> >> rm: Unable to remove directory
> >> /mnt/vdbench2/bigfileset/00000003/00000004: No such file or directory
> >> rm: WARNING: A subdirectory of /mnt/vdbench2/bigfileset/00000003 was
> >> moved or linked to another directory during the execution of rm
> >> rm: Unable to remove directory
> >> /mnt/vdbench2/bigfileset/00000003/00000004: No such file or directory
> >> rm: WARNING: A subdirectory of /mnt/vdbench2/bigfileset/00000003 was
> >> moved or linked to another directory during the execution of rm
> >> rm: Unable to remove directory
> >> /mnt/vdbench2/bigfileset/00000003/00000004: No such file or directory
> >> rm: WARNING: A subdirectory of /mnt/vdbench2/bigfileset/00000003 was
> >> moved or linked to another directory during the execution of rm
> >>
> >>
> >> The filesystem is not updated by any other than Ganesha server itself.
> >> And this issue only comes on Solaris, I have not see it on Linux.
> >>
> >> Thank you,
> >> Tushar
> >>
> >>
> >>
> >> On Wed, Mar 9, 2016 at 3:41 AM, Malahal Naineni <mala...@us.ibm.com>
> >> wrote:
> >> > Does the following fix the issue?
> >> >
> >> > commit d669eb56d19f1d722e1919d0f39e7055caa88ddd
> >> > Author: Krishna Harathi <khara...@exablox.com>
> >> > Date: Fri Nov 6 00:03:58 2015 +0000
> >> >
> >> > nfsv3 - fix malformed packet response in readdir when zero entries
> >> > are
> >> > returned. Also in cache_inode_readdir.
> >> >
> >> > Change-Id: If9489308b863d8a819511fbd8cb0ed7f633e283f
> >> > Signed-off-by: Krishna Harathi <khara...@exablox.com>
> >> >
> >> >
> >> > Tushar Shinde [mtk.tus...@gmail.com] wrote:
> >> >> Hi All,
> >> >>
> >> >> Env:
> >> >> Server: NFS Ganesha V2.2-stable x86_64, on RHEL 6.5
> >> >> Client: SunOS 5.10 Generic_150400-30 sun4v sparc
> >> >> SUNW,SPARC-Enterprise-T5120
> >> >> bash-3.2# cat /etc/release
> >> >> Oracle Solaris 10 1/13 s10s_u11wos_24a SPARC
> >> >> Copyright (c) 1983, 2013, Oracle and/or its affiliates. All rights
> >> >> reserved.
> >> >> Assembled 17 January 2013
> >> >>
> >> >>
> >> >> Issue:
> >> >> >From solaris client if we do rm -rf it randomly fails, Many time it
> >> >> says eexists or directory is moved or linked while rm.
> >> >>
> >> >> I was debugging this issue and following are observations. Please
> >> >> guide me to patch (if already fixed) or any clue to solve this.
> >> >>
> >> >> After debugging I found if unlink is in progress in same dir as of
> >> >> reading,
> >> >> while(ent = readdir()) {
> >> >> remove()
> >> >> }
> >> >>
> >> >> on solaris, getdents returns ENOENT on last call, rather it should get
> >> >> 0.
> >> >> Client is sending cookie which deleted on server by last remove call.
> >> >> Since there is no files left,
> >> >> Ganesha gets no new node and prints error like following.
> >> >>
> >> >> On Client:
> >> >> getdents(4, 0xFFFFFFFF7E700000, 8192) Err#2 ENOENT
> >> >>
> >> >> On Server
> >> >> 2016-03-08 12:53:28 : ganesha.nfsd-30982[work-9] nfs3_readdir :F_DBG
> >> >> :---> nfs3_readdir: count=8192 cookie=18411418809641280358
> >> >> estimated_num_entries=120
> >> >> 2016-03-08 12:53:28 : ganesha.nfsd-30982[work-9] cache_inode_readdir
> >> >> :F_DBG :Enter....
> >> >> 2016-03-08 12:53:28 : ganesha.nfsd-30982[work-9] cache_inode_access_sw
> >> >> :DEBUG :INODE: DEBUG: access_type=0X84000001
> >> >> 2016-03-08 12:53:28 : ganesha.nfsd-30982[work-9]
> >> >> cache_inode_avl_lookup_k :DEBUG :found deleted supremum (nil)
> >> >> 2016-03-08 12:53:28 : ganesha.nfsd-30982[work-9] cache_inode_readdir
> >> >> :F_DBG :seek to cookie=18411418809641280358 fail
> >> >>
> >> >> Thing to note here is cookie 18411418809641280358 is no more alive and
> >> >> hence lookup return nil.
> >> >>
> >> >> I tried sending CACHE_INODE_SUCCESS with 0 entries instead of
> >> >> CACHE_INODE_BAD_COOKIE but that also did not worked.
> >> >>
> >> >> What should be correct return code for readdir request with last
> >> >> cookie which is deleted on server? Any one know any relevant issues
> >> >> like this.
> >> >>
> >> >> (I tried to subscribe the mailing list multiple times but I never got
> >> >> confirmation email so sending message to
> >> >> nfs-ganesha-devel@lists.sourceforge.net, Please route to correct alias
> >> >> if not appropriate for this id.)
> >> >>
> >> >> Thank you,
> >> >> Tushar Shinde.
> >> >>
> >> >> ------------------------------------------------------------------------------
> >> >> Transform Data into Opportunity.
> >> >> Accelerate data analysis in your applications with
> >> >> Intel Data Analytics Acceleration Library.
> >> >> Click to learn more.
> >> >> http://makebettercode.com/inteldaal-eval
> >> >> _______________________________________________
> >> >> Nfs-ganesha-devel mailing list
> >> >> Nfs-ganesha-devel@lists.sourceforge.net
> >> >> https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel
> >> >>
> >> >
> >>
> >> ------------------------------------------------------------------------------
> >> Transform Data into Opportunity.
> >> Accelerate data analysis in your applications with
> >> Intel Data Analytics Acceleration Library.
> >> Click to learn more.
> >> http://pubads.g.doubleclick.net/gampad/clk?id=278785111&iu=/4140
> >> _______________________________________________
> >> Nfs-ganesha-devel mailing list
> >> Nfs-ganesha-devel@lists.sourceforge.net
> >> https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel
> >>
> >
> > --
> > Matt Benjamin
> > Red Hat, Inc.
> > 315 West Huron Street, Suite 140A
> > Ann Arbor, Michigan 48103
> >
> > http://www.redhat.com/en/technologies/storage
> >
> > tel. 734-707-0660
> > fax. 734-769-8938
> > cel. 734-216-5309
>
--
Matt Benjamin
Red Hat, Inc.
315 West Huron Street, Suite 140A
Ann Arbor, Michigan 48103
http://www.redhat.com/en/technologies/storage
tel. 734-707-0660
fax. 734-769-8938
cel. 734-216-5309
------------------------------------------------------------------------------
Transform Data into Opportunity.
Accelerate data analysis in your applications with
Intel Data Analytics Acceleration Library.
Click to learn more.
http://pubads.g.doubleclick.net/gampad/clk?id=278785231&iu=/4140
_______________________________________________
Nfs-ganesha-devel mailing list
Nfs-ganesha-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel
