I tried with 3.10.0-229.20.1.el7.x86_64 kernel (some 7.1 errata kernel) and
it did do 1MB i/os. CentOS7.2's GA kernel (kernel-3.10.0-327.el7.x86_64)
itself started doing 256K i/os. So this is an NFS client issue started with
RHEL7.2 kernels.
Regards, Malahal.
On Sat, Oct 22, 2016 at 7:46 AM, Malahal Naineni <nain...@us.ibm.com> wrote:
> Frame 45 has 1MB as max read/writes from ganesha.
>
> Regards, Malahal.
>
>
> ----- Original message -----
> From: Malahal Naineni/Beaverton/IBM
> To: Olaf Weiser/Germany/IBM@IBMDE
> Cc: d...@redhat.com, mala...@gmail.com, nfs-ganesha-devel@lists.
> sourceforge.net
> Subject: Re: [Nfs-ganesha-devel] nfs testing for SAP - status - new traces
> / new approach
> Date: Sat, Oct 22, 2016 7:35 AM
>
> This 64MB could be for the pseudo export. Let me check some more...
>
> Regards, Malahal.
>
>
> ----- Original message -----
> From: Malahal Naineni/Beaverton/IBM
> To: Olaf Weiser/Germany/IBM@IBMDE
> Cc: d...@redhat.com, mala...@gmail.com, nfs-ganesha-devel@lists.
> sourceforge.net
> Subject: Re: [Nfs-ganesha-devel] nfs testing for SAP - status - new traces
> / new approach
> Date: Sat, Oct 22, 2016 6:34 AM
>
> Ganesha is sending 64MB as max write and max read size in frame 17 in
> client's tcpdump you attached. I am assuming that you changed from the
> default 1MB. What is your client's distro/kernel version?
>
> Regards, Malahal.
>
>
> ----- Original message -----
> From: Olaf Weiser/Germany/IBM
> To: Daniel Gryniewicz <d...@redhat.com>
> Cc: Malahal Naineni <mala...@gmail.com>, Malahal Naineni <
> nain...@us.ibm.com>, NFS Ganesha Developers <nfs-ganesha-devel@lists.
> sourceforge.net>
> Subject: Re: [Nfs-ganesha-devel] nfs testing for SAP - status - new traces
> / new approach
> Date: Sat, Oct 22, 2016 1:37 AM
>
> Hi Malahal,
>
> as requested.. I collected 2 two tcpdump (server side and client side) -
> we try to extract some information, but could'nt find anything what turns
> out the 256K limit..
>
> @Danial - we use RHEL 7.1 ... and have this issue . do you have a solution
> ?
>
>
> *(See attached file: tcpdumfile.client)*
>
> *(See attached file: tcpdumfile.server)*
> *** i just cancel the tcpdump after the file system was mounted . .
>
> thank you very much in advance .for checking/ investigating the data..
>
>
> Mit freundlichen Grüßen / Kind regards
>
>
> Olaf Weiser
>
> EMEA Storage Competence Center Mainz, German / IBM Systems, Storage
> Platform,
> ------------------------------------------------------------
> ------------------------------------------------------------
> -------------------
> IBM Deutschland
> IBM Allee 1
> 71139 Ehningen
> Phone: +49-170-579-44-66
> E-Mail: olaf.wei...@de.ibm.com
> ------------------------------------------------------------
> ------------------------------------------------------------
> -------------------
> IBM Deutschland GmbH / Vorsitzender des Aufsichtsrats: Martin Jetter
> Geschäftsführung: Martina Koederitz (Vorsitzende), Susanne Peter, Norbert
> Janzen, Dr. Christian Keller, Ivo Koerner, Markus Koerner
> Sitz der Gesellschaft: Ehningen / Registergericht: Amtsgericht Stuttgart,
> HRB 14562 / WEEE-Reg.-Nr. DE 99369940
>
> [image: Inactive hide details for Daniel Gryniewicz ---10/21/2016 07:57:06
> AM---A user just showed up on IRC claiming this exact proble]Daniel
> Gryniewicz ---10/21/2016 07:57:06 AM---A user just showed up on IRC
> claiming this exact problem, but only on CentOS/RHEL. I know it doesn'
>
> From: Daniel Gryniewicz <d...@redhat.com>
> To: Malahal Naineni <nain...@us.ibm.com>
> Cc: Malahal Naineni <mala...@gmail.com>, NFS Ganesha Developers <
> nfs-ganesha-devel@lists.sourceforge.net>, Olaf Weiser/Germany/IBM@IBMDE
> Date: 10/21/2016 07:57 AM
> Subject: Re: [Nfs-ganesha-devel] nfs testing for SAP - status - new
> traces / new approach
> ------------------------------
>
>
>
> A user just showed up on IRC claiming this exact problem, but only on
> CentOS/RHEL. I know it doesn't happen on Fedora. Maybe it's related
> to default buffer sizes on sockets, or something?
>
> On Thu, Oct 20, 2016 at 10:44 PM, Malahal Naineni <nain...@us.ibm.com>
> wrote:
> > Please provide ganesha config and tcpdump that include FSINFO request
> (start
> > tcpdump before the mount and do a very small I/O before you kill the
> > tcpdump). Also, I am assuming that GPFS that deals with ganesha is not
> > splitting the I/O as these traces don't indicate what ganesha is actually
> > using, right?
> >
> > I know for a fact that we have seen I/Os with 1MB, so this can't be a
> > ganesha hard limitation (including NFSv4 that I just experimented).
> >
> > Regards, Malahal.
> >
> >
> > ----- Original message -----
> > From: Malahal Naineni <mala...@gmail.com>
> > To: Marc Eshel/Almaden/IBM@IBMUS
> > Cc: Frank Filz <ffilz...@mindspring.com>, Matt Benjamin
> > <mbenja...@redhat.com>, Malahal Naineni/Beaverton/IBM@IBMUS, NFS Ganesha
> > Developers <nfs-ganesha-devel@lists.sourceforge.net>, Olaf Weiser
> > <olaf.wei...@de.ibm.com>
> > Subject: Re: [Nfs-ganesha-devel] nfs testing for SAP - status - new
> traces /
> > new approach
> > Date: Fri, Oct 21, 2016 7:55 AM
> >
> > I think that rpc code is about receive/send size which is quite
> > different from NFS i/o size. Default max io size is 1M with GPFS fsal,
> > but it is the client that could also limit. Since the same client is
> > not limiting with kNFS, I am pretty sure you have something wrong in
> > your ganesh config. The actual value should be provided to NFS client
> > as part of FSINFO response. Please look at the FSINFO response with
> > tcpdump.
> >
> > I know for a fact we have seen I/Os with 1MB at least with NFSv3. I
> > will experiment with NFSv4 just in case, but I doubt this is due to
> > code.
> >
> > On Fri, Oct 21, 2016 at 7:19 AM, Marc Eshel <es...@us.ibm.com> wrote:
> >> We are not able to get IO bigger than 256K with Ganesha, same client and
> >> kNFS can get 1M.
> >> Is there something in Ganesha that limmits the IO size? see attached
> email
> >> maxsize = 256 * 1024; /* XXX */, is that a problem ?
> >> Marc.
> >>
> >>
> >>
> >> From: Sven Oehme/Almaden/IBM
> >> To: Olaf Weiser/Germany/IBM@IBMDE
> >> Cc: Malahal Naineni/Beaverton/IBM@IBMUS, dhil...@us.ibm.com,
> >> fschm...@us.ibm.com, gfsch...@us.ibm.com, Marc Eshel/Almaden/IBM@IBMUS,
> >> robg...@us.ibm.com
> >> Date: 10/20/2016 05:32 PM
> >> Subject: Re: nfs testing for SAP - status - new traces / new
> >> approach
> >>
> >>
> >> marc will send a email on what i found in the ganesha code. it seems
> that
> >> max rpc size is hard limited to 256k :
> >>
> >> * Find the appropriate buffer size
> >> */
> >> u_int /*ARGSUSED*/
> >> __rpc_get_t_size(int af, int proto, int size)
> >> {
> >> int maxsize, defsize;
> >>
> >> maxsize = 256 * 1024; /* XXX */
> >> switch (proto) {
> >> case IPPROTO_TCP:
> >> defsize = 64 * 1024; /* XXX */
> >> break;
> >> case IPPROTO_UDP:
> >> defsize = UDPMSGSIZE;
> >> break;
> >> default:
> >> defsize = RPC_MAXDATASIZE;
> >> break;
> >> }
> >> if (size == 0)
> >> return defsize;
> >>
> >> /* Check whether the value is within the upper max limit */
> >> return (size > maxsize ? (u_int) maxsize : (u_int) size);
> >> }
> >> 4:01:28 PM
> >> in : src/rpc_generic.c
> >>
> >>
> >> ------------------------------------------
> >> Sven Oehme
> >> Scalable Storage Research
> >> email: oeh...@us.ibm.com
> >> Phone: +1 (408) 824-8904
> >> IBM Almaden Research Lab
> >> ------------------------------------------
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> ------------------------------------------------------------
> ------------------
> >> Check out the vibrant tech community on one of the world's most
> >> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> >> _______________________________________________
> >> Nfs-ganesha-devel mailing list
> >> Nfs-ganesha-devel@lists.sourceforge.net
> >> https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel
> >
> >
> >
> >
> >
> > ------------------------------------------------------------
> ------------------
> > Check out the vibrant tech community on one of the world's most
> > engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> > _______________________________________________
> > Nfs-ganesha-devel mailing list
> > Nfs-ganesha-devel@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel
> >
>
>
>
>
>
>
>
>
>
>
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Nfs-ganesha-devel mailing list
Nfs-ganesha-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel
