Re: [Nfs-ganesha-devel] Deprecating non-support_ex

[Frank Filz]

> I'm currently working on converting FSAL_PROXY to support_ex but it will
> take time.

Hmm, maybe I should do a quick and dirty FSAL_PROXY update that still
doesn't do state.

Obviously to create a file it does need to use OPEN, but it could
immediately CLOSE.

The "global fd" will just be the appropriate special stateid depending on
what kind of I/O is being requested.

Frank

> On 01/10/17 23:33, Frank Filz wrote:
> > We had discussed that in 2.5 we would deprecate non-support_ex and
> > strip out the old API interfaces.
> >
> > I'm not sure if we still want to do that (I would like to, to simplify
> > code, and not continue to support clunky interfaces).
> >
> > I'm not sure we converted all in-tree FSALs to support_ex...
> >
> > Frank
> >
> >
> > ---
> > This email has been checked for viruses by Avast antivirus software.
> > https://www.avast.com/antivirus
> >
> >
> > --
> >  Developer Access Program for Intel Xeon Phi Processors Access
> > to Intel Xeon Phi processor-based developer platforms.
> > With one year of Intel Parallel Studio XE.
> > Training and support from Colfax.
> > Order your platform today. http://sdm.link/xeonphi
> > ___
> > Nfs-ganesha-devel mailing list
> > Nfs-ganesha-devel@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel
> 
> 
> --
> Patrice LUCAS
> Ingenieur-Chercheur, CEA-DAM/DSSI/SISR/LA2S tel : +33 (0)1 69 26 47 86 e-
> mail : patrice.lu...@cea.fr
> 
> 
>

--
> Developer Access Program for Intel Xeon Phi Processors
> Access to Intel Xeon Phi processor-based developer platforms.
> With one year of Intel Parallel Studio XE.
> Training and support from Colfax.
> Order your platform today. http://sdm.link/xeonphi
> ___
> Nfs-ganesha-devel mailing list
> Nfs-ganesha-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel


---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus


--
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
___
Nfs-ganesha-devel mailing list
Nfs-ganesha-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel

Re: [Nfs-ganesha-devel] Deprecating non-support_ex

[Daniel Gryniewicz]

PROXY and ZFS are the only unconverted ones.  We should definitely wait 
for PROXY.  I'm unsure about ZFS.  Is it used by anyone?  Does it work? 
I last built it ~1 year ago, and had to make changes to 
contrib/libzfswrap to build...

Daniel

On 01/11/2017 02:53 AM, LUCAS Patrice wrote:
> I'm currently working on converting FSAL_PROXY to support_ex but it will
> take time.
>
> Patrice
>
> On 01/10/17 23:33, Frank Filz wrote:
>> We had discussed that in 2.5 we would deprecate non-support_ex and strip out
>> the old API interfaces.
>>
>> I'm not sure if we still want to do that (I would like to, to simplify code,
>> and not continue to support clunky interfaces).
>>
>> I'm not sure we converted all in-tree FSALs to support_ex...
>>
>> Frank
>>
>>
>> ---
>> This email has been checked for viruses by Avast antivirus software.
>> https://www.avast.com/antivirus
>>
>>
>> --
>> Developer Access Program for Intel Xeon Phi Processors
>> Access to Intel Xeon Phi processor-based developer platforms.
>> With one year of Intel Parallel Studio XE.
>> Training and support from Colfax.
>> Order your platform today. http://sdm.link/xeonphi
>> ___
>> Nfs-ganesha-devel mailing list
>> Nfs-ganesha-devel@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel
>
>


--
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
___
Nfs-ganesha-devel mailing list
Nfs-ganesha-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel

[Nfs-ganesha-devel] Change in ffilz/nfs-ganesha[next]: WIP: Fix DRC refcounts

[GerritHub]

>From Malahal :

Malahal has uploaded a new change for review. ( 
https://review.gerrithub.io/342371


Change subject: WIP: Fix DRC refcounts
..

WIP: Fix DRC refcounts

Currently nfs_dupreq_start() takes a DRC ref with uncached requests.
Surprisingly, no reference is taken when a request is actually cached!
We seem to release DRC ref only when commands fail or get dropped.

Instrumented GPFS fsal to return EBADF/ERR_FSAL_NOT_OPENED on fsal write
request to mimic a real issue. With this, we don't ref DRC for a write
request but we do unref after its failure leading to underrun of ref
count error messages!

Change-Id: I79f1b5198be2bcd0964f22ac01d5accf8ee25e00
Signed-off-by: Malahal Naineni 
---
M src/RPCAL/nfs_dupreq.c
1 file changed, 35 insertions(+), 56 deletions(-)



  git pull ssh://review.gerrithub.io:29419/ffilz/nfs-ganesha 
refs/changes/71/342371/1
-- 
To view, visit https://review.gerrithub.io/342371
To unsubscribe, visit https://review.gerrithub.io/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I79f1b5198be2bcd0964f22ac01d5accf8ee25e00
Gerrit-Change-Number: 342371
Gerrit-PatchSet: 1
Gerrit-Project: ffilz/nfs-ganesha
Gerrit-Branch: next
Gerrit-Owner: Malahal 
--
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi___
Nfs-ganesha-devel mailing list
Nfs-ganesha-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel

Re: [Nfs-ganesha-devel] Readdir results

[Matt Benjamin]

Frank's result based on testing of fsal vfs can't be generalized to 
configurations which actually have latency, so, no?

Matt

- Original Message -
> From: "William Allen Simpson" 
> To: "Frank Filz" , 
> nfs-ganesha-devel@lists.sourceforge.net
> Sent: Wednesday, January 11, 2017 5:19:46 AM
> Subject: Re: [Nfs-ganesha-devel] Readdir results
> 
> On 1/10/17 4:29 PM, Frank Filz wrote:
> > It looks like dirent caching does buy us something, though interestingly
> > the
> > initial populating tends to be the quickest run...
> >
> If the initial populating is the quickest run, there must be some
> interaction between the cache and the re-fetch that's causing thrashing.
> 
> I make no pretense of understanding this code.  But my gut feeling at this
> point is to toss this caching entirely.  Faster stores and faster networks
> mean fetching the entries should be fairly low latency.  Need to optimize
> moving the data through the system, with less examining it.
> 
> --
> Developer Access Program for Intel Xeon Phi Processors
> Access to Intel Xeon Phi processor-based developer platforms.
> With one year of Intel Parallel Studio XE.
> Training and support from Colfax.
> Order your platform today. http://sdm.link/xeonphi
> ___
> Nfs-ganesha-devel mailing list
> Nfs-ganesha-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel
> 

-- 
Matt Benjamin
Red Hat, Inc.
315 West Huron Street, Suite 140A
Ann Arbor, Michigan 48103

http://www.redhat.com/en/technologies/storage

tel.  734-821-5101
fax.  734-769-8938
cel.  734-216-5309

--
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
___
Nfs-ganesha-devel mailing list
Nfs-ganesha-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel

Re: [Nfs-ganesha-devel] Readdir results

[William Allen Simpson]

On 1/10/17 4:29 PM, Frank Filz wrote:
> It looks like dirent caching does buy us something, though interestingly the
> initial populating tends to be the quickest run...
>
If the initial populating is the quickest run, there must be some
interaction between the cache and the re-fetch that's causing thrashing.

I make no pretense of understanding this code.  But my gut feeling at this
point is to toss this caching entirely.  Faster stores and faster networks
mean fetching the entries should be fairly low latency.  Need to optimize
moving the data through the system, with less examining it.

--
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
___
Nfs-ganesha-devel mailing list
Nfs-ganesha-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel

Re: [Nfs-ganesha-devel] Change in ffilz/nfs-ganesha[next]: Embed rpc_msg in svc_req

[William Allen Simpson]

As promised on yesterday's call, I've merged all the updates to
libntirpc in this patch set 4, even though its title would indicate a
patch in the middle of the past month's worth of updates.

This is a boring patch.  It has no substantive changes.  It consists
entirely of formatting and symbol name changes.  It has been waiting
since US Thanksgiving weekend, and was originally 19 months ago.

--
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
___
Nfs-ganesha-devel mailing list
Nfs-ganesha-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel

Re: [Nfs-ganesha-devel] Readdir results

[Swen Schillig]

On Di, 2017-01-10 at 13:29 -0800, Frank Filz wrote:
> I did some testing between having dirent caching and not (Dir_Max
> left
> default, or Dir_Max set to 1):
> 
> With 1 files, dirent caching enabled
> real  0m1.940s
> real  0m3.652s
> real  0m2.409s
> real  0m1.968s
> real  0m2.098s
> real  0m2.209s
> real  0m3.792s
> real  0m2.625s
> real  0m2.201s
> real  0m2.963s
> 
> With 1 files, dirent caching disabled
> real  0m2.423s
> real  0m4.142s
> real  0m3.883s
> real  0m2.939s
> real  0m2.283s
> real  0m5.105s
> real  0m3.050s
> real  0m3.609s
> real  0m2.574s
> real  0m2.727s

Do you see any differences w/ and w/o caching if the directory is being
modified while you're executing ls -l (like adding / removing files or
changing attributes) ?
I remember having to deal with customers doing something alike 
with >200k files in a single directory and the execution time jumped up
to 10-30 minutes.

Cheers Swen
> 
> It looks like dirent caching does buy us something, though
> interestingly the
> initial populating tends to be the quickest run...
> 
> These were run with Ganesha having freshly started, and the following
> script:
> 
> mount /mnt4
> time ls -l /mnt4/test1/many | wc
> echo 3 > /proc/sys/vm/drop_caches
> time ls -l /mnt4/test1/many | wc
> echo 3 > /proc/sys/vm/drop_caches
> time ls -l /mnt4/test1/many | wc
> echo 3 > /proc/sys/vm/drop_caches
> time ls -l /mnt4/test1/many | wc
> echo 3 > /proc/sys/vm/drop_caches
> time ls -l /mnt4/test1/many | wc
> echo 3 > /proc/sys/vm/drop_caches
> time ls -l /mnt4/test1/many | wc
> echo 3 > /proc/sys/vm/drop_caches
> time ls -l /mnt4/test1/many | wc
> echo 3 > /proc/sys/vm/drop_caches
> time ls -l /mnt4/test1/many | wc
> echo 3 > /proc/sys/vm/drop_caches
> time ls -l /mnt4/test1/many | wc
> echo 3 > /proc/sys/vm/drop_caches
> time ls -l /mnt4/test1/many | wc
> echo 3 > /proc/sys/vm/drop_caches
> umount /mnt4
> 
> With the results piped to grep real.
> 
> Frank
> 
> 
> ---
> This email has been checked for viruses by Avast antivirus software.
> https://www.avast.com/antivirus
> 
> 
> ---
> ---
> Developer Access Program for Intel Xeon Phi Processors
> Access to Intel Xeon Phi processor-based developer platforms.
> With one year of Intel Parallel Studio XE.
> Training and support from Colfax.
> Order your platform today. http://sdm.link/xeonphi
> ___
> Nfs-ganesha-devel mailing list
> Nfs-ganesha-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel
> 


--
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
___
Nfs-ganesha-devel mailing list
Nfs-ganesha-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel

Re: [Nfs-ganesha-devel] Deprecating non-support_ex

[LUCAS Patrice]

I'm currently working on converting FSAL_PROXY to support_ex but it will 
take time.

Patrice

On 01/10/17 23:33, Frank Filz wrote:
> We had discussed that in 2.5 we would deprecate non-support_ex and strip out
> the old API interfaces.
>
> I'm not sure if we still want to do that (I would like to, to simplify code,
> and not continue to support clunky interfaces).
>
> I'm not sure we converted all in-tree FSALs to support_ex...
>
> Frank
>
>
> ---
> This email has been checked for viruses by Avast antivirus software.
> https://www.avast.com/antivirus
>
>
> --
> Developer Access Program for Intel Xeon Phi Processors
> Access to Intel Xeon Phi processor-based developer platforms.
> With one year of Intel Parallel Studio XE.
> Training and support from Colfax.
> Order your platform today. http://sdm.link/xeonphi
> ___
> Nfs-ganesha-devel mailing list
> Nfs-ganesha-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel


-- 
Patrice LUCAS
Ingenieur-Chercheur, CEA-DAM/DSSI/SISR/LA2S
tel : +33 (0)1 69 26 47 86
e-mail : patrice.lu...@cea.fr


--
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
___
Nfs-ganesha-devel mailing list
Nfs-ganesha-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel