[Nfs-ganesha-devel] Change in ffilz/nfs-ganesha[next]: NFS4: allow reads and writes against delegations that have been recal...

[GerritHub]

>From Jeff Layton :

Jeff Layton has uploaded this change for review. ( 
https://review.gerrithub.io/400650


Change subject: NFS4: allow reads and writes against delegations that have been 
recalled but not yet returned
..

NFS4: allow reads and writes against delegations that have been recalled but 
not yet returned

We need to allow a client to flush its writes after a recall, and it's
possible we could get reads racing in here too.

Since revoked delegations are deleted immediately, we can just remove
this check in the read and write codepaths.

Change-Id: Id58c900b3b217d8a4b775db2db48968a50b6739c
Signed-off-by: Jeff Layton 
---
M src/Protocols/NFS/nfs4_op_read.c
M src/Protocols/NFS/nfs4_op_write.c
2 files changed, 2 insertions(+), 4 deletions(-)



  git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha 
refs/changes/50/400650/1
--
To view, visit https://review.gerrithub.io/400650
To unsubscribe, visit https://review.gerrithub.io/settings

Gerrit-Project: ffilz/nfs-ganesha
Gerrit-Branch: next
Gerrit-MessageType: newchange
Gerrit-Change-Id: Id58c900b3b217d8a4b775db2db48968a50b6739c
Gerrit-Change-Number: 400650
Gerrit-PatchSet: 1
Gerrit-Owner: Jeff Layton 
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Nfs-ganesha-devel mailing list
Nfs-ganesha-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel

Re: [Nfs-ganesha-devel] nfs ganesha vs nfs kernel performance

[Matt Benjamin]

Hi,

On Tue, Feb 20, 2018 at 8:12 AM, William Allen Simpson
 wrote:
> On 2/18/18 2:47 PM, Matt Benjamin wrote:
>>
>> On Fri, Feb 16, 2018 at 11:23 AM, William Allen Simpson
>>>
>>> But the planned 2.7 improvements are mostly throughput related, not IOPS.
>>
>>
>> Not at all, though I am trying to ensure that we get async FSAL ops
>> in.  There are people working on IOPs too.
>>
> async FSAL ops are not likely to further improve IOPS.
>
> As I've pointed out many times in the past, async only allows
> the same number of threads to handle more concurrent operations.
>
> But it's actually slower.  It basically doubles the number of
> system calls.  System calls are one of the reasons that Ganesha is
> slower than knfsd.  It also ruins CPU cache coherency.
>
> If we're CPU bound or network bandwidth constrained, it won't help.

Not everything being worked on is a direct answer to this problem.

>
> The effort that I've been waiting for *3* years -- add io vectors to
> the FSAL interface, so that we can zero-copy buffers -- is more
> likely to improve throughput.

Which, as you've noted, also is happening.

>
> Moreover, going through the code and removing locking other such
> bottlenecks is more likely to improve IOPS.

No one is disputing this.  It is not a new discovery, however.

>
>
>>> If Ganesha is adding 6 ms to every read operation, we have a serious
>>> problem, and need to profile immediately!
>>>
>>
>> That's kind of what our team is doing.  I look forward to your work
>> with rpc-ping-mt.
>>
> Well, you were able to send me a copy after 6 pm on a Friday, so I'm
> now taking a look at it.  Hopefully I'll have something by Friday.

It came up in a meeting on Friday, that's why I sent it Friday.  I got
busy in meetings and issues, that's why after 6 pm.

>
> But I really wish you'd posted it 3 years ago.  It doesn't really test
> IOPS, other than whatever bandwidth limit is imposed by the interface,
> but it does test the client call interface.

It measures minimal request latency all the way to nfs-ganesha's, or
the Linux kernel's, rpcnull handler--the "top" of the request handling
stack, in the given client/server/network configuration.  Scaled up,
it can report the max such calls/s, which is a kind of best-possible
value for iops, taking FSAL ops to have 0 latency.

It was posted to this list by Tigran, iirc, in 2011 or 2012.

>
> We've been using Jeff Layton's delegation callback work to test, and
> this test would have been better and easier.
>
> But a unit test is not what we need.  I wrote "profile".  We need to
> know where the CPU bottlenecks are in Ganesha itself.

You also wrote unit test.

Matt

-- 

Matt Benjamin
Red Hat, Inc.
315 West Huron Street, Suite 140A
Ann Arbor, Michigan 48103

http://www.redhat.com/en/technologies/storage

tel.  734-821-5101
fax.  734-769-8938
cel.  734-216-5309

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Nfs-ganesha-devel mailing list
Nfs-ganesha-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel

[Nfs-ganesha-devel] gtest results? profiles?

[William Allen Simpson]

Now that we have a pile of nice gtests, who is compiling the results?
Please post them here

Also, DanG told me yesterday that he has a profile of the lookup
test.  Please post that here.  That will allow us to better target
the CPU bottlenecks.

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Nfs-ganesha-devel mailing list
Nfs-ganesha-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel

Re: [Nfs-ganesha-devel] nfs ganesha vs nfs kernel performance

[William Allen Simpson]

On 2/18/18 2:47 PM, Matt Benjamin wrote:

On Fri, Feb 16, 2018 at 11:23 AM, William Allen Simpson

But the planned 2.7 improvements are mostly throughput related, not IOPS.


Not at all, though I am trying to ensure that we get async FSAL ops
in.  There are people working on IOPs too.


async FSAL ops are not likely to further improve IOPS.

As I've pointed out many times in the past, async only allows
the same number of threads to handle more concurrent operations.

But it's actually slower.  It basically doubles the number of
system calls.  System calls are one of the reasons that Ganesha is
slower than knfsd.  It also ruins CPU cache coherency.

If we're CPU bound or network bandwidth constrained, it won't help.

The effort that I've been waiting for *3* years -- add io vectors to
the FSAL interface, so that we can zero-copy buffers -- is more
likely to improve throughput.

Moreover, going through the code and removing locking other such
bottlenecks is more likely to improve IOPS.



If Ganesha is adding 6 ms to every read operation, we have a serious
problem, and need to profile immediately!



That's kind of what our team is doing.  I look forward to your work
with rpc-ping-mt.


Well, you were able to send me a copy after 6 pm on a Friday, so I'm
now taking a look at it.  Hopefully I'll have something by Friday.

But I really wish you'd posted it 3 years ago.  It doesn't really test
IOPS, other than whatever bandwidth limit is imposed by the interface,
but it does test the client call interface.

We've been using Jeff Layton's delegation callback work to test, and
this test would have been better and easier.

But a unit test is not what we need.  I wrote "profile".  We need to
know where the CPU bottlenecks are in Ganesha itself.

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Nfs-ganesha-devel mailing list
Nfs-ganesha-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel

Re: [Nfs-ganesha-devel] Change in ffilz/nfs-ganesha[next]: "mdc_lookup" do not dispatch to FSAL

[William Allen Simpson]

On 2/19/18 12:12 PM, Sachin Punadikar wrote:

Hi Bill,
I rechecked the logs & discussed with Daniel. I missed to see the log entries 
related to FSAL.
So for this customer it looks like FSAL issue than a Ganesha issue


Thanks for the update.  The default log levels don't always show enough.



--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Nfs-ganesha-devel mailing list
Nfs-ganesha-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel