Frank Batschulat (Home) wrote:
> On Mon, 01 Jun 2009 19:02:11 +0200, Mark Weghorst <mark at
> productioninsight.com> wrote:
>
>
>> I have a HPC application that consists of a compute grid of Linux systems
>> which read a common set of data from read-only NFS mounts and then use that
>> data to produce computations which are consumed by an IBM mainframe.
>>
>> I have run into some problems with the Linux kernel and I am wondering if I
>> would have a similar issue on OSx86. There is significant work involved in
>> a port, so I would like to get an idea if I would have the same issues prior
>> to undertaking this development effort.
>>
>> The NFS workload is 100% read-only (RO mount) and consists of the following:
>>
>> GETATTR 31.34%
>> LOOKUP 6.61%
>> [b] ACCESS 44.15% [/b]
>> READ 11.98%
>> READDIRPLUS 5.21%
>>
>> The problem is that there is considerable cost (in the deployment and
>> maintenance of NAS appliances)so each and every NFS operation has a cost
>> associated with it. [b] In our case 44% of our NFS operations are
>> unnecessary, as we don't use any POSIX ACL's in our application. By
>> eliminating these RPC calls, we will save a considerable amount of money on
>> storage [/b]
>>
>> If we were able to disable the ACCESS RPC, then that would help us to get a
>> higher density of compute nodes to NAS. In the Linux 2.4 kernels there was
>> a "noacl" > option that would suppress these calls, and this is exactly what
>> we need for our application. However, that was dropped in the 2.6 kernels,
>> and will not be reintroduced.
>>
>> With that context, I have a few questions about the OS NFS client:
>>
>> 1) Does the OS NFS client support a way to "suppress" ACCESS calls?
>>
>
> not that I am aware of.
>
> but I'm not quite sure I understand your point of ACLs along with the ACCESS
> procedure, even if you do not use ACLs
> on the servers file system, the client will and has to perform OTW permission
> checking when acessing objects on
> the server, and that is what is being done with ACCESS.
>
>
yes, clarifications would help. The client can also try to increase the
attribute cache duration using the 'actiemo' mount option. See
mount_nfs(1M).
Mahesh
> however, if you use NFS v4, which is the default in Solaris since Solaris 10
> anyways, you will get reduced use
> of ACCESS operations. that is because with V4, there's the new OPEN
> procedure, and the server can and will verify
> access permissions at the time it receives the OPEN for the file from the
> client. the client may still use
> ACCESS though in some situations, eg. when the application on the client
> directly invokes the access(2) system call.
>
>
>> 2) Does the OS NFS client have a memory cache similar to file system
>> buffers? Our application does a huge amount of disk I/O and we currently
>> use the NFS memory > cache in the Linux kernel to reduce our NFS ops/sec.
>> For reference 32GB of RAM is enough to prevent 95% of disk I/O
>>
>
> the Solaris NFS client does use the page cache to cache pages involved in its
> I/O operations. (NB. this has nothing todo with the buffer cache)
>
>
>> 3) Does the OS NFS client support an attribute cache, and the nocto option?
>>
>
> yes, see mount_nfs(1M)
>
> nocto
>
> Do not perform the normal close-to-open consistency.
> When a file is closed, all modified data associated
> with the file is flushed to the server and not held
> on the client. When a file is opened the client
> sends a request to the server to validate the
> client's local caches. This behavior ensures a
> file's consistency across multiple NFS clients. When
> -nocto is in effect, the client does not perform the
> flush on close and the request for validation,
> allowing the possiblity of differences among copies
> of the same file as stored on multiple clients.
>
> This option can be used where it can be guaranteed
> that accesses to a specified file system are made
> from only one client and only that client. Under
> such a condition, the effect of -nocto can be a
> slight performance gain.
>
>
> cheers
>
