On Mon, 01 Jun 2009 19:02:11 +0200, Mark Weghorst <mark at
productioninsight.com> wrote:
> I have a HPC application that consists of a compute grid of Linux systems
> which read a common set of data from read-only NFS mounts and then use that
> data to produce computations which are consumed by an IBM mainframe.
>
> I have run into some problems with the Linux kernel and I am wondering if I
> would have a similar issue on OSx86. There is significant work involved in a
> port, so I would like to get an idea if I would have the same issues prior to
> undertaking this development effort.
>
> The NFS workload is 100% read-only (RO mount) and consists of the following:
>
> GETATTR 31.34%
> LOOKUP 6.61%
> [b] ACCESS 44.15% [/b]
> READ 11.98%
> READDIRPLUS 5.21%
>
> The problem is that there is considerable cost (in the deployment and
> maintenance of NAS appliances)so each and every NFS operation has a cost
> associated with it. [b] In our case 44% of our NFS operations are
> unnecessary, as we don't use any POSIX ACL's in our application. By
> eliminating these RPC calls, we will save a considerable amount of money on
> storage [/b]
>
> If we were able to disable the ACCESS RPC, then that would help us to get a
> higher density of compute nodes to NAS. In the Linux 2.4 kernels there was a
> "noacl" > option that would suppress these calls, and this is exactly what we
> need for our application. However, that was dropped in the 2.6 kernels, and
> will not be reintroduced.
>
> With that context, I have a few questions about the OS NFS client:
>
> 1) Does the OS NFS client support a way to "suppress" ACCESS calls?
not that I am aware of.
but I'm not quite sure I understand your point of ACLs along with the ACCESS
procedure, even if you do not use ACLs
on the servers file system, the client will and has to perform OTW permission
checking when acessing objects on
the server, and that is what is being done with ACCESS.
however, if you use NFS v4, which is the default in Solaris since Solaris 10
anyways, you will get reduced use
of ACCESS operations. that is because with V4, there's the new OPEN procedure,
and the server can and will verify
access permissions at the time it receives the OPEN for the file from the
client. the client may still use
ACCESS though in some situations, eg. when the application on the client
directly invokes the access(2) system call.
> 2) Does the OS NFS client have a memory cache similar to file system buffers?
> Our application does a huge amount of disk I/O and we currently use the NFS
> memory > cache in the Linux kernel to reduce our NFS ops/sec. For reference
> 32GB of RAM is enough to prevent 95% of disk I/O
the Solaris NFS client does use the page cache to cache pages involved in its
I/O operations. (NB. this has nothing todo with the buffer cache)
> 3) Does the OS NFS client support an attribute cache, and the nocto option?
yes, see mount_nfs(1M)
nocto
Do not perform the normal close-to-open consistency.
When a file is closed, all modified data associated
with the file is flushed to the server and not held
on the client. When a file is opened the client
sends a request to the server to validate the
client's local caches. This behavior ensures a
file's consistency across multiple NFS clients. When
-nocto is in effect, the client does not perform the
flush on close and the request for validation,
allowing the possiblity of differences among copies
of the same file as stored on multiple clients.
This option can be used where it can be guaranteed
that accesses to a specified file system are made
from only one client and only that client. Under
such a condition, the effect of -nocto can be a
slight performance gain.
cheers
--
frankB
