One major issue is the maxFilesToCache and maybe the maxStatCache (though I
hear that Linux negates the use of this parameter now? I don’t quite
remember). Ganesha apparently likes to hold open a large number of files and
this means that it will quickly fill up the maxFilesToCache. When this happens
the [gpfsSwapdKproc] process will start to eat up CPU time. This is the daemon
that tries to find a file to evict from the cache when a new file is opened.
This overhead will also hurt performance.
IBM in a PMR we opened suggested setting this to something like 5 Million for
the protocol nodes. I think we started with 1.5 Million. You have to be
mindful of memory requirements on the token servers to handle the total sum of
all maxFilesToCache settings from all nodes that mount the file system.
Of course the other, standard NFS tuning parameters for number of threads and
NFS client mount options still should be adjusted too.
Hope that helps,
[mailto:gpfsug-discuss-boun...@spectrumscale.org] On Behalf Of Oesterlin, Robert
Sent: Sunday, October 16, 2016 7:06 PM
To: gpfsug main discussion list <email@example.com>
Subject: [gpfsug-discuss] CES and NFS Tuning suggestions
Looking for some pointers or suggestions on what I should look at changing in
Linux and/or GPFS "mmchconfg" settings to help boost NFS performance. Out of
the box it seems "poor".
Sr Storage Engineer, Nuance HPC Grid
Note: This email is for the confidential use of the named addressee(s) only and
may contain proprietary, confidential or privileged information. If you are not
the intended recipient, you are hereby notified that any review, dissemination
or copying of this email is strictly prohibited, and to please notify the
sender immediately and destroy this email and any attachments. Email
transmission cannot be guaranteed to be secure or error-free. The Company,
therefore, does not make any guarantees as to the completeness or accuracy of
this email or any attachments. This email is for informational purposes only
and does not constitute a recommendation, offer, request or solicitation of any
kind to buy, sell, subscribe, redeem or perform any type of transaction of a
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org