this really depends on your application workload.
If your 2 node NSD cluster is the storage cluster for some remote cluster(s), you probably don't need such a high MaxFilesToCache value (nor a similar high maxStatCache value) at all.
If there is an application workload on that cluster as well, and this is mainly working on a high number of files (like a login node for hpc applications of numerous users would do) a high number of maxStatCache would only waste these 500 bytes (x 3000000) (even if that is just 1.5GB) since the information is in the maxFilesToCache entry as well.
If your node however does loads of metadata scans, inode traversal work ... mmbackup, policy runs, then a high number for maxStatCache would be beneficial.
John's (or Tomer's) presentation on memory usage gives details on what the memory footprint of one or the other would look like.
Mit freundlichen Grüßen / Kind regards
Achim Rehor
From:
Philipp Grau <[email protected]>
To:
gpfsug main discussion
list <[email protected]>
Date:
13/03/2020 13:49
Subject:
[EXTERNAL] [gpfsug-discuss]
maxStatCache and maxFilesToCache: Tip "gpfs_maxstatcache_low".
Sent by:
[email protected]
Hello,
we have a two node NSD cluster based on a DDN system. Currently we
run Spectrum Scale 5.0.4.1 in an HPC environment.
Mmhealth shows a tip stating "gpfs_maxstatcache_low". Our current settings are:
# mmdiag --config | grep -i cache
! maxFilesToCache 3000000
maxStatCache 10000
maxFilesToCache was tuned during installion and maxStatCache is the
according default value.
After discussing this issue on the german spectumscale meeting, I
understand that it is difficult to give a formula on howto calulate
this values.
But I learnt that a FilesToCache entry costs about 10 kbytes of memory
and a StatCache entry about 500 bytes. And typically maxStatCache
should (obviously) be greater than maxFilesToCache. There is a average
100 GB memory usage on our systems (with a total of 265 GB RAM).
So setting maxStatCache to at least 3000000 should be no problem. But
is that correct or to high/low?
Has anyone some hints or thoughts on this topic? Help is welcome.
Regards,
Philipp
--
Philipp Grau | Freie Universitaet Berlin
[email protected] | Zentraleinrichtung fuer Datenverarbeitung
Tel: +49 (30) 838 56583 | Fabeckstr. 32
Fax: +49 (30) 838 56721 | 14195 Berlin
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
_______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
