As an aside, my initial attempt was to use Ganesha via CES but the
performance was significantly worse than CNFS for this workload. The
docs seem to suggest that CNFS performs better for metadata intensive
workloads which certainly seems to fit the bill here.
-Aaron
On 9/10/17 8:43 PM, Aaron Knister wrote:
Hi All (but mostly Sven),
I stumbled across this great gem:
files.gpfsug.org/presentations/2014/UG10_GPFS_Performance_Session_v10.pdf
and I'm wondering which, if any, of those tuning parameters are still
relevant with the 4.2.3 code. Specifically for a CNFS cluster. I'm
exporting a gpfs fs as an NFS root to 1k nodes. The boot storm is
particularly ugly and the storage doesn't appear to be bottlenecked.
I see a lot of waiters like these:
Waiting 0.0009 sec since 20:41:31, monitored, thread 2881
InodePrefetchWorkerThread: on ThCond 0x1800635A120 (LkObjCondvar),
reason 'waiting for LX lock'
Waiting 0.0009 sec since 20:41:31, monitored, thread 26231
InodePrefetchWorkerThread: on ThCond 0x1800635A120 (LkObjCondvar),
reason 'waiting for LX lock'
Waiting 0.0009 sec since 20:41:31, monitored, thread 26146
InodePrefetchWorkerThread: on ThCond 0x1800635A120 (LkObjCondvar),
reason 'waiting for LX lock'
Waiting 0.0009 sec since 20:41:31, monitored, thread 18637
InodePrefetchWorkerThread: on ThCond 0x1800635A120 (LkObjCondvar),
reason 'waiting for LX lock'
Waiting 0.0009 sec since 20:41:31, monitored, thread 25013
InodePrefetchWorkerThread: on ThCond 0x1800635A120 (LkObjCondvar),
reason 'waiting for LX lock'
Waiting 0.0009 sec since 20:41:31, monitored, thread 27879
InodePrefetchWorkerThread: on ThCond 0x1800635A120 (LkObjCondvar),
reason 'waiting for LX lock'
Waiting 0.0009 sec since 20:41:31, monitored, thread 26553
InodePrefetchWorkerThread: on ThCond 0x1800635A120 (LkObjCondvar),
reason 'waiting for LX lock'
Waiting 0.0009 sec since 20:41:31, monitored, thread 25334
InodePrefetchWorkerThread: on ThCond 0x1800635A120 (LkObjCondvar),
reason 'waiting for LX lock'
Waiting 0.0009 sec since 20:41:31, monitored, thread 25337
InodePrefetchWorkerThread: on ThCond 0x1800635A120 (LkObjCondvar),
reason 'waiting for LX lock'
and I'm wondering if there's anything immediate one would suggest to
help with that.
-Aaron
--
Aaron Knister
NASA Center for Climate Simulation (Code 606.2)
Goddard Space Flight Center
(301) 286-2776
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
