Hi Scale-folk. I have an IBM ESS GH14S building block currently configured for my HPC workloads.
I've got about 1PB of /scratch filesystem configured in mechanical spindles via GNR and about 20TB of SSD/flash sitting in another GNR filesystem at the moment. My intention is to destroy that stand-alone flash filesystem eventually and use storage pools coupled with GPFS policy to warm up workloads into that flash storage: https://www.ibm.com/support/knowledgecenter/en/STXKQY_4.2.3/com.ibm.spectrum.scale.v4r23.doc/bl1adv_storagepool.htm A little dated, but that kind of thing. Does anyone have any experience in this space in using flash storage inside a pool with pre/post flight SLURM scripts to puppeteer GPFS policy to warm data up? I had a few ideas for policy construction around file size, file count, file access intensity. Someone mentioned heat map construction and mmdiag --iohist to me the other day. Could use some background there. If anyone has any SLURM specific integration tips for the scheduler or pre/post flight bits for SBATCH, it'd be really very much appreciated. This array really does fly along and surpassed my expectations - but, I want to get the most out of it that I can for my users - and I think storage pool automation and good file placement management is going to be an important part of that. Thank you. -jc
_______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
