On Tuesday, 12 June 2018 6:06:06 PM AEST John Hearns via Beowulf wrote: > What do most sites do for scratch space?
At ${JOB-1} we used GPFS and so for the scratch filesystem we used the GPFS policy engine to identify and remove files that had not been read/written for more than the defined number of days (twice the length of time of our longest permitted job, so 2 x 30 days). With the policy engine you can parallelise that across NSD servers too (we had separate metadata and data NSD servers, so identifying the files was just parallelised across the former which accessed the same shared SSD array). Our compute nodes were diskless too so for jobs we used a plugin which created per-job directories under scratch and then the epilog would clean them up. cheers! Chris -- Chris Samuel : http://www.csamuel.org/ : Melbourne, VIC _______________________________________________ Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf