We do this in the job epilog. We do not allow node sharing, so all our jobs run exclusively and there’s no real harm. From time to time, in the past, the Lustre client might experience a crash leading to a node reboot (thus the reason we do it in epilog), but that’s become very rare or stopped.
Best, Bill. -- Bill Barth, Ph.D., Director, HPC [email protected] | Phone: (512) 232-7069 Office: ROC 1.435 | Fax: (512) 475-9445 On 3/30/17, 10:52 AM, "Chad Cropper" <[email protected]> wrote: We would like to clean our memory buffers/cache on a regular basis without rebooting nodes. We can easily do this manually with “sync; echo 3 > /proc/sys/vm/drop_caches“. Is anyone else out there doing anything like this? Does SLURM offer anything builtin for running this when it sees an empty node? Outside of rotating nodes into a drain state for maintenance and then running this command, I have yet to see any other option. Any suggestions are greatly appreciated. -Chad Cropper ________________________________________ *** The information contained in this communication may be confidential, is intended only for the use of the recipient(s) named above, and may be legally privileged. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution, or copying of this communication, or any of its contents, is strictly prohibited. If you have received this communication in error, please return it to the sender immediately and delete the original message and any copies of it. If you have any questions concerning this message, please contact the sender. ***
