Hi, I've set up epilog in slurm.conf, it runs well and can be called to clean the user processes even when I manually cancel a job. But when a user process crashed a node of running out of memory, the epilog was no longer called.
I am not quite clear if it is normal behaviour or not as the slurm might not be able to run epilog when running out the memory. Or the slurm failed to trig epilog cancel signal, or slurm.epilog.clean must be installed in slurm etc directory which can be called by slurm epilog cancel signal as per /etc/slurm https://github.com/chaos/slurm/blob/master/etc/slurm.epilog.clean? Appreciate clarification Anyway, I've added slurm.epilog.clean to slurm etc directory and hope I can test verify it by reproducing out of memory, unfortunately, I could not reproduce it by even running a fake memory leak program, the node just won't crash to the status of out of memory. Thank you. Kind regards, jupiter
