The message below should read "epilog" rather than "prolog".
Slurm only tracks the processes that it's daemons launch (most MPI implementations can launch their tasks using slurm). Anything launched outside of Slurm can be killed as part of a job epilog, but accounting and job step management are outside of Slurm's control. The epilog can check if user still has a Slurm job allocated to the node and if not, kill all processes owned by that user. Quoting Moe Jette <[email protected]>: > > Slurm only tracks the processes that it's daemons launch (most MPI > implementations can launch their tasks using slurm). Anything launched > outside of Slurm can be killed as part of a job prolog, but accounting > and job step management are outside of Slurm's control. > > Quoting Michael Colonno <[email protected]>: > >> >> SLURM gurus ~ >> >> I'm trying to configure a commercial MPI code to run through SLURM. >> I can launch this code through either srun or sbatch without any >> issues (the good) but the processes manage to run completely >> disconnected from SLURM's notice (the bad). i.e. the job is running >> just fine but SLURM thinks it's completed and hence does not report >> anything running. I'm guessing this is due to the fact that this >> tool runs a pre-processing-type executable and then launches >> sub-processes to solve (MPI on a local system) without connecting >> the process IDs(?) In any event, I'm guessing I'm not the first >> person to run into this. Is there a recommended solution to >> configure SLURM to track codes like this? >> >> Thanks, >> ~Mike C. >> >> > >
