Hi all, I'm submitting computing jobs using the sbatch --array option. I'd like to get notified about the progress of the "master" job, but not about the progress of each of its up to several hundred sub tasks.
I use the sbatch command like this to launch the jobs: sbatch --array 0-59 --mail-type=ALL --mail-user=<myemail> rfi.job but that notifies me of the progress of each of the 60 sub tasks, which is not what I intended. The reason I think this should work by notifying me only for the parent task is that when the rfi.job batch is called for each array item, the SLURM_JOB_ID in the environment is the same for each of the 60 calls, which would hint at all of the tasks running under the same job ID. The SLURM_ARRAY_TASK_ID, as expected, then changes for every call. Showing the job queue using squeue, I see all tasks having the same job ID, with _tasknumber attached, again as expected. But when using sacct, I see that each array task has a different job ID. This is where the confusion enters, are they really separate jobs, or are they one parent job and sub tasks? If the latter is true, that would explain the email behaviour. But that also then points at a fairly obvious need: Isn't there a built in way to group or attach an array of jobs to one master job, and get notified when the first sub-task begins execution, and when the last sub-task has completed, plus notification if any of the sub tasks have failed? Any input greatly appreciated. Cheers - Balt Dr Balthasar Indermuehle CSIRO Astronomy and Space Science t: +61 2 9372 4274 m: +61 4 2791 2856
