The Slurm APIs recognize two different formats for job array IDs. A
job array with two tasks might be known by the names "123_1" and
"123_2" OR "123" and "124". The tasks of the job array are known by
the first format at submit time. The second format will not be known
until allocation time. Could you use the first format for job IDs?
Quoting E V <[email protected]>:
Hmm, that's a bit of an issue as that behaviour appears to break the
drmaa_run_bulk_jobs() API and the drmaa slurm bindings in c, perl,
python, ruby and java as far as I know. I guess my hope for a
straightforward upgrade beyond 14.03 isn't happening any time soon.
On Wed, Sep 2, 2015 at 1:02 PM, Moe Jette <[email protected]> wrote:
Quoting E V <[email protected]>:
Huh, so we don't know the jobs id's of the individual jobs at the
array submit time anymore?
No. We have some users submitting job arrays with millions of tasks and do
minimal processing at job submit time for performance reasons. The job IDs
for individual tasks are selected at resource allocation time.
I think the drmaa wrapper I use assumes we
get back a list of job id's from the array submission. Any way to
query this information?
Generally not before resource allocation time.
On Tue, Sep 1, 2015 at 5:56 PM, Moe Jette <[email protected]> wrote:
For scalability reasons, when a job array is submitted only one job
record
is created. As individual tasks in the job array start running, a new job
record is created for each task. The relevant parts in the job record
(from
slurm.h) are:
uint32_t array_job_id; /* job_id of a job array or 0 if N/A */
uint32_t array_task_id; /* task_id of a job array */
uint32_t array_max_tasks; /* Maximum number of running tasks */
char *array_task_str; /* string expression of task IDs in this
record */
Here's the code from src/api/job_info.c that demonstrates how to use that
information:
if (job_ptr->array_job_id) {
if (job_ptr->array_task_str) {
snprintf(tmp_line, sizeof(tmp_line),
"ArrayJobId=%u ArrayTaskId=%s ",
job_ptr->array_job_id,
job_ptr->array_task_str);
} else {
snprintf(tmp_line, sizeof(tmp_line),
"ArrayJobId=%u ArrayTaskId=%u ",
job_ptr->array_job_id,
job_ptr->array_task_id);
}
xstrcat(out, tmp_line);
}
Quoting E V <[email protected]>:
Trying drmaa with slurm > 14.03 using job arrays for the first time.
Doesn't work.
The drmaa code in session.c:slurmdrmaa_session_run_bulk() successfully
submits my 2-job array(via slurm_submit_batch_job().) However, it
calls slurm_load_job(&job_info,...) and expects job_info->record_count
to be == the number of jobs in the array in order to iterate and
create jobs for all the jobs in the array (via slurmdrmaa_job_new().)
I guess this behaviour has changed in 14.11+, because record_count is
now == 1. I see this new job_array_resp_msg_t type, but I don't see a
corresponding load_job function in slurm.h using it. How does the
handling of job arrays work now in slurm >14.03?
--
Morris "Moe" Jette
CTO, SchedMD LLC
Commercial Slurm Development and Support
===============================================================
Slurm User Group Meeting, 15-16 September 2015, Washington D.C.
http://slurm.schedmd.com/slurm_ug_agenda.html
--
Morris "Moe" Jette
CTO, SchedMD LLC
Commercial Slurm Development and Support
===============================================================
Slurm User Group Meeting, 15-16 September 2015, Washington D.C.
http://slurm.schedmd.com/slurm_ug_agenda.html
--
Morris "Moe" Jette
CTO, SchedMD LLC
Commercial Slurm Development and Support
===============================================================
Slurm User Group Meeting, 15-16 September 2015, Washington D.C.
http://slurm.schedmd.com/slurm_ug_agenda.html
