The Slurm APIs take both formats for exactly that reason.
Quoting E V <[email protected]>:
I think we could use the first format, if we had corresponding
functions to interact with the jobs using those id's, querying job
info and holding and resuming jobs and such. It would be nice if these
functions worked with either of the 2 job formats you mentioned so
the drmaa code doesn't always need to fork on the type of job when
interacting with jobs. I know it already naturally has two code paths
on the job submission between arrays and regular jobs, but I don't
think most of the other API calls do. I'm not an expert on the drmaa
code base though so might be more gotchas lurking, however it
certainly sounds less daunting to me then redesigning the drmaa api's
;-)
On Wed, Sep 2, 2015 at 3:32 PM, Moe Jette <[email protected]> wrote:
The Slurm APIs recognize two different formats for job array IDs. A job
array with two tasks might be known by the names "123_1" and "123_2" OR
"123" and "124". The tasks of the job array are known by the first format at
submit time. The second format will not be known until allocation time.
Could you use the first format for job IDs?
Quoting E V <[email protected]>:
Hmm, that's a bit of an issue as that behaviour appears to break the
drmaa_run_bulk_jobs() API and the drmaa slurm bindings in c, perl,
python, ruby and java as far as I know. I guess my hope for a
straightforward upgrade beyond 14.03 isn't happening any time soon.
On Wed, Sep 2, 2015 at 1:02 PM, Moe Jette <[email protected]> wrote:
Quoting E V <[email protected]>:
Huh, so we don't know the jobs id's of the individual jobs at the
array submit time anymore?
No. We have some users submitting job arrays with millions of tasks and
do
minimal processing at job submit time for performance reasons. The job
IDs
for individual tasks are selected at resource allocation time.
I think the drmaa wrapper I use assumes we
get back a list of job id's from the array submission. Any way to
query this information?
Generally not before resource allocation time.
On Tue, Sep 1, 2015 at 5:56 PM, Moe Jette <[email protected]> wrote:
For scalability reasons, when a job array is submitted only one job
record
is created. As individual tasks in the job array start running, a new
job
record is created for each task. The relevant parts in the job record
(from
slurm.h) are:
uint32_t array_job_id; /* job_id of a job array or 0 if N/A */
uint32_t array_task_id; /* task_id of a job array */
uint32_t array_max_tasks; /* Maximum number of running tasks */
char *array_task_str; /* string expression of task IDs in
this
record */
Here's the code from src/api/job_info.c that demonstrates how to use
that
information:
if (job_ptr->array_job_id) {
if (job_ptr->array_task_str) {
snprintf(tmp_line, sizeof(tmp_line),
"ArrayJobId=%u ArrayTaskId=%s ",
job_ptr->array_job_id,
job_ptr->array_task_str);
} else {
snprintf(tmp_line, sizeof(tmp_line),
"ArrayJobId=%u ArrayTaskId=%u ",
job_ptr->array_job_id,
job_ptr->array_task_id);
}
xstrcat(out, tmp_line);
}
Quoting E V <[email protected]>:
Trying drmaa with slurm > 14.03 using job arrays for the first time.
Doesn't work.
The drmaa code in session.c:slurmdrmaa_session_run_bulk() successfully
submits my 2-job array(via slurm_submit_batch_job().) However, it
calls slurm_load_job(&job_info,...) and expects job_info->record_count
to be == the number of jobs in the array in order to iterate and
create jobs for all the jobs in the array (via slurmdrmaa_job_new().)
I guess this behaviour has changed in 14.11+, because record_count is
now == 1. I see this new job_array_resp_msg_t type, but I don't see a
corresponding load_job function in slurm.h using it. How does the
handling of job arrays work now in slurm >14.03?
--
Morris "Moe" Jette
CTO, SchedMD LLC
Commercial Slurm Development and Support
===============================================================
Slurm User Group Meeting, 15-16 September 2015, Washington D.C.
http://slurm.schedmd.com/slurm_ug_agenda.html
--
Morris "Moe" Jette
CTO, SchedMD LLC
Commercial Slurm Development and Support
===============================================================
Slurm User Group Meeting, 15-16 September 2015, Washington D.C.
http://slurm.schedmd.com/slurm_ug_agenda.html
--
Morris "Moe" Jette
CTO, SchedMD LLC
Commercial Slurm Development and Support
===============================================================
Slurm User Group Meeting, 15-16 September 2015, Washington D.C.
http://slurm.schedmd.com/slurm_ug_agenda.html
--
Morris "Moe" Jette
CTO, SchedMD LLC
Commercial Slurm Development and Support
===============================================================
Slurm User Group Meeting, 15-16 September 2015, Washington D.C.
http://slurm.schedmd.com/slurm_ug_agenda.html
