Hmm, that's a bit of an issue as that behaviour appears to break the drmaa_run_bulk_jobs() API and the drmaa slurm bindings in c, perl, python, ruby and java as far as I know. I guess my hope for a straightforward upgrade beyond 14.03 isn't happening any time soon.
On Wed, Sep 2, 2015 at 1:02 PM, Moe Jette <[email protected]> wrote: > > Quoting E V <[email protected]>: >> >> Huh, so we don't know the jobs id's of the individual jobs at the >> array submit time anymore? > > > No. We have some users submitting job arrays with millions of tasks and do > minimal processing at job submit time for performance reasons. The job IDs > for individual tasks are selected at resource allocation time. > >> I think the drmaa wrapper I use assumes we >> get back a list of job id's from the array submission. Any way to >> query this information? > > > Generally not before resource allocation time. > > >> On Tue, Sep 1, 2015 at 5:56 PM, Moe Jette <[email protected]> wrote: >>> >>> >>> For scalability reasons, when a job array is submitted only one job >>> record >>> is created. As individual tasks in the job array start running, a new job >>> record is created for each task. The relevant parts in the job record >>> (from >>> slurm.h) are: >>> uint32_t array_job_id; /* job_id of a job array or 0 if N/A */ >>> uint32_t array_task_id; /* task_id of a job array */ >>> uint32_t array_max_tasks; /* Maximum number of running tasks */ >>> char *array_task_str; /* string expression of task IDs in this >>> record */ >>> >>> Here's the code from src/api/job_info.c that demonstrates how to use that >>> information: >>> if (job_ptr->array_job_id) { >>> if (job_ptr->array_task_str) { >>> snprintf(tmp_line, sizeof(tmp_line), >>> "ArrayJobId=%u ArrayTaskId=%s ", >>> job_ptr->array_job_id, >>> job_ptr->array_task_str); >>> } else { >>> snprintf(tmp_line, sizeof(tmp_line), >>> "ArrayJobId=%u ArrayTaskId=%u ", >>> job_ptr->array_job_id, >>> job_ptr->array_task_id); >>> } >>> xstrcat(out, tmp_line); >>> >>> } >>> >>> >>> Quoting E V <[email protected]>: >>>> >>>> >>>> Trying drmaa with slurm > 14.03 using job arrays for the first time. >>>> Doesn't work. >>>> The drmaa code in session.c:slurmdrmaa_session_run_bulk() successfully >>>> submits my 2-job array(via slurm_submit_batch_job().) However, it >>>> calls slurm_load_job(&job_info,...) and expects job_info->record_count >>>> to be == the number of jobs in the array in order to iterate and >>>> create jobs for all the jobs in the array (via slurmdrmaa_job_new().) >>>> I guess this behaviour has changed in 14.11+, because record_count is >>>> now == 1. I see this new job_array_resp_msg_t type, but I don't see a >>>> corresponding load_job function in slurm.h using it. How does the >>>> handling of job arrays work now in slurm >14.03? >>> >>> >>> >>> >>> -- >>> Morris "Moe" Jette >>> CTO, SchedMD LLC >>> Commercial Slurm Development and Support >>> =============================================================== >>> Slurm User Group Meeting, 15-16 September 2015, Washington D.C. >>> http://slurm.schedmd.com/slurm_ug_agenda.html > > > > -- > Morris "Moe" Jette > CTO, SchedMD LLC > Commercial Slurm Development and Support > =============================================================== > Slurm User Group Meeting, 15-16 September 2015, Washington D.C. > http://slurm.schedmd.com/slurm_ug_agenda.html
