I think we could use the first format, if we had corresponding functions to interact with the jobs using those id's, querying job info and holding and resuming jobs and such. It would be nice if these functions worked with either of the 2 job formats you mentioned so the drmaa code doesn't always need to fork on the type of job when interacting with jobs. I know it already naturally has two code paths on the job submission between arrays and regular jobs, but I don't think most of the other API calls do. I'm not an expert on the drmaa code base though so might be more gotchas lurking, however it certainly sounds less daunting to me then redesigning the drmaa api's ;-)
On Wed, Sep 2, 2015 at 3:32 PM, Moe Jette <[email protected]> wrote: > > The Slurm APIs recognize two different formats for job array IDs. A job > array with two tasks might be known by the names "123_1" and "123_2" OR > "123" and "124". The tasks of the job array are known by the first format at > submit time. The second format will not be known until allocation time. > Could you use the first format for job IDs? > > > Quoting E V <[email protected]>: >> >> Hmm, that's a bit of an issue as that behaviour appears to break the >> drmaa_run_bulk_jobs() API and the drmaa slurm bindings in c, perl, >> python, ruby and java as far as I know. I guess my hope for a >> straightforward upgrade beyond 14.03 isn't happening any time soon. >> >> On Wed, Sep 2, 2015 at 1:02 PM, Moe Jette <[email protected]> wrote: >>> >>> >>> Quoting E V <[email protected]>: >>>> >>>> >>>> Huh, so we don't know the jobs id's of the individual jobs at the >>>> array submit time anymore? >>> >>> >>> >>> No. We have some users submitting job arrays with millions of tasks and >>> do >>> minimal processing at job submit time for performance reasons. The job >>> IDs >>> for individual tasks are selected at resource allocation time. >>> >>>> I think the drmaa wrapper I use assumes we >>>> get back a list of job id's from the array submission. Any way to >>>> query this information? >>> >>> >>> >>> Generally not before resource allocation time. >>> >>> >>>> On Tue, Sep 1, 2015 at 5:56 PM, Moe Jette <[email protected]> wrote: >>>>> >>>>> >>>>> >>>>> For scalability reasons, when a job array is submitted only one job >>>>> record >>>>> is created. As individual tasks in the job array start running, a new >>>>> job >>>>> record is created for each task. The relevant parts in the job record >>>>> (from >>>>> slurm.h) are: >>>>> uint32_t array_job_id; /* job_id of a job array or 0 if N/A */ >>>>> uint32_t array_task_id; /* task_id of a job array */ >>>>> uint32_t array_max_tasks; /* Maximum number of running tasks */ >>>>> char *array_task_str; /* string expression of task IDs in >>>>> this >>>>> record */ >>>>> >>>>> Here's the code from src/api/job_info.c that demonstrates how to use >>>>> that >>>>> information: >>>>> if (job_ptr->array_job_id) { >>>>> if (job_ptr->array_task_str) { >>>>> snprintf(tmp_line, sizeof(tmp_line), >>>>> "ArrayJobId=%u ArrayTaskId=%s ", >>>>> job_ptr->array_job_id, >>>>> job_ptr->array_task_str); >>>>> } else { >>>>> snprintf(tmp_line, sizeof(tmp_line), >>>>> "ArrayJobId=%u ArrayTaskId=%u ", >>>>> job_ptr->array_job_id, >>>>> job_ptr->array_task_id); >>>>> } >>>>> xstrcat(out, tmp_line); >>>>> >>>>> } >>>>> >>>>> >>>>> Quoting E V <[email protected]>: >>>>>> >>>>>> >>>>>> >>>>>> Trying drmaa with slurm > 14.03 using job arrays for the first time. >>>>>> Doesn't work. >>>>>> The drmaa code in session.c:slurmdrmaa_session_run_bulk() successfully >>>>>> submits my 2-job array(via slurm_submit_batch_job().) However, it >>>>>> calls slurm_load_job(&job_info,...) and expects job_info->record_count >>>>>> to be == the number of jobs in the array in order to iterate and >>>>>> create jobs for all the jobs in the array (via slurmdrmaa_job_new().) >>>>>> I guess this behaviour has changed in 14.11+, because record_count is >>>>>> now == 1. I see this new job_array_resp_msg_t type, but I don't see a >>>>>> corresponding load_job function in slurm.h using it. How does the >>>>>> handling of job arrays work now in slurm >14.03? >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Morris "Moe" Jette >>>>> CTO, SchedMD LLC >>>>> Commercial Slurm Development and Support >>>>> =============================================================== >>>>> Slurm User Group Meeting, 15-16 September 2015, Washington D.C. >>>>> http://slurm.schedmd.com/slurm_ug_agenda.html >>> >>> >>> >>> >>> -- >>> Morris "Moe" Jette >>> CTO, SchedMD LLC >>> Commercial Slurm Development and Support >>> =============================================================== >>> Slurm User Group Meeting, 15-16 September 2015, Washington D.C. >>> http://slurm.schedmd.com/slurm_ug_agenda.html > > > > -- > Morris "Moe" Jette > CTO, SchedMD LLC > Commercial Slurm Development and Support > =============================================================== > Slurm User Group Meeting, 15-16 September 2015, Washington D.C. > http://slurm.schedmd.com/slurm_ug_agenda.html
