[slurm-dev] Re: DRMAA job submission mis-calculates NumCPUs

[Moe Jette]

It works for me with versiion 2.6.2, but the results might vary  
depending upon your configuration.

$ sbatch -n1 --cpus-per-task=2 tmp
Submitted batch job 3

$ scontrol show job
JobId=3 Name=tmp
   UserId=jette(1001) GroupId=jette(1001)
   Priority=4294901759 Account=root QOS=normal
   JobState=PENDING Reason=Resources Dependency=(null)
   NumNodes=1 NumCPUs=2 CPUs/Task=2 ReqS:C:T=*:*:*

$ scontrol show job
JobId=3 Name=tmp
   UserId=jette(1001) GroupId=jette(1001)
   Priority=4294901759 Account=root QOS=normal
   JobState=RUNNING Reason=None Dependency=(null)
   NumNodes=1 NumCPUs=2 CPUs/Task=2 ReqS:C:T=*:*:*

The relevant code is in src/slurmctld/job_mgr.c around line 6249:

                        if (IS_JOB_COMPLETING(job_ptr) && job_ptr->cpu_cnt) {
                                pack32(job_ptr->cpu_cnt, buffer);
                                pack32((uint32_t) 0, buffer);
                        } else if (job_ptr->total_cpus) {
                                pack32(job_ptr->total_cpus, buffer);
                                pack32((uint32_t) 0, buffer);
                        } else {
                                pack32(detail_ptr->min_cpus, buffer);
                                if (detail_ptr->max_cpus != NO_VAL)
                                        pack32(detail_ptr->max_cpus, buffer);
                                else
                                        pack32((uint32_t) 0, buffer);

                        }

Quoting Michael Gutteridge <michael.gutteri...@gmail.com>:


Hi

Got a bit of a hairy problem.  Slurm 2.6.2, MWM 7.2.6 doing the
scheduling with the wiki2 plugin.  We have tools using DRMAA to submit
jobs: python-drmaa->slurm-drmaa->slurm

I recently added '--cpus-per-task' to the native specifications
supported by slurm-drmaa. I apparently have more work to do as jobs
aren't properly created- the job doesn't have NumCPUs set properly:

   NumNodes=1 NumCPUs=1 CPUs/Task=2 ReqS:C:T=*:*:*
   MinCPUsNode=2 MinMemoryNode=0 MinTmpDiskNode=0

This is giving Moab a headache- it reports 0 tasks for the job and
refuses to run it.

So anyway- I don't know if it's a slurm or slurm-drmaa issue- I'd like
to fix this, but I'm unsure of the proper place to implement the
repair.  Where is the NumCPUs calculation done? Is there something in
the API I should use or is the API supposed to be doing that
calculation?

Any hints appreciated.

Thanks

Michael