I think that what you are seeing is that since the whole node is being allocated for the job, the 542 is the charge for all cpus on the node for length of time the job ran, even the ones left idle. The step usage is based on the cpu usage for the number of cpus actually used over the same length of time.
Phil Eckert LLNL From: Robert Stober <[email protected]<mailto:[email protected]>> Reply-To: slurm-dev <[email protected]<mailto:[email protected]>> Date: Monday, August 11, 2014 at 11:48 AM To: slurm-dev <[email protected]<mailto:[email protected]>> Subject: [slurm-dev] Re: How to interpret sacct output Hi David, One more question with regard to the sacct output: sacct -o jobid,ncpus,state,cputimeraw,maxrss -j 72 JobID NCPUS State CPUTimeRAW MaxRSS ------------ ---------- ---------- ---------- ---------- 72 2 COMPLETED 542 72.batch 1 COMPLETED 271 6708K 72.0 1 FAILED 122 4228K 72.1 1 FAILED 121 4228K I don't understand why the cputimeraw of the batch step doesn't equal the sum of the two job steps, which should be 243? Is this because the batch step cputimeraw also includes system time that was used to run the job but not used by the job itself? Also, how should memory be interpreted? Why is the memory consumption of the batch step higher than either of the two job steps? Thank you, Robert On 8/5/2014 1:41 PM, David Bigagli wrote: Yes that is correct. The first entry is the allocation which has 2 cpus, -n 2 was specified, the second entry is the batch step that run for 81 seconds, so the total cpu time used was 81 8*2. Form 'man sacct': CPUTime Formatted (Elapsed time * CPU) count used by a job or step. CPUTimeRaw Unlike above non formatted (Elapsed time * CPU) count for a job or step. Units are cpu-seconds On 08/05/2014 11:48 AM, Robert Stober wrote: Hi David, That's because I forgot to include my sacct command: sacct -ap -S 2014-01-01 -E now --format=jobid,user,partition,ncpus,cputimeraw,nodelist So in my example, the single job step (59.batch) ran on one core and took 81 seconds. The first entry (which I'm referring to as the "primary job") shows that he reserved (or requested) two cpus. I'm assuming that since the two cpus he requested were unavailable to anyone else, Slurm is just showing that he effectively used 162 cpu seconds (81 x 2). Thank you, Robert On 8/5/2014 9:53 AM, David Bigagli wrote: Hi Robert, the first line is the allocation and the second the batch step, the batch step runs on one cpu. I am not sure what are the columns with values 162 and 81. The elapsed time of the allocation should be the sum of elapsed time of all steps, so if the step ran for 81 seconds I would expect the elapsed time of the allocation to be 81. For exaple: david@prometeo ~/slurm/work>\sacct -o jobid,ncpus,state,elapsed -j 78475 JobID NCPUS State Elapsed ------------ ---------- ---------- ---------- 78475 2 COMPLETED 00:00:20 78475.batch 1 COMPLETED 00:00:20 78475.0 2 COMPLETED 00:00:10 78475.1 2 COMPLETED 00:00:10 On 08/05/2014 09:19 AM, Robert Stober wrote: Please consider this sacct output. My question is how should this output be understood? 59|usera|defq|2|162| 59.batch|||1|81| Here's how I'm reading it: 1. usera submitted jobid 59 and requested 2 CPUs. 2. He actually used only one CPU for 81 seconds. 3. As a result, 2 CPUs were reserved and unavailable to anyone during the 81 second run time, so 162 CPU seconds were effectively used (by virtue of both CPUs being unavailable during the entire job run time). Do I have an accurate understanding of this? If so, then it appears that if I want the total "chargeable" resource usage I should just look at the primary job record (59 in this case) and skip the records for each of the job steps since they're included in the primary. Would you concur? Thank you, Robert -- brightcomputing_logo.png Mr. Robert Stober Senior Systems Architect Mob: +1 209 986 9298 Skype: rstober [email protected]<mailto:[email protected]> <mailto:[email protected]><mailto:[email protected]> Bright Computing, Inc. 2880 Zanker Road Suite 203 San Jose, CA 95134 Tel: +1 408 300 9448 Fax: +1 408 715 0102 www.BrightComputing.com<http://www.BrightComputing.com><http://www.brightcomputing.com><http://www.brightcomputing.com> -- brightcomputing_logo.png Mr. Robert Stober Senior Systems Architect Mob: +1 209 986 9298 Skype: rstober [email protected]<mailto:[email protected]> <mailto:[email protected]><mailto:[email protected]> Bright Computing, Inc. 2880 Zanker Road Suite 203 San Jose, CA 95134 Tel: +1 408 300 9448 Fax: +1 408 715 0102 www.BrightComputing.com<http://www.BrightComputing.com><http://www.brightcomputing.com><http://www.brightcomputing.com> -- [brightcomputing_logo.png] Mr. Robert Stober Senior Systems Architect Mob: +1 209 986 9298 Skype: rstober [email protected]<mailto:[email protected]> Bright Computing, Inc. 2880 Zanker Road Suite 203 San Jose, CA 95134 Tel: +1 408 300 9448 Fax: +1 408 715 0102 www.BrightComputing.com<http://www.brightcomputing.com>
