My suggestion would be evaluate your job's need and use the parameters that match your job's needed resources. Carefully read docs too. I've found that even skimming the docs when I'm looking at a particular parameter is not enough. Things like the default behaviors and what other parameters change the behavior are also not always directly under a parameter's description. A browser's "Find text" (CTRL+F on my desktop) can be very handy when you want to see all the ways the behavior of a particular parameter can change.
It's hard to describe the difference between the --ntasks and --ntasks-per-node options without quoting the entire description in the docs. When to use one vs another depends on your configuration (ie partitions, qos, SelectTypeParameters, etc) as well as what your job needs. I typically am very explicit in my sbatch allocation requests and will use --nodes, --ntasks-per-node and --cpus-per-task. - Trey ============================= Trey Dockendorf Systems Analyst I Texas A&M University Academy for Advanced Telecommunications and Learning Technologies Phone: (979)458-2396 Email: [email protected] Jabber: [email protected] ----- Original Message ----- > From: "Eva Hocks" <[email protected]> > To: "slurm-dev" <[email protected]> > Sent: Tuesday, September 23, 2014 7:23:22 PM > Subject: [slurm-dev] Re: Requested node configuration is not available > > > > > > Yes, yo are right, I do not understand what --ntasks means and > when to use it or --ntasks-per-node > > I assumed it meant " n tasks " for the job, in my case, 2 tasks for > this job, running on 1 node with 1 cpu per task, which would make it > 1 > nodes/2 cpu allocation. > > > As for the "default is one task per node" : when using --nodes=1 I > get > an allocation of 1 node/ 8 cpus : > > NumNodes=1 NumCPUs=8 CPUs/Task=1 > > so I get a default of 1 task per cpu, not one task per node. > > Thanks for your help, > Eva > > > On Tue, 23 Sep 2014, Trey Dockendorf wrote: > > > > > Eva, > > > > I think your misunderstanding the purpose and default behavior of > > "--ntasks". > > > > If you read the last sentence of the description for --ntasks - > > "The default is one task per node". > > > > So from your first email, using "--nodes=1-1 --ntasks=2 > > --cpus-per-task=1" will not work. Your asking for 2 tasks and a > > max of one node, and the default for number of tasks is one per > > node (see quote above). If you just said "--nodes=1" then that > > may work since your max was undefined and as the docs mention for > > --nodes, the job will be allocated enough nodes to satisfy your > > number of tasks. > > > > Those options work just fine, and as expected, in 14.03.3 up to > > 14.03.6 (versions I've used in past months). > > > > - Trey > > > > ============================= > > > > Trey Dockendorf > > Systems Analyst I > > Texas A&M University > > Academy for Advanced Telecommunications and Learning Technologies > > Phone: (979)458-2396 > > Email: [email protected] > > Jabber: [email protected] > > > > ----- Original Message ----- > > > From: "Eva Hocks" <[email protected]> > > > To: "slurm-dev" <[email protected]> > > > Sent: Tuesday, September 23, 2014 5:59:28 PM > > > Subject: [slurm-dev] Re: Requested node configuration is not > > > available > > > > > > > > > > > > > > > > > > > > > > > > I found > > > > > > > > > #SBATCH --ntasks-per-node=2 > > > > > > and now the job runs with NumNodes=1-1 NumCPUs=2 CPUs/Task=2 > > > > > > > > > The docu on www.schedmd.com/slurmdocs/sbatch.html > > > > > > shows > > > > > > -n, --ntasks ...... > > > > > > > > > Is this flag still working for job steps to run within the > > > allocation > > > will launch a maximum of "number" tasks ? Doesn't look like it. > > > > > > Thanks > > > Eva > > > > > > > > > On Tue, 23 Sep 2014, Eva Hocks wrote: > > > > > > > > > > > > > > > > > > > > > > > If I use > > > > > > > > #SBATCH -N 1-1 > > > > #SBATCH -c 2 the job starts and uses > > > > > > > > NumNodes=1 NumCPUs=8 CPUs/Task=2 > > > > > > > > > > > > Eva > > > > > > > > > > > > > > > > On Tue, 23 Sep 2014, Eva Hocks wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > slurm Version=14.03: > > > > > > > > > > I am trying to run a simple job with > > > > > > > > > > #SBATCH --nodes=1-1 > > > > > #SBATCH --ntasks=2 > > > > > #SBATCH --cpus-per-task=1 > > > > > > > > > > on a test cluster with 2 nodes both configured: > > > > > CPUAlloc=0 CPUErr=0 CPUTot=8 > > > > > > > > > > but whenever I try sbatch it refuses: Requested node > > > > > configuration is > > > > > not available. The nodes have 8 CPUs configured so why can't > > > > > I > > > > > run a 2 > > > > > task job on 2 of those CPUs? I can run a 1 task job or if I > > > > > don't > > > > > include the ntasks I get by default 8 tasks, that works. But > > > > > I > > > > > cannot > > > > > specify anything >1 task. sbatch --verbose isn't very > > > > > helpful. > > > > > The slurm > > > > > log has: > > > > > > > > > > > > > > > [2014-09-23T14:58:03.834] cons_res: cr_job_test: evaluating > > > > > job > > > > > 152 on 3 nodes > > > > > [2014-09-23T14:58:03.834] cons_res: _can_job_run_on_node: 8 > > > > > cpus > > > > > on hpc-0-4(1), mem 0/24151 > > > > > [2014-09-23T14:58:03.834] cons_res: _can_job_run_on_node: 8 > > > > > cpus > > > > > on hpc-0-5(0), mem 0/24151 > > > > > [2014-09-23T14:58:03.834] cons_res: _can_job_run_on_node: 8 > > > > > cpus > > > > > on hpc-0-6(1), mem 0/24151 > > > > > [2014-09-23T14:58:03.834] cons_res: eval_nodes:0 consec c=24 > > > > > n=3 > > > > > b=0 e=2 r=-1 > > > > > [2014-09-23T14:58:03.834] cons_res: cr_job_test: test 0 fail: > > > > > insufficient resources > > > > > [2014-09-23T14:58:03.834] no job_resources info for job 152 > > > > > rc=-1 > > > > > [2014-09-23T14:58:03.834] _pick_best_nodes: job 152 never > > > > > runnable > > > > > [2014-09-23T14:58:03.835] _slurm_rpc_submit_batch_job: > > > > > Requested > > > > > node configuration is not available > > > > > > > > > > > > > > > > > > > > and scontrol shows the job with > > > > > JobState=FAILED Reason=BadConstraints > > > > > NumNodes=1 NumCPUs=2 CPUs/Task=1 > > > > > > > > > > > > > > > BTW mysql is running and I can with no problem connect to the > > > > > server > > > > > manually. > > > > > > > > > > > > > > > Any insight how to run a 1 node/2CPU job in slurm? > > > > > > > > > > > > > > > Thanks > > > > > Eva > > > > > > > > > > > > > > > > > > >
