Magnus, I'm not sure exactly what pattern of allocation and binding you're trying to achieve. If you can describe clearly the layout you want I might be able to suggest the combination of options you need to use. If you want each task to be confined to a single socket, you should be able to do that using the command line option --distribution=xxxx:block (where xxxx is either block or cyclic), but only if you're allocating the whole node to your job. I don't think the _task_layout_lllp_multi() function is relevant. Martin
From: Magnus Jonsson <[email protected]> To: "slurm-dev" <[email protected]>, Date: 02/18/2013 06:38 AM Subject: [slurm-dev] Re: task/affinity, --cpu_bind=socket and -c > 1 Hi! For us this is in most cases a bad default behaviour and I have not found a way to set a default value either (other then changing the code and recompile). One other thing that I'm still curious about is then will the default:-statement of the switch/case in my first mail (see below) happen? task_dist seems to be initialized to SLURM_DIST_CYCLIC (1) and all the other cases of task_dist that comes to this point are defined in the switch/case. I have not bin able to reach it and activate the _task_layout_lllp_multi() function. /Magnus On 2013-02-15 19:52, [email protected] wrote: > Assuming you're using the default allocation and distribution methods, > the behavior you describe sounds correct. Available cpus will be > selected cyclically across the sockets for allocation to the job. > Allocated cpus will be selected cyclically across the sockets for > distribution to tasks for binding. And each task will be bound to all > of the allocated cpus on each socket from which a cpu was distributed to > it. For -n 8 -c 6, I would expect each of the 8 tasks to be bound to 36 > cpus (6 cpus on each of 6 sockets). > > See the CPU Management Guide in the Slurm documentation for more info. > Examples 11 thru 13 illustrate socket binding. > > Martin Perry > Bull Phoenix > > > > From: Moe Jette <[email protected]> > To: "slurm-dev" <[email protected]>, > Date: 02/15/2013 10:33 AM > Subject: [slurm-dev] Re: task/affinity, --cpu_bind=socket and -c > 1 > ------------------------------------------------------------------------ > > > > > Have you tried the --ntasks-per-socket or --ntasks-per-core options? > > Quoting Magnus Jonsson <[email protected]>: > > > Hi! > > > > I have noticed strange behaviour in the task/affinity plugin if I > > use --cpu_bind=socket and -c > 1. > > > > My task are distributed one on each socket (I have 8) and if I say > > -c 6 six of my sockets are allocated to my first task. If I have 8 > > tasks each task get 6 of the 8 sockets. > > > > This sounds like a bad behaviour but is might be as design? > > > > I have traced it down to the lllp_distribution() function in > > task/affinity/dist_task.c > > > > In this switch statement: > > > > switch (req->task_dist) { > > case SLURM_DIST_BLOCK_BLOCK: > > case SLURM_DIST_CYCLIC_BLOCK: > > case SLURM_DIST_PLANE: > > /* tasks are distributed in blocks within a plane */ > > rc = _task_layout_lllp_block(req, node_id, &masks); > > break; > > case SLURM_DIST_CYCLIC: > > case SLURM_DIST_BLOCK: > > case SLURM_DIST_CYCLIC_CYCLIC: > > case SLURM_DIST_BLOCK_CYCLIC: > > rc = _task_layout_lllp_cyclic(req, node_id, &masks); > > break; > > default: > > if (req->cpus_per_task > 1) > > rc = _task_layout_lllp_multi(req, node_id, &masks); > > else > > rc = _task_layout_lllp_cyclic(req, node_id, &masks); > > req->task_dist = SLURM_DIST_BLOCK_CYCLIC; > > break; > > } > > > > in the default block there is a diffrent function called if > > cpus_per_task > 1. Should the cyclic block be the same as the > > default block? > > > > Or should SLURM_DIST_CYCLIC, SLURM_DIST_BLOCK be the same as default? > > > > Best regards, > > Magnus > > > > -- > > Magnus Jonsson, Developer, HPC2N, Umeå Universitet > > > > > > > -- Magnus Jonsson, Developer, HPC2N, Umeå Universitet
