Hi, I submitted a patch that makes --ntasks-per-socket available also for CR_Core. This patch is not in slurm-2.6.x but has been merged on the development branch (e.g., it is in the slurm-13.12.0-0pre4 development release). My patch (which should apply against slurm-2.6.x can be found here):
https://groups.google.com/forum/#!searchin/slurm-devel/armin/slurm-devel/HrMMiJKzmig/w3ukKQXmrx4J Regards, Armin On 12/17/2013 05:56 AM, Christopher Samuel wrote: > > Hi folks, > > We cater for a mix of single CPU jobs and large MPI jobs and I've just > been asked how can someone launch a single MPI rank per *socket* for a > hybrid MPI+SMP code. > > Now we run with: > > SelectType=select/cons_res > SelectTypeParameters=CR_Core,CR_ONE_TASK_PER_CORE > > because we have to deal with a lot of single CPU jobs. > > However, the --ntasks-per-socket option is documented as only working > with CR_Socket, which says that it will not permit jobs to share sockets. > > Is there a way to get the socket as a consumable resource but only to > have it used when requested, i.e. default to the per-core settings we > have? > > At the moment I've told the user that they can request a whole node > and use Open-MPI's 1.6.x binding options thus: > > [samuel@barcoo ~]$ mpirun --num-sockets 2 --bind-to-socket > --npersocket 1 --display-map numactl --show | fgrep physcpubind > physcpubind: 0 1 2 3 4 5 6 7 > physcpubind: 0 1 2 3 4 5 6 7 > physcpubind: 8 9 10 11 12 13 14 15 > physcpubind: 8 9 10 11 12 13 14 15 > > OMPI 1.7 drops the need to specify how many sockets there are per node > with --num-sockets as it'll discover that itself, but getting there > from 1.6 to 1.7 is another story. :-) > > All the best, > Chris >
