You might find this useful: http://www.schedmd.com/slurmdocs/mc_support.html
Quoting "Daniel M. Weeks" <[email protected]>: > > On 07/08/2013 11:34 PM, Teddy - wrote: >> Dear all, >> >> >> I am not quite understand on how to assign slurm task by actual physical >> core. >> >> Lets say i have 3 slurm nodes with different specs. >> Node A : 16 physical AMD processor, no hyperthreading ==> 16 cpu threads >> Node B : 8 pyhsical Intel processor, with hyperthreading ==> 16 cpu threads >> Node C : 4 physical Intel processor, no hyprethreading ==> 4 cpu threads. >> >> If i am not mistaken, by assigning standard slurm task (srun). slurm >> will see all the nodes with the CPU threads, right? > > Yes, but don't think of the hardware threads as anything unique - they > are part of each core. The Linux kernel maps all hardware threads to > logical CPUs and that's where the number of "processors" is coming from. > >> >> If yes, is there a way to run a slurm task and it will automatically use >> actual physical cores ( >> So node B will be recognized as 8 cpu node) without have to set the node >> manually in the slurm.conf? >> This is because only certain slurm tasks need to use actual cores, other >> slurm tasks dont matter. > > This is tricky to answer in a simple form. > > If you specify only 8 tasks on the a system with 16 logical CPUs but > only 8 physical cores (i.e. 2 threads per core), the kernel scheduler is > still free to schedule those 8 tasks wherever it wants. This means you > can still have 8 tasks running on only 4 physical cores and it would be > nothing like hyperthreading being turned off. > > Affinity is what enables you to specify the hardware thread (a logical > CPU) on which a task is allowed to be scheduled. However, each MPI > implementation handles affinity in a slightly different way. (OpenMPI > being the most obnoxious because it wants to do everything itself.) > > To get back to your question, you can specify -n and -N and Slurm with > balance tasks across the nodes. However, affinity usually isn't assigned > automatically (IIRC, mvapich2 will do a bit). > > If you want to assign affinity using srun, take a look at the cpu_bind > option. In the case of OpenMPI, you'll need to use its affinity options > (useful options added ~1.4). You can check out the OpenMPI FAQ for more > info: http://www.open-mpi.org/faq/?category=tuning#maffinity-defs > >> >> >> >> Regards, >> >> Teddy >> >> >> > > -- > Daniel M. Weeks > Systems Programmer > Computational Center for Nanotechnology Innovations > Rensselaer Polytechnic Institute > Troy, NY 12180 > 518-276-4458 >
