On 07/08/2013 11:34 PM, Teddy - wrote: > Dear all, > > > I am not quite understand on how to assign slurm task by actual physical > core. > > Lets say i have 3 slurm nodes with different specs. > Node A : 16 physical AMD processor, no hyperthreading ==> 16 cpu threads > Node B : 8 pyhsical Intel processor, with hyperthreading ==> 16 cpu threads > Node C : 4 physical Intel processor, no hyprethreading ==> 4 cpu threads. > > If i am not mistaken, by assigning standard slurm task (srun). slurm > will see all the nodes with the CPU threads, right?
Yes, but don't think of the hardware threads as anything unique - they are part of each core. The Linux kernel maps all hardware threads to logical CPUs and that's where the number of "processors" is coming from. > > If yes, is there a way to run a slurm task and it will automatically use > actual physical cores ( > So node B will be recognized as 8 cpu node) without have to set the node > manually in the slurm.conf? > This is because only certain slurm tasks need to use actual cores, other > slurm tasks dont matter. This is tricky to answer in a simple form. If you specify only 8 tasks on the a system with 16 logical CPUs but only 8 physical cores (i.e. 2 threads per core), the kernel scheduler is still free to schedule those 8 tasks wherever it wants. This means you can still have 8 tasks running on only 4 physical cores and it would be nothing like hyperthreading being turned off. Affinity is what enables you to specify the hardware thread (a logical CPU) on which a task is allowed to be scheduled. However, each MPI implementation handles affinity in a slightly different way. (OpenMPI being the most obnoxious because it wants to do everything itself.) To get back to your question, you can specify -n and -N and Slurm with balance tasks across the nodes. However, affinity usually isn't assigned automatically (IIRC, mvapich2 will do a bit). If you want to assign affinity using srun, take a look at the cpu_bind option. In the case of OpenMPI, you'll need to use its affinity options (useful options added ~1.4). You can check out the OpenMPI FAQ for more info: http://www.open-mpi.org/faq/?category=tuning#maffinity-defs > > > > Regards, > > Teddy > > > -- Daniel M. Weeks Systems Programmer Computational Center for Nanotechnology Innovations Rensselaer Polytechnic Institute Troy, NY 12180 518-276-4458
