Hi Aurélien, I have said the same thing to many users over the years. Our colleagues at NERSC, however, have found that 20% of their codes work better when using HT. Some codes benefit from SMT2 (i.e. HT) and even SMT4 (available on Power8) in order to provide enough latency hiding of memory accesses.
As with everything in computer science, the answer is “It depends”. Try with and without for each new generation of hardware. Scott > On Mar 23, 2016, at 4:32 PM, Aurélien Bouteiller <boute...@icl.utk.edu> wrote: > > To add to what Ralf said, you probably do not want to use Hyper Threads for > HPC workloads, as that generally results in very poor performance (as you > noticed). Set the number of slots to the number of real cores (not HT), that > would yield optimal results 95% of the time. > > Aurélien > > -- > Aurélien Bouteiller, Ph.D. ~~ https://icl.cs.utk.edu/~bouteill/ > >> Le 23 mars 2016 à 16:24, Ralph Castain <r...@open-mpi.org> a écrit : >> >> “Slots” are an abstraction commonly used by schedulers as a way of >> indicating how many processes are allowed to run on a given node. It has >> nothing to do with hardware, either cores or HTs. >> >> MPI programmers frequently like to bind a process to one or more hardware >> assets (cores or HTs). Thus, you will see confusion in the community where >> people mix the term “slot” with “cores” or “cpus”. This is unfortunate as it >> the terms really do mean very different things. >> >> In OMPI, we chose to try and “help” the user by not requiring them to >> specify detailed info in a hostfile. So if you don’t specify the number of >> “slots” for a given node, we will sense the number of cores on that node and >> set the slots to match that number. This best matches user expectations >> today. >> >> If you do specify the number of slots, then we use that to guide the desired >> number of processes assigned to each node. We then bind each of those >> processes according to the user-provided guidance. >> >> HTH >> Ralph >> >>> On Mar 23, 2016, at 9:35 AM, Federico Reghenzani >>> <federico1.reghenz...@mail.polimi.it> wrote: >>> >>> Ok, I've investigated further today, it seems "--map-by hwthread" does not >>> remove the problem. However, if I specified in the hostfile "node0 >>> slots=32" it runs really slower than specifying only "node0". In both cases >>> I run mpirun with -np 32. So I'm quite sure I didn't understand what slots >>> are. >>> >>> __ >>> Federico Reghenzani >>> M.Eng. Student @ Politecnico di Milano >>> Computer Science and Engineering >>> >>> >>> >>> 2016-03-22 18:56 GMT+01:00 Federico Reghenzani >>> <federico1.reghenz...@mail.polimi.it>: >>> Hi guys, >>> >>> I'm really confused about slots in resource allocation: I thought that >>> slots are the number of processes spawnable in a certain node, so it should >>> correspond to the number of Processing Elements of the node. For example, >>> on each of my nodes I have 2 processors, total 16 cores with >>> hyperthreading, so a total of 32 processing elements per node (i.e. 32 hw >>> threads). However, considering a single node, passing in the hostfile 32 >>> slots and requesting "-np 32" results is a performance degradation of 20x >>> slower than using only "-np 16". The problem disappears specifing --map-by >>> hwthread. >>> >>> Investigating on the problem I found these counterintuitive things: >>> - here is stated "slots are Open MPI's representation of how many >>> processors are available" >>> - here is stated "Slots indicate how many processes can potentially execute >>> on a node. For best performance, the number of slots may be chosen to be >>> the number of cores on the node or the number of processor sockets" >>> - I tried to remove the slots information from the hostfile, so according >>> to this should be interpreted as "1", but it spawns anyway 32 processes >>> - I'm not sure what --map-by and --rank-by do >>> >>> In custom RAS we are developing, what we have to send to mpirun? The number >>> of processor sockets, the number of cores or the number of hwthread >>> available? How --map-by and --rank-by affect the spawn policy? >>> >>> >>> Thank you! >>> >>> >>> OFFTOPIC: is someone going to EuroMPI 2016 in September? We will be there >>> to present our migration technique. >>> >>> >>> Cheers, >>> Federico >>> >>> __ >>> Federico Reghenzani >>> M.Eng. Student @ Politecnico di Milano >>> Computer Science and Engineering >>> >>> >>> >>> _______________________________________________ >>> devel mailing list >>> de...@open-mpi.org >>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel >>> Link to this post: >>> http://www.open-mpi.org/community/lists/devel/2016/03/18723.php >> >> _______________________________________________ >> devel mailing list >> de...@open-mpi.org >> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel >> Link to this post: >> http://www.open-mpi.org/community/lists/devel/2016/03/18724.php > > _______________________________________________ > devel mailing list > de...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel > Link to this post: > http://www.open-mpi.org/community/lists/devel/2016/03/18725.php
