Ok, I've investigated further today, it seems "--map-by hwthread" does not remove the problem. However, if I specified in the hostfile "node0 slots=32" it runs really slower than specifying only "node0". In both cases I run mpirun with -np 32. So I'm quite sure I didn't understand what slots are.
__ Federico Reghenzani M.Eng. Student @ Politecnico di Milano Computer Science and Engineering 2016-03-22 18:56 GMT+01:00 Federico Reghenzani < federico1.reghenz...@mail.polimi.it>: > Hi guys, > > I'm really confused about *slots* in resource allocation: I thought that > slots are the number of processes spawnable in a certain node, so it should > correspond to the number of Processing Elements of the node. For example, > on each of my nodes I have 2 processors, total 16 cores with > hyperthreading, so a total of 32 processing elements per node (i.e. 32 hw > threads). However, considering a single node, passing in the hostfile 32 > slots and requesting "-np 32" results is a performance degradation of 20x > slower than using only "-np 16". The problem disappears specifing --map-by > hwthread. > > Investigating on the problem I found these counterintuitive things: > - here > <https://www.open-mpi.org/faq/?category=running#slots-without-hostfiles> is > stated "*slots* are Open MPI's representation of how many processors are > available" > - here <https://www.open-mpi.org/doc/v1.10/man1/mpirun.1.php#sect6> is > stated "Slots indicate how many processes can potentially execute on a > node. For best performance, the number of slots may be chosen to be the > number of cores on the node or the number of processor sockets" > - I tried to remove the slots information from the hostfile, so according > to this > <https://www.open-mpi.org/faq/?category=running#slots-without-hostfiles> > should be interpreted as "1", but it spawns anyway 32 processes > - I'm not sure what --map-by and --rank-by do > > In custom RAS we are developing, what we have to send to mpirun? The > number of processor sockets, the number of cores or the number of hwthread > available? How --map-by and --rank-by affect the spawn policy? > > > Thank you! > > > OFFTOPIC: is someone going to EuroMPI 2016 in September? We will be there > to present our migration technique. > > > Cheers, > Federico > > __ > Federico Reghenzani > M.Eng. Student @ Politecnico di Milano > Computer Science and Engineering > > >
