Reuti <re...@staff.uni-marburg.de> writes: > Do all of them have an internal bookkeeping of granted cores to slots > - i.e. not only the number of scheduled slots per job per node, but > also which core was granted to which job? Does Open MPI read this > information would be the next question then.
OMPI works with the bindings it's handed via orted (if the processes are started that way). >> My understanding is that Torque delegates to OpenMPI the process placement >> and binding (beyond the list of nodes/cpus available for >> the job). Can't/doesn't torque start the MPI processes itself? Otherwise, yes, since orted gets the binding. >> My guess is that OpenPBS behaves the same as Torque. >> >> SLURM and SGE/OGE *probably* have pretty much the same behavior. > > SGE/OGE: no, any binding request is only a soft request. I don't understand that. Does it mean the system-specific "strict" and "non-strict" binding in hwloc, in which case I don't see how UGE can do anything different? > UGE: here you can request a hard binding. But I have no clue whether this > information is read by Open MPI too. > > If in doubt: use only complete nodes for each job (which is often done > for massively parallel jobs anyway). There's no need with a recent SGE. All our jobs get core bindings -- unless they use all the cores, since binding them all is equivalent to binding none -- and OMPI inherits them. See <http://arc.liv.ac.uk/SGE/howto/sge-configs.html#_core_binding> for the SGE+OMPI configuration. -- Community Grid Engine: http://arc.liv.ac.uk/SGE/
