Am 13.06.2014 um 18:02 schrieb [email protected]: > > We're running SoGE 8.1.6, and I wanted to understand how SoGE manages > CPU resources for jobs that are both multi-threaded and MPI-parallel. > > We have slots configured as a consumable resource, with the number of > slots per-node equal to the number of CPU-cores. > > We use OpenMPI with tight SGE integration. > > We use a core binding strategy of "linear_automatic", set in a JSV, > to allocate the requested number of cores for each job. > > We will have a job that has an initial MPI phase and later in the > same job a multi-threaded phase. Each parallel process in MPI phase is > single-threaded. > > If the job requests 10 slots of each type (and we have individual nodes > with more than 10 cores), submitted like: > > qsub -pe threaded 10 -pe openmpi 10 myjob
Did you try this? It should throw an error. Even if this would work: this is not an issue of SGE on its own, but how should Open MPI get and interpret this information? You need to submit with 100 slots (maybe in a "fixed 10" allocation) and alter the $PE_HOSTFILE and point the variable to the altered one, maybe adjusting the number of slots too so that Open MPI will start only 10 instances on 10 hosts by `qrsh -inherit ...` as these will start the threads then. -- Reuti > is SoGE 'smart' enough to do the following: > > [when resources are available] launch the job on a compute > node, 'consuming' 10 slots from the available count on > that node > > execute the 10 OpenMPI threads on the same compute node, using > the cores allocated by the core-binding > > when the MPI portion of the job is complete, be aware that > the cores used by the MPI code are available and run the > multithreaded portion on the same cores > > Is any specific configuration required to get that behavior? > > Thanks, > > Mark > > _______________________________________________ > users mailing list > [email protected] > https://gridengine.org/mailman/listinfo/users _______________________________________________ users mailing list [email protected] https://gridengine.org/mailman/listinfo/users
