On Tue, 26 Mar 2013 18:07:39 +0100 Daniel Gruber wrote: [...]
> >> 2) core binding. I have it configured for serial and smp jobs, but > >> which is the correct strategy and configuration for mpi jobs? > >> Is linear going to span jobs acros different host sockets? > > > > AFAICS the request is applied on all machines which you get granted > > for the job. I.e. applied per `qrsh -inherit ...` besides setting > > it for the jobscript already. This is hard to handle in case of a > > round robin allocation, as you don't know in advance whether you > > get just one slot per machine or more. Maybe the best would be to > > use it with a fixed allocation rule only. > > Yes, linear spans across sockets, while it tries to allocate cores on > one socket first. Basically it chooses the socket with most free > cores and fills it up, then it chooses the second socket, and so on. > Something like "packing" jobs close to shared cache levels. In Univa > Grid Engine it is not a per qrsh -inherit call anymore (as it is like > for SGE 6.2u5), it is now a per host request because core management > was moved in 8.1.0 from execd level into the scheduler itself. The > scheduler has a global view on used resources. > > When requesting linear with JSV you need to request > "linear_automatic" since "linear" equals to something like "qsub > -binding linear:2:0,0" while "linear_automatic" equals to the more > common "qsub -binding linear:2". > > If you are using OpenMPI you can also generate a rankfile out of the > PE hostfile and delegate the core selection to OpenMPI. But in SGE > you have the same core selection for each host hence the jobs must > run host exclusively, which is no real advantage. In Univa Grid > Engine you don't have this limitation anymore, again because the > scheduler selects cores with having a global view. > > Maybe this is interesting for you: > http://www.gridengine.eu/grid-engine-internals/119-boosting-openmpi-performance-with-rankfiles-core-binding-and-univa-grid-engine on a first read, looks like what I'm looking for, but I should migate to Univa... something not possible now. Thanks for the link, really interesting. BTW, rss of your blog? > Daniel Arnau _______________________________________________ users mailing list [email protected] https://gridengine.org/mailman/listinfo/users
