Sabuj Pattanayek wrote: > I've unset all the resources_*.nodes settings for the queue
Rightfully since min and max on these does not make any sense to PBS as they are considered as text resources. > and gotten > maui/torque to actually read the ncpus= value from the PBS script file > since this is actually what sets the PROC limit. The ncpus limits is useful only for single-node jobs. If you ask -lncpus=32 while your nodes only have 16 cores, your jobs will be rejected for sure. > According to the > source src/moab/MJob.c line 11483, ncpus is tied to the PROC limit. > Also the comment in src/moab/MPBSI.c on line 1676 validates this. > However, for some reason ncpus was not being read previously from the > pbs script file even when all references to resources_*.ncpus were > unset in qmgr. If Maui does not set the PROC limit without -lncpus, I consider that this is a bug. The PROC column in the output of showq is ok, though, with -lnodes=1:ppn=16. But before giving up, make sure that you have "JOBNODEMATCHPOLICY EXACTNODE" in maui.cfg. > Now that I know how to get the PROC resource limit to change, I'll > play around with the qmgr settings again to try to set limits before a > user even submits a job. Then I'll test to see if these limits are > enforced for MPI programs (across nodes). I think that you will get nowhere with ncpus on a cluster, unless you use these limits only for single-node jobs. -- Michel Béland, analyste en calcul scientifique [email protected] bureau S-250, pavillon Roger-Gaudry (principal), Université de Montréal téléphone : 514 343-6111 poste 3892 télécopieur : 514 343-2155 RQCHP (Réseau québécois de calcul de haute performance) www.rqchp.qc.ca _______________________________________________ mauiusers mailing list [email protected] http://www.supercluster.org/mailman/listinfo/mauiusers
