On 3/10/2011 9:36 AM, Vicker, Darby (JSC-EG311) wrote:
Right, I understand how the hard and soft limits work. That's not the issue I'm having - the scheduler is respecting the hard and soft limits we have configured and is behaving as expected.If after doing so there still remain available resources, the scheduler would then select all jobs which meet the less constraining hard per user MAXJOB limit of 8 jobs. These jobs would then be scheduled and/or reserved as available resources allowed.The issue is that showbf doesn't appear to understand the statement above. That is, its not reporting the idle resources beyond the soft limits that should be available to a user.
soft limit is limiting the resource user see hard limit is extra that allow user to go beyond the soft limit it seems that work as expected your setting USERCFG[DEFAULT] MAXJOB=2,20 MAXPROC=512,1344 NODEACCESSPOLICY SINGLEUSER may be change to USERCFG[DEFAULT] MAXJOB=20 MAXPROC=1344 NODEACCESSPOLICY shared
On Mar 9, 2011, at 7:04 PM, Hung-Sheng Tsao (Lao Tsao 老曹) Ph. D. wrote:Soft limits help address this problem by providing additional scheduling flexibility. They allow sites to specify two tiers of limits, the more constraining limits, soft limits, are basically in effect in heavily loaded situations and reflect tight fairness constraints. The more flexible hard limits specify how flexible the scheduler can be in selecting jobs when there are idle resources available after all jobs meeting the tighter soft limits have been started. Soft and hard limits are specified in the format [<SOFTLIMIT>,]<HARDLIMIT>. For example, a given site may want to use the following configuration: ------- # maui.cfg USERCFG[DEFAULT] MAXJOB=2,8 ------- With this configuration, the scheduler would select all jobs which meet the per user MAXJOB limit of 2. It would then attempt to start and or reserve resources for all of these selected jobs. If after doing so there still remain available resources, the scheduler would then select all jobs which meet the less constraining hard per user MAXJOB limit of 8 jobs. These jobs would then be scheduled and/or reserved as available resources allowed. On 3/9/2011 6:08 PM, Vicker, Darby (JSC-EG311) wrote:Sorry - I didn't really give enough info in the last email. NODEACCESSPOLICY would only affect nodes that are partially used. In this case we currently have several nodes that are completely used (job-exclusive) and several idle nodes. [dvicker@service0 dvicker]% showbf -u lmarek backfill window (user: 'lmarek' group: 'eg3' partition: ALL) Wed Mar 9 16:50:32 no procs available [dvicker@service0 dvicker]% showbf -u dvicker backfill window (user: 'dvicker' group: 'eg3' partition: ALL) Wed Mar 9 16:51:03 384 procs available with no timelimit [dvicker@service0 dvicker]% pbsnodes -a | grep "state = free" | wc -l 40 [dvicker@service0 dvicker]% The showbf output for lmarek is inaccurate. If that user starts another job with less than 384 procs, it starts immediately. And the showbf output is accurate for dvicker - we have ppn=12 nodes in this system and 8 of them are reserved for the debug standing reservation so there are indeed 32*12 = 384 procs available. But you are onto something here. There are only two users with jobs in the queue right now. The showbf command is currently showing no procs available for those two users but 384 available for anyone else. Is it possible that the soft limit of 2 jobs per person (USERCFG[DEFAULT] MAXJOB=2,20 MAXPROC=512,1344) is causing showbf to report no procs available? I just did a test and that does indeed appear to be the problem. If a user has none or one job in the queue, showq reports the accurate number of free procs. Once the soft limit of 2 jobs is meet or exceeded for a given user, showbf reports no proc available for that user. But new jobs do start for that user if free resources are available. This seems like a bug in showbf to me. On Mar 9, 2011, at 4:35 PM, Hung-Sheng Tsao (Lao Tsao 老曹) Ph. D. wrote:this setting show that only single user can use the nodes On 3/9/2011 5:17 PM, Vicker, Darby (JSC-EG311) wrote:NODEACCESSPOLICY SINGLEUSER<laotsao.vcf>_______________________________________________ mauiusers mailing list [email protected] http://www.supercluster.org/mailman/listinfo/mauiusers<laotsao.vcf>
<<attachment: laotsao.vcf>>
_______________________________________________ mauiusers mailing list [email protected] http://www.supercluster.org/mailman/listinfo/mauiusers
