Also, I agree that overriding limits manually can often be the most graceful way to tune a queue temporarily; in this case it is important to realize that even though 'qrun' can be very, very bad for Maui (since it ignores scheduler policies), 'runjob' can be very, very good (since it only ignores the groups of policies you tell it to ignore).
That way you can schedule manually with enhanced control (for example, you can manually specify nodes which cannot run other jobs for some reason) and automatic scheduling will resume when the user falls back below their limit. When I was a queue admin, I liked to manually schedule some N jobs over a user's limit at the end of the day and then temporarily increase their limit by N/2 so that they would get a big boost immediately and then a small boost overnight, which would decrease the probability of excess usage by the next morning. I'm almost certain this overall strategy can be helpful in Moab as well, though I don't have access to it. Phil On Wed, Jun 25, 2014 at 5:19 PM, Macbeth R. <[email protected]> wrote: > Lot much better thanks a lot! :D > I will just wrap that into some kind of bash switch, so it always switch > to the OTHER setting when run, in case I want to use it manually sometimes. > > Anyway to read that setting for that purpose? Something like: getparam > does exist? > > > On Tue, Jun 24, 2014 at 9:14 PM, Phil Regier <[email protected]> > wrote: > >> Actually, if you are OK using cron, then the cleaner solution is to use >> changeparam: >> >> changeparam USERCFG[render] MAXPROC=2848 >> >> from the command line to alter a running instance *without* a (i.e., >> *until the next*) service restart. If Maui does get restarted, the render >> user's MAXPROC will go back to the default (so the default should probably >> be 1024 in this case). >> >> Do you like that better? >> >> Phil >> >> >> On Tue, Jun 24, 2014 at 4:28 PM, Macbeth R. <[email protected]> >> wrote: >> >>> Hey Thanks a lot for all the Info!! :D >>> >>> Well I' m on a tight schedule (render feature film), so I cant afford >>> testing with your first approach (also you don't seem to belive it will >>> work, :P) >>> >>> So for the second approach if I have no choice but to use a cronjob I >>> will test with a bash script with sed, something more straightforward.. >>> just to comment or uncomment the limiting line in maui.cfg and restarting >>> maui, like: >>> >>> #!/bin/sh >>> STRINGSILLO="#USERCFG[render] MAXPROC=2848" >>> if grep -Fxq "$STRINGSILLO" /usr/local/maui-3.3.1/maui.cfg >>> then >>> sed -i 's/#USERCFG\[render\]/USERCFG\[render\]/g' >>> /usr/local/maui-3.3.1/maui.cfg >>> else >>> sed -i 's/USERCFG\[render\]/#USERCFG\[render\]/g' >>> /usr/local/maui-3.3.1/maui.cfg >>> fi >>> >>> sleep 5 >>> /etc/init.d/maui.d restart >>> >>> Cause in my case I cant use QOS for that purpose because I am already >>> using it as if it was just priority, just to get wich one starts first, so >>> it will be a mess if I create other QOS to manage number of cores also (and >>> I still have to use a cronjob to change of QOS on weekends) >>> >>> Also MAUI acts weird when changing QOS of already running jobs (I only >>> use JOB ARRAYS). >>> >>> Anyway if there is something more clean, please share. >>> >>> I will keep the cronjob/bash/sed approach by now. >>> >>> THANKS A LOT FOR THE HELP!!! >>> >>> >>> On Tue, Jun 24, 2014 at 1:36 PM, Phil Regier <[email protected]> >>> wrote: >>> >>>> I believe this would be done via standing reservations, though the >>>> policy may be too complex for Maui to handle properly. I don't presently >>>> have access to any head nodes, but in theory it should be something like >>>> >>>> SRCFG[renderweekday] ACCESS=SHARED USERLIST=render RESOURCES=PROCS:1024 >>>> DAYS=Mon,Tue,Wed,Thu,Fri >>>> SRCFG[renderweekend] ACCESS=SHARED USERLIST=render RESOURCES=PROCS:2848 >>>> DAYS=Sat,Sun >>>> SRCFG[everyoneelse] ACCESS=SHARED USERLIST=!render >>>> >>>> I don't think this will work, however, in part because ACCESS=SHARED is >>>> questionable on the best of days, and in part because the negative ACLs >>>> (which were not officially supported, apparently) can do strange and >>>> unexpected things in Maui sometimes. >>>> >>>> If you do try the above, be sure to use 'mdiag' to see which of your >>>> settings take effect, and submit some test jobs to make sure you are not >>>> unexpectedly restricting access which should be allowed. >>>> >>>> You can also try ugly workarounds, such as the following: >>>> >>>> USERCFG[render] QLIST=default,weekend >>>> USERCFG[render] MAXPROC[QOS:default]=1024 MAXPROC=2848 >>>> >>>> SRCFG[weekday] QLIST=default DAYS=Mon,Tue,Wed,Thu,Fri >>>> SRCFG[weekend] QLIST=default,weekend DAYS=Sat,Sun >>>> >>>> This, however, requires that the user launch and/or qalter their extra >>>> jobs to the 'weekend' QOS, possibly using a cron script (ugh). I've heard >>>> that this can interfere with jobs which would span reservations, so you may >>>> also want to double-check with a long job. >>>> >>>> It may also help to try combining several different throttling limits >>>> instead of a strict day-driven policy; for example, you can set a TIMELIMIT >>>> in conjunction with a MAXPS, along with MAXPROC and/or MAXJOB, perhaps with >>>> soft and hard limits, to tie limitations to queue utilization and other >>>> users' likely wait times (which, I assume, are the two primary factors >>>> motivating your intended policy). >>>> >>>> Sorry I can't do better, but since Maui is deprecated, you sometimes >>>> have to experiment to figure out what will work. The short answer (AFAIK) >>>> is that 'simple' day-driven limits are not so simple in Maui, since >>>> reservation sharing can be clumsy and because detailed quantitative >>>> limitations work better with credentials (USER, GROUP, ACCOUNT, CLASS, and >>>> QOS) than they do with reservations. If anyone else has a more elegant >>>> approach (or even ideas/suggestions/thoughts) I'd be very interested. >>>> >>>> Phil >>>> >>>> On Tue, Jun 24, 2014 at 11:17 AM, Macbeth R. <[email protected]> >>>> wrote: >>>> >>>>> I want to schedule MAXPROC for a specific user with different settings >>>>> on weekends, for example: >>>>> >>>>> weekdays: >>>>> USERCFG[testuser] MAXPROC=1024 >>>>> >>>>> on weekend: >>>>> USERCFG[render] MAXPROC=2848 >>>>> >>>>> >>>>> How can I achieve this? is there any settings like PRIME TIME, and >>>>> NON PRIME settings in MAUI? like torque? >>>>> >>>>> THANKS in advance! >>>>> >>>>> >>>>> -- >>>>> CONFIDENTIALITY NOTICE: This e-mail message is intended only for the >>>>> above-mentioned recipient(s). Its content is confidential. If you have >>>>> received this e-mail by error, please notify us immediately and delete it >>>>> without making a copy, nor disclosing its content, nor taking any action >>>>> based thereon. Thank you. >>>>> >>>>> _______________________________________________ >>>>> torqueusers mailing list >>>>> [email protected] >>>>> http://www.supercluster.org/mailman/listinfo/torqueusers >>>>> >>>>> >>>> >>> >>> >>> -- >>> CONFIDENTIALITY NOTICE: This e-mail message is intended only for the >>> above-mentioned recipient(s). Its content is confidential. If you have >>> received this e-mail by error, please notify us immediately and delete it >>> without making a copy, nor disclosing its content, nor taking any action >>> based thereon. Thank you. >>> >> >> > > > -- > CONFIDENTIALITY NOTICE: This e-mail message is intended only for the > above-mentioned recipient(s). Its content is confidential. If you have > received this e-mail by error, please notify us immediately and delete it > without making a copy, nor disclosing its content, nor taking any action > based thereon. Thank you. >
_______________________________________________ mauiusers mailing list [email protected] http://www.supercluster.org/mailman/listinfo/mauiusers
