Hi Danny, Mark, Just to follow up on this question a little..
> > At the moment we are using SLURM with Moab (for scheduling) and Gold > > (for accounting), but I'm having a look at whether we can move to an > > all-SLURM setup to do the same thing. We have the same setup (but with maui instead of moab), and would also like to move to an all-SLURM setup if possible. Banking/reporting is a requirement for our setup, due to the nature of our funding. > > Using this fairly simple setup I can enforce user's jobs to be run > > against only accounts that they're associated with. The next thing I > > need to do (that I'm currently stuck on) is work out how to assign > > quotas to these accounts (a number of Core-Hours if you like) that > > decrease by an appropriate amount every time a job is run. The > > documentation often refers to accounts as "bank accounts" which makes > > me think that this can be done and I just haven't work out how to yet. > > What documentation are you looking at is one question. You can probably look > at https://computing.llnl.gov/linux/slurm/accounting.html to get a good idea > how to do what you want. Each association and QOS have a GrpCPUMin limit. > If you decide you don't want to do fairshare (the preferred way of doing > things) you can do the hard limit stuff using the this limit along with the > priority/multifactor plugin explained here > https://computing.llnl.gov/linux/slurm/priority_multifactor.html. > > Look primarily at the PriorityDecayHalfLife and PriorityUsageResetPeriod > options for the slurm.conf. The functionality that I think SLURM will do easily is this: * have individual associations/accounts, with one or more users in each * potentially hierarchical accounts (not a biggie at present) The (Gold) functionality that I'm not sure about: * being able to easily `deposit' resources (e.g. CPU hours) into accounts -- although maybe 'sacctmgr modify ... set GrpCPUMins=XXX' will do that (it would be nice if the value can be additive/subtractive, rather than absolute) * being able to see the current `balance' for a given account, for both the user and the site admin * lifetimes/deadlines for deposited CPU hours (e.g. a project might only have a lifetime of 6 months) * (optional) different charge rates based on node features (e.g. if most nodes are uniform, but you have a subset of large memory or more cores; or different rates per partitions) * (optional) having an extra charge for reservations, and/or a charge for un-used hours in a reservation * (optional) the equivalent of Gold's audit trail, of being able to see the history of when the account/association was updated I'm thinking that setting PriorityDecayHalfLife=0 so that the resources (e.g. GrpCPUMins) are absolute will get us most of the way there, but I'm not sure about these other requirements. Any ideas about these features? Are they already there in slurm? Thanks, Paddy -- Paddy Doyle Trinity Centre for High Performance Computing, Lloyd Building, Trinity College Dublin, Dublin 2, Ireland. http://www.tchpc.tcd.ie/
