Thanks for the reply! A concrete example is interactive jobs (R, python, etc). I only want the users to request minimum needed amount of memory but then I don't want their session to generate an error if they try to allocate more memory - and the free memory is available in the system. The upper limit I was referring to is not for the users to request _initially_, only if they try to allocate that much in the course of their interactive session.
So I imagine this sequence of events: 1. A user launches interactive R job and request 10G of memory. 2. R session starts with 10G of guaranteed memory allocation. 3. The user creates or loads some data in R which consume more than 10G. The partition-specific cap (the upper bound I was referring to) is, say 50G. 4. If there is free memory on the server, the memory is allocated and the user continues her work. 5. The user wants to allocate 100G more memory in the same R session. 6. The request returns an out of memory error because the total memory for the session will be more than the 50G cap - regardless of whether the free memory is available. The question is how to achieve that with job/partition configuration? Thanks, Alex On Tue, Apr 25, 2017 at 1:14 PM, E.S. Rosenberg < esr+slurm-...@mail.hebrew.edu> wrote: > (I am by no means an expert, these observations are based on how our > clusters function and some things may actually be more configurable) > > On Tue, Apr 25, 2017 at 9:06 PM, Alexander Vorobiev < > alexander.vorob...@gmail.com> wrote: > >> I seem to be confused about memory configuration for slurm jobs. Is it >> possible to submit jobs with memory bounds? >> >> For instance a job would be guaranteed to have some minimum amount of >> memory but would not be allowed to allocate more than certain maximum. I >> have cgroups on the servers which I can use for Slurm. >> >> How do I configure the partition and the job so that it starts executing >> as soon as it's minimum memory is available but will be denied any memory >> requests beyond the upper bound? >> > The user can set the amount of memory they would like for their job in > several ways (see --mem and --mem-per-cpu in man sbatch), a job that > exceeds the predefined (requested) memory ends up being killed. > As cluster admin you can set the upper bound of how much memory a user can > request and you also set the default amount of memory they get if they > don't make a specific memory request. > >> >> Slightly more complicated case is when a job spawns (forks) child >> processes. How do I control their memory limits? >> > All processes part of the cgroup count toward the memory (and CPU) usage, > if the combined set of processes exceeds the max memory the cgroup dies. > HTH, > Eli > >> >> Thanks, >> Alex >> > >
