On Mon, Oct 03, 2016 at 02:56:05PM +0200, Raimo Niskanen wrote:
> On Fri, Sep 30, 2016 at 01:02:10PM +0200, Otto Moerbeek wrote:
> > On Fri, Sep 30, 2016 at 09:10:21AM +0200, Raimo Niskanen wrote:
> >
> > > On Wed, Sep 28, 2016 at 09:19:51AM +0200, Raimo Niskanen wrote:
> > > > Dear misc@
> > > >
> > > > I have searched the archives and read the documentation of
> > > > login.conf(5),
> > > > ksh(1):ulimit and can not find how to limit the amount of physical
> > > > memory a
> > > > process may use.
> > > >
> > > > I have the following limits where I have set down ulimit -m and ulimit
> > > > -l
> > > > to 10000 kbytes in an attempt to limit the process I spawn which is
> > > > the Erlang VM.
> > > >
> > > > $ ulimit -a
> > > > time(cpu-seconds) unlimited
> > > > file(blocks) unlimited
> > > > coredump(blocks) unlimited
> > > > data(kbytes) 33554432
> > > > stack(kbytes) 8192
> > > > lockedmem(kbytes) 10000
> > > > memory(kbytes) 10000
> > > > nofiles(descriptors) 1024
> > > > processes 1024
> > > >
> > > > Note that the machine has got 8 GB of physical memory and 8 GB of swap
> > > > and
> > > > that I have set datasize=infinity in /etc/login.conf. I got
> > > > datasize=33554432 which seems to be the same as kern.shminfo.shmmax.
> > > > The datasize is twice the physical memory + swap.
> > > >
> > > > Then I start the Erlang VM and tell it to allocate an address block of
> > > > 30000
> > > > MByte for future use where it will store all literal data in the same
> > > > block
> > > > (this is a garbage collector optimization). Not much of this data is
> > > > actually used.
> > > >
> > > > 68196 beam CALL
> > > > mmap(0,0x753000000,0<PROT_NONE>,0x1002<MAP_PRIVATE|MAP_ANON>,-1,0)
> > > > 68196 beam RET mmap 11871265173504/0xacbfe8b3000
> > > >
> > > > Note the protection flags on the block. No access is allowed. This
> > > > trick
> > > > works just fine; here is what top says:
> > > >
> > > > load averages: 0.15, 0.13, 0.09 frerin.otp.ericsson.se
> > > > 08:49:46
> > > > 48 processes: 47 idle, 1 on processor up
> > > > 13:49
> > > > CPU0 states: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt,
> > > > 100% idle
> > > > CPU1 states: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt,
> > > > 100% idle
> > > > Memory: Real: 43M/636M act/tot Free: 7028M Cache: 508M Swap: 0K/8155M
> > > >
> > > > PID USERNAME PRI NICE SIZE RES STATE WAIT TIME CPU
> > > > COMMAND
> > > > 68196 raimo 2 0 29G 15M sleep poll 0:00 1.42% beam
> > > >
> > > > So I have a process with a data size of 29 GB on a machine with 16 GB
> > > > memory + swap. I have also tried to start an additional Erlang VM that
> > > > also allocates 29 GB of virtual memory which also works.
> > > >
> > > > That this is allowed is just fine for me - this trick of allocating a
> > > > "large enough" PROT_NONE memory to get one address range for some
> > > > special
> > > > data type is very useful for the Erlang VM. But I wonder how to limit
> > > > the
> > > > actual memory use? Setting down ulimit -m and ulimit -l to 10000 kbytes
> > > > did not prevent this process from getting 15 MByte of "RES" memory...
> > > >
> > > > Is there some way to limit the actual amount of memory for a process
> > > > when I
> > > > need to set up the datasize to allow for large unused virtual memory
> > > > blocks?
> > >
> > > I have found clues in getrlimit,setrlimit(2):
> > >
> > > RLIMIT_DATA The maximum size (in bytes) of the data segment for a
> > > process; this includes memory allocated via malloc(3)
> > > and all other anonymous memory mapped via mmap(2).
> > > :
> > > RLIMIT_RSS The maximum size (in bytes) to which a process's
> > > resident set size may grow. This imposes a limit
> > > on the amount of physical memory to be given to a
> > > process; if memory is tight, the system will prefer
> > > to take memory from processes that are exceeding
> > > their declared resident set size.
> > >
> > > Now I try to figure out the implications of this... If I set the data
> > > size
> > > so the sum of the data sizes for all processes in the system is larger
> > > than
> > > physical memory + swap, then any process may allocate the last block of
> > > memory in the system so a more important process later will fail to
> > > allocate?
> >
> > yes.
> >
> > >
> > > And the memoryuse limit is rather toothless since there is no immediate
> > > check of this limit. When the system gets low on memory; is all that
> > > happens that processes that exceed their memoryuse limit probably will get
> > > blocks swapped out?
> >
> > RLIMIT_DATA *is* enforced, but it could be that PROT_NONE memory is
> > not counted. I don;t know atm.
>
> That PROT_NONE is not counted sounds just as we want it to be...
>
> That RLIMIT_DATA *is* enforced does not rhyme with what I saw, or I do not
> know what I saw... As you can se above I had set ulimit -m 10000 (kbytes)
> and yet top reports RES 15M. Is that not over the limit? The PROT_NONE
> memory is reported in the 29GB entry by top. I can easily within the
> erlang emulator construct a large list of integers that can not be in
> PROT_NONE memory and squeeze the RES entry up to above 10000M...
RES vs non-RES is not something that matters for RLIMIT_DATA.
RLMIT_DATA only applies to anonymous mappings. SIZE and RES show both
anonymous and non-anonymous data. I expect thta you have
some non-anonymous mappings as wel. procmap(8) shows more details.
Looing again at your ulimit numbers. You have a high limit on data. So it is
no surprise you do not hit that limit.
AFAIK, the memory limit *is* applied to RES, buth only if there is a general
shortage on physical mem.
-Otto
