Michael van Elst <mlel...@serpens.de> writes: > On Wed, Oct 23, 2024 at 11:59:17PM +0700, Robert Elz wrote: > >> Everything should be counted in one of those three categories for resource >> limit purposes. > > The three categories no longer match the real world when people started > to use shared memory, and you can not pretend that these are still > sufficient as a model. > > But more important, the limits are also no longer fit for their purpose. > What does a memory limit mean, when you can still allocate address space > and what does an address space limit mean, when it doesn't affect > memory usage ?
I am coming into this discussion wanting to set limits that do what I want: to control the amount of memory in use by a process, in order to: a) protect my system from piggy processes, both because they slow things down, and because of other bugs running out of memory leads to lockups b) understand whether a workload fits in a limit, to help debug a program that should use less memory than it does POSIX has some words, and one can argue about how that should be mapped to the modern world. My take is that when I run a program, the "text" size is the program's size(1) plus that of the shlibs. There's a minor complexity with dlopening further libraries, but generally, this is not where the pain is. POSIX doesn't have a label/limit for this and that's a clue that it is not mysterious, hard to debug, and a source of woe. And, the "data" size I see as the initial data plus anything that smells like heap or allocation. I intend to limit the total amount of memory used to store data. So anon mmap, is really the same thing as sbrk, from the user/sysadmin viewpoint of "did the program get some memory to use" and "does the system now have less available". The fact that malloc is using anon mmap instead of sbrk -- even though they are basically the same thing operationally -- is just an implementation detail. I can see that mmap of a file is different, as that is letting you see data whose storage is part of the buffer cache (loosely speaking as I am intending to, because I'm talking about unix in general, not just netbsd). The virtual limit is awkward as there is a lot involved and it's really not obvious what anything means. And, it's not what I want, which is "address space allocated that is going to be backed by allocated pages if I use it". The rss limit seems conceptually broken as all the rest are "shall not" and are limits. rss "limit" is like vm.filemax which is "sort of protected up to x". The setrlimit man page describes it incorrectly: "The maximum size (in bytes) to which a process's resident set size may grow." The real problem is that if I want to say "this program is allowed 8192 MB of data, and I want malloc to fail after that", I can't. And I can't do it portably. I found that on Linux, "ulimit -d" did what I expected (and what I want). I think kre@ is basically saying that's how NetBSD should behave also. My impression is that it wasn't all that long ago that I had to bump up 'datasize' in login.conf to get programs to run.
