Re: killed: out of swap
> Perhaps my understanding is wrong No.
Re: killed: out of swap
On 2022-06-15 18:41, Mouse wrote: By the way. This obviously does not at all solve the problem that the OP had. He was writing code with the expectation that malloc() should fail. [...] A killed process won't make the OP happy, even if it was his own program/process. I'm not sure that last sentence is true. As I read it, the reaction to malloc failing would have been that we've put as much pressure on as we can, so it's time to exit. If so, killing that process is a reasonable reaction - and that's what my proposed approaches were based on. Perhaps my understanding is wrong, in which case not much will help except no overcommit. I might have misunderstood the purpose. I thought there was a desire to stress things a bit once you get to the memory full state to see how things were behaving. So you might be absolutely right. Johnny -- Johnny Billquist || "I'm on a bus || on a psychedelic trip email: b...@softjar.se || Reading murder books pdp is alive! || tryin' to stay hip" - B. Idol
Re: killed: out of swap
> By the way. This obviously does not at all solve the problem that > the OP had. He was writing code with the expectation that malloc() > should fail. [...] A killed process won't make the OP happy, even > if it was his own program/process. I'm not sure that last sentence is true. As I read it, the reaction to malloc failing would have been that we've put as much pressure on as we can, so it's time to exit. If so, killing that process is a reasonable reaction - and that's what my proposed approaches were based on. Perhaps my understanding is wrong, in which case not much will help except no overcommit. /~\ The ASCII Mouse \ / Ribbon Campaign X Against HTMLmo...@rodents-montreal.org / \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
Re: killed: out of swap
By the way. This obviously does not at all solve the problem that the OP had. He was writing code with the expectation that malloc() should fail. For this, we need to get something that will not allow overcommitting memory, and where malloc() can then return an error instead of the process getting killed. A killed process won't make the OP happy, even if it was his own program/process. Johnny On 2022-06-15 17:41, Johnny Billquist wrote: On 2022-06-15 16:01, Michael van Elst wrote: b...@softjar.se (Johnny Billquist) writes: They might be the reason for the memory shortage. You can prefer large processes as victims or protect system services to keep the system managable. So when one process tries to grow, you'd kill a process that currently have no issues in running? All processes have issues on that system and the goal is to keep things alive so that you can recover, a system hang, crash or reboot is the worst outcome. Maybe, but not definitely. And the outcome is in general processes being killed, which basically should never result in an outright crash or reboot. Not even a hang, although if the wrong process is killed, you might end up not being able to access the system, so it's a bit of a grey area. Obviously there is no heuristic that can predict what action will have the best outcome and which causes the least damage. Guessing on the cost of various kinds of damage is an impossible task by itself as that is fairly subjective. Agreed. But the one thing that is known at specific point in time is that there is one process who needed one more page, which could not be satisfied. All the other processes at that moment in time are not in trouble. Which also means, we do not know if killing another process is enough to keep this process going, and we do not know if that other process would ever get into trouble at all. So we are faced with the choice of killing one process we know are in trouble, or speculatively kill something else, and then hope that would help. The suggestion that we'd add some kind of hinting could at least help some, but it is rather imperfect. And if we don't have any hints, we're back in the same place again. But there can be a heuristic that helps in many cases, and for the rest you can hint the system. If you can come up with some heuristics, it would be interesting to see them. I don't see any easy ones. Johnny -- Johnny Billquist || "I'm on a bus || on a psychedelic trip email: b...@softjar.se || Reading murder books pdp is alive! || tryin' to stay hip" - B. Idol
Re: killed: out of swap
On 2022-06-15 16:01, Michael van Elst wrote: b...@softjar.se (Johnny Billquist) writes: They might be the reason for the memory shortage. You can prefer large processes as victims or protect system services to keep the system managable. So when one process tries to grow, you'd kill a process that currently have no issues in running? All processes have issues on that system and the goal is to keep things alive so that you can recover, a system hang, crash or reboot is the worst outcome. Maybe, but not definitely. And the outcome is in general processes being killed, which basically should never result in an outright crash or reboot. Not even a hang, although if the wrong process is killed, you might end up not being able to access the system, so it's a bit of a grey area. Obviously there is no heuristic that can predict what action will have the best outcome and which causes the least damage. Guessing on the cost of various kinds of damage is an impossible task by itself as that is fairly subjective. Agreed. But the one thing that is known at specific point in time is that there is one process who needed one more page, which could not be satisfied. All the other processes at that moment in time are not in trouble. Which also means, we do not know if killing another process is enough to keep this process going, and we do not know if that other process would ever get into trouble at all. So we are faced with the choice of killing one process we know are in trouble, or speculatively kill something else, and then hope that would help. The suggestion that we'd add some kind of hinting could at least help some, but it is rather imperfect. And if we don't have any hints, we're back in the same place again. But there can be a heuristic that helps in many cases, and for the rest you can hint the system. If you can come up with some heuristics, it would be interesting to see them. I don't see any easy ones. Johnny -- Johnny Billquist || "I'm on a bus || on a psychedelic trip email: b...@softjar.se || Reading murder books pdp is alive! || tryin' to stay hip" - B. Idol
Re: killed: out of swap
hello. One algorithm I think that might be a good option and would address the concerns that I've seen on this topic is to kill the process with the latest start time when looking for resources to free. Obviously this would fail in the case where a long-running process suddenly starts consuming memory, but in general, it seems that badly behaved processes would typically begin behaving badly from the beginning of their inception. In any case, it should address the issue of X or syslogd getting killed, since they would often have older start times than processes that cause the trouble. -Brian
Re: killed: out of swap
b...@softjar.se (Johnny Billquist) writes: >> They might be the reason for the memory shortage. You can prefer large >> processes as victims or protect system services to keep the system >> managable. >So when one process tries to grow, you'd kill a process that currently >have no issues in running? All processes have issues on that system and the goal is to keep things alive so that you can recover, a system hang, crash or reboot is the worst outcome. Obviously there is no heuristic that can predict what action will have the best outcome and which causes the least damage. Guessing on the cost of various kinds of damage is an impossible task by itself as that is fairly subjective. But there can be a heuristic that helps in many cases, and for the rest you can hint the system.
Re: killed: out of swap
On 2022-06-15 11:09, David Brownlee wrote: On Wed, 15 Jun 2022 at 08:31, Johnny Billquist wrote: On 2022-06-15 06:57, Michael van Elst wrote: b...@softjar.se (Johnny Billquist) writes: I don't see any realistic way of doing anything with that. It's basically the first process that tries to allocate another page when there are no more. There are no other processes at that moment in time that have the problem, so why should any of them be considered? They might be the reason for the memory shortage. You can prefer large processes as victims or protect system services to keep the system managable. So when one process tries to grow, you'd kill a process that currently have no issues in running? Which means you might end up killing a lot of non-problematic processes because of one runaway process? Seems to me to not be a good decision. As opposed to the process which had a successful malloc some time ago and is running without issues, and is just about to try to use some of its existing allocation? That is speculation, which is my problem here. You are trading a known requester of non-existant memory for speculation that another process *might* want non-existant memory. Both options are wrong in some cases. Having a way to influence the order in which processes are chosen would seem to be the best way to end up with a better outcome. The existing behaviour should remain an option, but (at least for me) it would not be the one chosen I (obviously) disagree. :-) Johnny -- Johnny Billquist || "I'm on a bus || on a psychedelic trip email: b...@softjar.se || Reading murder books pdp is alive! || tryin' to stay hip" - B. Idol
Re: killed: out of swap
On Wed, 15 Jun 2022 at 08:31, Johnny Billquist wrote: > > On 2022-06-15 06:57, Michael van Elst wrote: > > b...@softjar.se (Johnny Billquist) writes: > > > >> I don't see any realistic way of doing anything with that. > >> It's basically the first process that tries to allocate another page > >> when there are no more. There are no other processes at that moment in > >> time that have the problem, so why should any of them be considered? > > > > They might be the reason for the memory shortage. You can prefer large > > processes as victims or protect system services to keep the system > > managable. > > So when one process tries to grow, you'd kill a process that currently > have no issues in running? Which means you might end up killing a lot of > non-problematic processes because of one runaway process? Seems to me to > not be a good decision. As opposed to the process which had a successful malloc some time ago and is running without issues, and is just about to try to use some of its existing allocation? Both options are wrong in some cases. Having a way to influence the order in which processes are chosen would seem to be the best way to end up with a better outcome. The existing behaviour should remain an option, but (at least for me) it would not be the one chosen David
Re: killed: out of swap
On 2022-06-15 06:57, Michael van Elst wrote: b...@softjar.se (Johnny Billquist) writes: I don't see any realistic way of doing anything with that. It's basically the first process that tries to allocate another page when there are no more. There are no other processes at that moment in time that have the problem, so why should any of them be considered? They might be the reason for the memory shortage. You can prefer large processes as victims or protect system services to keep the system managable. So when one process tries to grow, you'd kill a process that currently have no issues in running? Which means you might end up killing a lot of non-problematic processes because of one runaway process? Seems to me to not be a good decision. Johnny -- Johnny Billquist || "I'm on a bus || on a psychedelic trip email: b...@softjar.se || Reading murder books pdp is alive! || tryin' to stay hip" - B. Idol
Re: killed: out of swap
Den Tue, 14 Jun 2022 20:48:48 + skrev David Holland : > NetBSD doesn't do this very well; usually in my experience an OOM > situation results in syslogd getting killed, which is both unhelpful > in terms of freeing up resources and actively bad for other reasons. > It's also not uncommon for the ax to fall on the X server, which does > usually end up releasing resources but is pretty much never what the > user wants. This matches my expericence very well, in that either syslog or X is the victim. In my case, it's usually darktable that decides that obese isn't fat enough and starts growing until there is no more memory. So killing X indirectly also ends in killing the offender. However in my case, a selection that killed the largest process would be a very nice option. -- Eivind Nicolay Evensen
Re: killed: out of swap
b...@softjar.se (Johnny Billquist) writes: >I don't see any realistic way of doing anything with that. >It's basically the first process that tries to allocate another page >when there are no more. There are no other processes at that moment in >time that have the problem, so why should any of them be considered? They might be the reason for the memory shortage. You can prefer large processes as victims or protect system services to keep the system managable.
Re: killed: out of swap
>> > I have a program that keeps malloc()ing (and scribbling a bit >> > into the allocated memory) until malloc() fails. The >> > intention is to put pressure on the VM system to find out how >> > much pool cache memory it can reclaim. >> Such a program would be a prime candidate for declaring itself a >> preferred out-of-swap victim. >> Perhaps even better would be a way for userland to tell the kernel >> "pretend you're under severe RAM pressure and free what you can" >> without needing to actually run the system out of pages. > Is this something madvise(2) could be extended to do? In principle? It could be, sure. I would, however, suggest something else. These operations do not apply to specific ranges of VM, so an API that's designed for actions on particular ranges of VM seems inappropriate. For the former suggestion, for declaring a process a preferred (or maybe even dispreferred) out-of-swap victim, sysctl's proc.$PID hierarchy strikes me as a better tool, especially because root can then protect certain processes from the command line, by making them less preferred. I didn't go into details of what I was imagining. I was thinking of each process having a preference; when the kernel runs out of swap, instead of simply killing the faulting process it would kill one of the processes with the maximum victim preference value, perhaps a signed int with the default being zero or some such. Then root could set preference negative for particularly important processes, or positive for fluff, or very positive for things like the "try to reclaim RAM" process outlined. I would say it should be like nice: any process can make itself get less-preferred treatment, but it should take privilege to go the other way. For the latter suggestion, for provoking what reclaims are possible without actually risking running out of pages...I'm not sure. A new syscall? A new sysctl? An ioctl on /dev/mem? There are many possibilities. /~\ The ASCII Mouse \ / Ribbon Campaign X Against HTMLmo...@rodents-montreal.org / \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
Re: killed: out of swap
On Tue, Jun 14, 2022 at 07:59:33PM +0200, Johnny Billquist wrote: > > What might be interesting is a way to influence the order in which > > processes are chosen to kill... > > I don't see any realistic way of doing anything with that. > It's basically the first process that tries to allocate another page when > there are no more. There are no other processes at that moment in time that > have the problem, so why should any of them be considered? There are certainly ways. The history of the linux oom-killer makes for pretty good reading, actually. You want to find the process that's actually causing the problem, not some random process that happens to be the one to step in the hole. NetBSD doesn't do this very well; usually in my experience an OOM situation results in syslogd getting killed, which is both unhelpful in terms of freeing up resources and actively bad for other reasons. It's also not uncommon for the ax to fall on the X server, which does usually end up releasing resources but is pretty much never what the user wants. -- David A. Holland dholl...@netbsd.org
Re: killed: out of swap
>> > I have a program that keeps malloc()ing (and scribbling a bit
>> > into the allocated memory) until malloc() fails. The
>> > intention is to put pressure on the VM system to find out how
>> > much pool cache memory it can reclaim.
>> Such a program would be a prime candidate for declaring itself a
>> preferred out-of-swap victim.
> Well. First of all, such a program don't exist, as the malloc is not
> failing.
Strictly speaking, yes. I was talking about a program that satisfies
the intention, rather than the letter of the description. In this
case, something like "a program that keeps grabbing and touching memory
as long as it can, to provoke memory reclaims in the kernel", with its
death being due to out-of-swap rather than as a reaction to a failing
malloc.
>> It probably wouldn't be easy - the process which incurred the page
>> fault would have to be put to sleep pending the death of the victim
>> process - but it could provide for much better behaviour in
>> situations like this.
> Second - are you proposing that you'd keep some kind of statistics on
> mallocs done in the past in some way, in order to decide that this
> process should now be a candidate for a kill when you run out of
> pages?
No. Trying to do something "smart" with process behaviour history
strikes me as a disaster waiting to happen.
What I'm suggesting is that a program with the intent sketched above
should do something like
set_swap_kill_preference(1);
(or whatever the chosen API is - perhaps using sysctl?) in order to
indicate to the kernel that it is volunteering itself as a preferred
out-of-swap victim, even if it's not the process that incurs the
problematic page fault.
> Even more, how do we decide that it's actually malloc, or are any
> memory demand treated equally?
It's not truly malloc(); like most of this discussion, it's really
talking about a process that needs a new page to satisfy a write to a
COW or demand-zero page when there are no pages available.
To put it loosely, anything that could provoke an out-of-swap SIGKILL
from the kernel.
>> Perhaps even better would be a way for userland to tell the kernel
>> "pretend you're under severe RAM pressure and free what you can"
>> without needing to actually run the system out of pages.
> Well, the process killing only happens when we are really out of
> pages, so no amount of "free what you can" helps (unless I'm
> confused).
This was an attempt to address not the symptom but the underlying
desire. In this case, the original poster was saying, approximately,
"I wanted to do $THING and tried to do so [...outline...], and am
having $PROBLEM". Most of this thread has been talking about
addressing $PROBLEM. The above quote of mine ("Perhaps even
better...") was about addressing the desire for $THING differently, in
a way that doesn't risk provoking $PROBLEM.
/~\ The ASCII Mouse
\ / Ribbon Campaign
X Against HTMLmo...@rodents-montreal.org
/ \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
Re: killed: out of swap
hello. Is this something madvise(2) could be extended to do? -thanks -Brian On Jun 14, 2:47pm, Mouse wrote: } Subject: Re: killed: out of swap } >> What might be interesting is a way to influence the order in which } >> processes are chosen to kill... } > I don't see any realistic way of doing anything with that. It's } > basically the first process that tries to allocate another page when } > there are no more. There are no other processes at that moment in } > time that have the problem, so why should any of them be considered? } } To answer that, consider the original poster's situation: } } > I have a program that keeps malloc()ing (and scribbling a bit } > into the allocated memory) until malloc() fails. The } > intention is to put pressure on the VM system to find out how } > much pool cache memory it can reclaim. } } Such a program would be a prime candidate for declaring itself a } preferred out-of-swap victim. SunOS chill(1) - or was it chill(8)? - } might be another example, though that's of minimal relevance to NetBSD. } } It probably wouldn't be easy - the process which incurred the page } fault would have to be put to sleep pending the death of the victim } process - but it could provide for much better behaviour in situations } like this. } } Perhaps even better would be a way for userland to tell the kernel } "pretend you're under severe RAM pressure and free what you can" } without needing to actually run the system out of pages. } } /~\ The ASCII Mouse } \ / Ribbon Campaign } X Against HTML mo...@rodents-montreal.org } / \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B >-- End of excerpt from Mouse
Re: killed: out of swap
On 2022-06-14 20:47, Mouse wrote: What might be interesting is a way to influence the order in which processes are chosen to kill... I don't see any realistic way of doing anything with that. It's basically the first process that tries to allocate another page when there are no more. There are no other processes at that moment in time that have the problem, so why should any of them be considered? To answer that, consider the original poster's situation: > I have a program that keeps malloc()ing (and scribbling a bit > into the allocated memory) until malloc() fails. The > intention is to put pressure on the VM system to find out how > much pool cache memory it can reclaim. Such a program would be a prime candidate for declaring itself a preferred out-of-swap victim. SunOS chill(1) - or was it chill(8)? - might be another example, though that's of minimal relevance to NetBSD. Well. First of all, such a program don't exist, as the malloc is not failing. It probably wouldn't be easy - the process which incurred the page fault would have to be put to sleep pending the death of the victim process - but it could provide for much better behaviour in situations like this. Second - are you proposing that you'd keep some kind of statistics on mallocs done in the past in some way, in order to decide that this process should now be a candidate for a kill when you run out of pages? Are you sure it is a better candidate than the current process, who actually are demanding more pages at the moment? The other process might not in fact ever be hitting the condition, and would run on happily ever after, even though it did call malloc a number of times earlier. Even more, how do we decide that it's actually malloc, or are any memory demand treated equally? And yes, it also raises the question on how to handle the current process that caused a page demand that cannot be fulfilled at the moment. Perhaps even better would be a way for userland to tell the kernel "pretend you're under severe RAM pressure and free what you can" without needing to actually run the system out of pages. Well, the process killing only happens when we are really out of pages, so no amount of "free what you can" helps (unless I'm confused). Johnny -- Johnny Billquist || "I'm on a bus || on a psychedelic trip email: b...@softjar.se || Reading murder books pdp is alive! || tryin' to stay hip" - B. Idol
Re: killed: out of swap
>> What might be interesting is a way to influence the order in which >> processes are chosen to kill... > I don't see any realistic way of doing anything with that. It's > basically the first process that tries to allocate another page when > there are no more. There are no other processes at that moment in > time that have the problem, so why should any of them be considered? To answer that, consider the original poster's situation: > I have a program that keeps malloc()ing (and scribbling a bit > into the allocated memory) until malloc() fails. The > intention is to put pressure on the VM system to find out how > much pool cache memory it can reclaim. Such a program would be a prime candidate for declaring itself a preferred out-of-swap victim. SunOS chill(1) - or was it chill(8)? - might be another example, though that's of minimal relevance to NetBSD. It probably wouldn't be easy - the process which incurred the page fault would have to be put to sleep pending the death of the victim process - but it could provide for much better behaviour in situations like this. Perhaps even better would be a way for userland to tell the kernel "pretend you're under severe RAM pressure and free what you can" without needing to actually run the system out of pages. /~\ The ASCII Mouse \ / Ribbon Campaign X Against HTMLmo...@rodents-montreal.org / \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
Re: killed: out of swap
On 2022-06-14 19:57, David Brownlee wrote: On Tue, 14 Jun 2022 at 13:33, Robert Elz wrote: NetBSD implements overcommitted swap - many processes malloc() (or mmap() which that really becomes in the current implementation) far more memory than they're ever going to actually use. It is only when some real physical memory is required (rather than simply a marker "zero filled page might be required here") that the system actually allocates any real resources. Similarly pages mapped from a file only need swap space if they're altered - otherwise the file serves as the backing store for it. Once upon a time there was a method to turn overcommitted swap off, and require actual allocations (of RAM or swap) to be made for all reserved (virtual) memory. I used to enable that all the time - but I haven't seen any mention of it in ages, and the mechanism might no longer still exist. What might be interesting is a way to influence the order in which processes are chosen to kill... I don't see any realistic way of doing anything with that. It's basically the first process that tries to allocate another page when there are no more. There are no other processes at that moment in time that have the problem, so why should any of them be considered? Johnny -- Johnny Billquist || "I'm on a bus || on a psychedelic trip email: b...@softjar.se || Reading murder books pdp is alive! || tryin' to stay hip" - B. Idol
Re: killed: out of swap
On Tue, 14 Jun 2022 at 13:33, Robert Elz wrote: > > NetBSD implements overcommitted swap - many processes malloc() > (or mmap() which that really becomes in the current implementation) > far more memory than they're ever going to actually use. It is only > when some real physical memory is required (rather than simply a marker > "zero filled page might be required here") that the system actually > allocates any real resources. Similarly pages mapped from a file only > need swap space if they're altered - otherwise the file serves as the > backing store for it. > > Once upon a time there was a method to turn overcommitted swap off, and > require actual allocations (of RAM or swap) to be made for all reserved > (virtual) memory. I used to enable that all the time - but I haven't seen > any mention of it in ages, and the mechanism might no longer still exist. What might be interesting is a way to influence the order in which processes are chosen to kill... David
Re: killed: out of swap
> I assume my impression is completely wrong (today). OK, thanks for all the explanations and insights.
Re: killed: out of swap
NetBSD implements overcommitted swap - many processes malloc() (or mmap() which that really becomes in the current implementation) far more memory than they're ever going to actually use. It is only when some real physical memory is required (rather than simply a marker "zero filled page might be required here") that the system actually allocates any real resources. Similarly pages mapped from a file only need swap space if they're altered - otherwise the file serves as the backing store for it. Once upon a time there was a method to turn overcommitted swap off, and require actual allocations (of RAM or swap) to be made for all reserved (virtual) memory. I used to enable that all the time - but I haven't seen any mention of it in ages, and the mechanism might no longer still exist. kre
Re: killed: out of swap
> I have a program that keeps malloc()ing (and scribbling a bit into > the allocated memory) until malloc() fails. The intention is to put > pressure on the VM system to find out how much pool cache memory it > can reclaim. > When I run that program (with swap space unconfigured), it doesn't > terminate normally, but gets killed by the kernel with "out of swap". > Unfortunately, other processes happening to malloc() during that time > may get killed, too. As I think someone else said, it doesn't die in malloc, but in a page fault incurred when accessing the new memory. Other processes that get killed also get killed in page faults, not in malloc()s. > I don't quite get what the rationale for that is (or maybe I'm doing > something stupidely wrong). If I malloc(), and that fails, that > should fail and not kill me, no? One would think. But the system overcommits VM, allowing the total VM to exceed the available RAM (well, RAM+swap, but you said you had no swap configured). This is semi-necessary, to support things like a large process doing fork+exec (where almost all the VM is COW-shared between the fork and the exec, but doubling the VM use of the process would exceed available RAM). But it does mean that trying to allocate a new page, usually during a write pagefault to a COW or demand-zero page, can leave the OS with no available pages. Since there is no way to return an error to the "offending" process, the only alternatives are to kill the process, kill some _other_ process, or put the faulting process to sleep indefinitely in the hope that someone else will free up some memory. Someone chose the first of those options. (It is arguably the best of a rather bad lot.) I don't know whether there is a way to turn off VM overcommit. /~\ The ASCII Mouse \ / Ribbon Campaign X Against HTMLmo...@rodents-montreal.org / \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
Re: killed: out of swap
On 2022-06-14 12:59, Edgar Fuß wrote: So what should the kernel do? I don't know how thigs work under the hood today (I might have partially known in the times of sbrk()), but I would suppose that malloc() will ultimatively result in some system call enlarging the heap/data segment/whatever. That system call could simply fail. I assume my impression is completely wrong (today). But then, how can a malloc() fail before the process gets killed? Process limits for one. But I guess if your virtual memory becomes fragmented, and you request a too big chunk would be another reason. But malloc today relies on the lazy memory grabbing of the pager. Until you actually reference the memory, it don't yet have to be backed by anything. (Unless I remember something wrong.) Johnny -- Johnny Billquist || "I'm on a bus || on a psychedelic trip email: b...@softjar.se || Reading murder books pdp is alive! || tryin' to stay hip" - B. Idol
Re: killed: out of swap
> So what should the kernel do? I don't know how thigs work under the hood today (I might have partially known in the times of sbrk()), but I would suppose that malloc() will ultimatively result in some system call enlarging the heap/data segment/whatever. That system call could simply fail. I assume my impression is completely wrong (today). But then, how can a malloc() fail before the process gets killed?
Re: killed: out of swap
It's not the malloc that fails. It's the vm system trying to get a page for you. At which point it might not be your process that is trying to get a page when there are none free... So what should the kernel do? Johnny On 2022-06-14 12:01, Edgar Fuß wrote: I have a program that keeps malloc()ing (and scribbling a bit into the allocated memory) until malloc() fails. The intention is to put pressure on the VM system to find out how much pool cache memory it can reclaim. When I run that program (with swap space unconfigured), it doesn't terminate normally, but gets killed by the kernel with "out of swap". Unfortunately, other processes happening to malloc() during that time may get killed, too. I don't quite get what the rationale for that is (or maybe I'm doing something stupidely wrong). If I malloc(), and that fails, that should fail and not kill me, no? I'm surely missing something. -- Johnny Billquist || "I'm on a bus || on a psychedelic trip email: b...@softjar.se || Reading murder books pdp is alive! || tryin' to stay hip" - B. Idol
killed: out of swap
I have a program that keeps malloc()ing (and scribbling a bit into the allocated memory) until malloc() fails. The intention is to put pressure on the VM system to find out how much pool cache memory it can reclaim. When I run that program (with swap space unconfigured), it doesn't terminate normally, but gets killed by the kernel with "out of swap". Unfortunately, other processes happening to malloc() during that time may get killed, too. I don't quite get what the rationale for that is (or maybe I'm doing something stupidely wrong). If I malloc(), and that fails, that should fail and not kill me, no? I'm surely missing something.
