> So what should the kernel do? I don't know how thigs work under the hood today (I might have partially known in the times of sbrk()), but I would suppose that malloc() will ultimatively result in some system call enlarging the heap/data segment/whatever. That system call could simply fail.
I assume my impression is completely wrong (today). But then, how can a malloc() fail before the process gets killed?