Martin Buchholz wrote:
Below is a java program and a shell extract demonstrating the problem
on a Ubuntu-hardy-like system. The fork() fails from within a large
Java process (but normal commands started from an
independent shell continue to work fine).
Linux overcommit heuristics are hard to understand.
The failing fork problem becomes more serious on a server system
running only one significant large program, which is using most of
the memory on the system, perhaps by sysctl'ing
the vm.overcommit_ratio to a value higher than the default 50%.
On systems with two large processes using most of memory,
you may see rare failures when both try to fork() at the same time..
I played around with your test and I see similar behaviour on my Ubuntu
8.04 system.
When you switch on "always overcommit" (not the default) then you get
the expected
behaviour, ie. you can go quite close to the VM limit, and manage to
spawn a small command
from Java.
However, in the default "heuristic" mode, it doesn't work. The comment
on the web page
you mentioned says:
"Heuristic overcommit handling. Obvious overcommits of
address space are refused. Used for a typical system. It
ensures a seriously wild allocation fails while allowing
overcommit to reduce swap usage:
Perhaps, this specifically does not suit the Java VM situation where
fork() of a very
large process, is considered an "obvious overcommit".
In that context, I'll look into using the clone() system call on Linux.
It seems
to be quite versatile and configurable. It looks likely that we can
avoid the use
of the helper command with it as well.
- Michael.
