Unbelievable as this may seem, this crazy over-committing malloc behavior is by now "a classic" -- I first fought against it in 1990, when IBM released AIX 3 for its then-new RS/6000 line of workstations; in a later minor release they did provide a way to optionally switch this off, but, like on Linux, it's a system-wide switch, NOT per-process:-(.
I concur with http://www.win.tue.nl/~aeb/linux/lk/lk-9.html (the best explanation I know of the subject, and recommended reading) which, on this subject, says "Linux on the other hand is seriously broken." (just like AIX 3 was). Sad to learn that BSD is now also broken in the same way:-(. Alex On Wed, Sep 17, 2008 at 8:00 AM, James Y Knight <[EMAIL PROTECTED]> wrote: > > On Sep 17, 2008, at 10:53 AM, Andrew MacIntyre wrote: > >> Martin v. Löwis wrote: >>>> >>>> I haven't yet tried posting a query to a FreeBSD list, as it could >>>> simply >>>> be a bug on amd64, but I was wondering whether there was anything (other >>>> than deactivating tests and documenting use of ulimit -v on this >>>> platform) that could be done to work around this behaviour. >>> >>> I think it should be possible to debug this (for people with access to >>> such a system, and appropriate skills). >>> I find it hard to believe that a large malloc will simply crash the >>> process, rather than returning NULL. More likely, there is a NULL >>> returned somewhere, and Python (or libc) fails to check for it. >> >> A simple C program doing a repetitive malloc(), much as pymalloc would >> with continuous demand, does indeed not see any NULL from malloc() when >> swap is exhausted but the process gets KILLed (the allocated memory does >> have to be written to to force the situation...) >> >> I'll take this up with FreeBSD folk, but I'm open to ideas as to how >> best to deal with the problem in the context of the test suite pending >> resolution by FreeBSD. > > Linux does the same thing, unless the user has explicitly configured that > behavior off. Search the web for linux overcommit. It's controlled by the > vm.overcommit_memory sysctl. Although linux's default is some heuristic that > might make Python's test case work right in most cases, depending on malloc > returning NULL is not something you can actually depend on. > > James > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/aleaxit%40gmail.com > _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
