Mike Coleman wrote:
Andrew, this is on an (intel) x86_64 box with 64GB of RAM. I don't
recall the maker or details of the architecture off the top of my
head, but it would be something "off the rack" from Dell or maybe HP.
There were other users on the box at the time, but nothing heavy or
that gave me any reason to think was affecting my program.
It's running CentOS 5 I think, so that might make glibc several years
old. Your malloc idea sounds plausible to me. If it is a libc
problem, it would be nice if there was some way we could tell malloc
to "live for today because there is no tomorrow" in the terminal phase
of the program.
I'm not sure exactly how to attack this. Callgrind is cool, but no
way will work on something this size. Timed ltrace output might be
interesting. Or maybe a gprof'ed Python, though that's more work.
Some malloc()s (notably FreeBSD's) can be externally tuned at runtime
via options in environment variables or other mechanisms - the malloc
man page on your system might be helpful if your platform has something
like this.
It is likely that PyMalloc would be better with a way to disable the
free()ing of empty arenas, or move to an arrangement where (like the
various type free-lists in 2.6+) explicit action can force pruning of
empty arenas - there are other usage patterns than yours which would
benefit (performance wise) from not freeing arenas automatically.
--
-------------------------------------------------------------------------
Andrew I MacIntyre "These thoughts are mine alone..."
E-mail: andy...@bullseye.apana.org.au (pref) | Snail: PO Box 370
andy...@pcug.org.au (alt) | Belconnen ACT 2616
Web: http://www.andymac.org/ | Australia
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
