On Jan 17, 2010, at 11:31 AM, Ashley Pittman wrote: > On 10 Jan 2010, at 03:45, Barrett, Brian W wrote: > >> We should absolutely not change this. For simple applications, yes, things >> work if large blocks are allocated on the heap. However, ptmalloc (and most >> allocators, really), can't rationally cope with repeated allocations and >> deallocations of large blocks. It would be *really bad* (as we've seen >> before) to change the behavior of our version of ptmalloc from that which is >> provided by Linux. Pain and suffering is all that path has ever lead to. > > I think there are two different issues here that are in danger of being > confused. Having OMPI use it's own ptmalloc implantation as it has done at > times in the past has been problematic and should be avoided in my opinion.
I agree, and I also don't believe I'm confusing them. The shipping of ptmalloc2 is unfortunate and yet required until a better solution appears. The proposal/request which started this e-mail chain was to change a hard-coded constant in the ptmalloc2 shipped with Open MPI. While the fact that we can do this is an artifact of Open MPI providing it's own ptmalloc implementation, it none the less fundamentally changes how the memory allocator works when applications are linked against Open MPI, away from the default behavior found on Linux systems. > Tuning the libc malloc implementation using the options they provide to do is > is valid and provides real benefit to a lot of applications. For the record > we used to disable mmap based allocations by default on Quadrics systems and > I can't think of a single case of people who needed to re-enable it. I'm glad that you've been so fortunate. Unfortunately, I have a large application base in which that is not the case, and we have had to use mmap based allocation, otherwise memory usage would essentially grow without bound. So we go back to the age-old debate of correctness vs. performance. Brian
