On May 24, 2008, at 9:31 PM, Patrick Geoffray wrote:
the topic of the memory hooks came up again. Brian was wondering if
we should [finally] revisit this topic -- there's a few things that
could be done to make life "better". Two things jump to mind:
- using mallopt on Linux
What about using the (probably) upcoming mmu notifiers and avoid ugly
hacks in user space ?
That would also be great. I don't know anything about these mmu
notifiers (I'm not much of a kernel guy), but anything that allows us
to get rid of the ugly hacks would be awesome. I assume that this
would only be for very new/recent Linux kernels...? But even if you
have to have Linux kernel v2.6.500, it would be good to use such
things if it provides a good path forward. It's easy enough to add
configure tests to check if such mechanisms are available.
We still need to support the old kernels, though. So mallopt might be
ok in a few cases:
1. when you're not using a high performance network (because I assume
we can disable mallopt's use at runtime, vs. ptmalloc which we have to
decide to use at OMPI configure/compile time)
2. if you're using a high performance network, but using ptmalloc
presents problems
3. if you're using a high performance network and don't care about
giving memory back to the OS (i.e., your app's memory usage doesn't
cause problems if you never give memory back to the OS).
So to be clear: I'm not proposing that we ditch ptmalloc and replace
it with mallopt. I'm proposing that we have both ptmalloc and
mallopt, and the user can choose which to use. And if some upcoming
Linux MMU notifier mechanism will work, we can add it into the bag of
tricks as well; it'll be another option. It's the Open MPI Way. :-)
- doing *something* on Solaris
Implementing the same kind of notifiers in Solaris ?
Brian mentioned that we either don't do anything in Solaris or do even
more horrid things there (I don't remember offhand). I assume that
the Sun guys would want to do something reasonable... :-)
--
Jeff Squyres
Cisco Systems
