Thanks for catching this. I agree with you, the first option seems
like it is closer to what we want to have happen in Open MPI. I'm
less concerned with dropping a couple of pages of memory as I am with
searching our registration cache more often. It shall be committed
to SVN today.
Brian
On Jan 18, 2006, at 9:54 AM, Gleb Natapov wrote:
Hello,
I found one more problem with ptmalloc and registration cache.
In arena.c:grow_heap() when heap is shrinking ptmalloc tries to be
smart
and is using mmap() to change pages protection instead of mprotect
() because
as a side effect mmap() drops underlying pages. In the case the
area is
registered we cannot drop pages without notifying registration cache.
I see two solutions either change mmap() to mprotect() or call
opal_mem_hooks_release_hook() after mmap() to remove the area from
cache. I think first approach is better but both patch are included
for
your consideration :)
Index: opal/mca/memory/ptmalloc2/arena.c
===================================================================
--- opal/mca/memory/ptmalloc2/arena.c (revision 8727)
+++ opal/mca/memory/ptmalloc2/arena.c (working copy)
@@ -614,10 +614,8 @@
new_size = (long)h->size + diff;
if(new_size < (long)sizeof(*h))
return -1;
- /* Try to re-map the extra heap space freshly to save memory, and
- make it inaccessible. */
- if((char *)MMAP((char *)h + new_size, -diff, PROT_NONE,
- MAP_PRIVATE|MAP_FIXED) == (char *) MAP_FAILED)
+
+ if(mprotect((char *)h + new_size, -diff, PROT_NONE) != 0)
return -2;
/*fprintf(stderr, "shrink %p %08lx\n", h, new_size);*/
}
Index: opal/mca/memory/ptmalloc2/arena.c
===================================================================
--- opal/mca/memory/ptmalloc2/arena.c (revision 8727)
+++ opal/mca/memory/ptmalloc2/arena.c (working copy)
@@ -619,6 +619,8 @@
if((char *)MMAP((char *)h + new_size, -diff, PROT_NONE,
MAP_PRIVATE|MAP_FIXED) == (char *) MAP_FAILED)
return -2;
+
+ opal_mem_hooks_release_hook ((char *)h + new_size, -diff, 1);
/*fprintf(stderr, "shrink %p %08lx\n", h, new_size);*/
}
h->size = new_size;
--
Gleb.
_______________________________________________
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel
--
Brian Barrett
Open MPI developer
http://www.open-mpi.org/
