On 05.10.2007 [09:37:06 +1000], David Gibson wrote: > On Wed, Oct 03, 2007 at 09:43:39AM -0500, Mike Kistler wrote: > > Thanks for the info. Unfortunately, I already have the code to > > open and mmap a file in the hugetlbfs. What I am looking for is an > > allocator that will go on top of this with an explicit interface, > > like huge_malloc(bytes). > > Yeah, for that you'd need to look at some allocator library with a > changable backend. Heap Layers > (http://www.cs.umass.edu/~emery/heaplayers.html) or Hoard > (http://www.cs.umass.edu/~emery/hoard/index.html) perhaps?
Indeed, see below, but we specifically do not provide this functionality (an interface into hugepage malloc). > > My concern with sending *all* my mallocs to huge pages is that my > > app has many mallocs will have no benefit from being satisfied from > > large pages, and these could wind up fragmenting the large page > > storage area to the point that the structures that really do need to > > be in large pages won't fit there. > > That doesn't sound all that likely, if put enough hugepages into the > pool. Worth a shot, surely, before deciding it won't work.. Just to clarify, libhugetlbfs doesn't technically override malloc() itself. Or even provide an alternative malloc(). It simply notifies glibc (via a function pointer) that we provide a source of memory for dynamic allocations. So rather than going into arenas and all that, glibc simply asks us for memory. From what I've seen, at least, this means that as long as the so-far-allocated dynamic area is large enough, glibc won't ask us for more hugepages to use. When the area needs to be grown, we grow the hugetlb file that backs the heap. malloc() and brk() transparently work the same way as before just the source of memory is different. To be honest, I'm not sure about memory fragmentation (as to how glibc decides to place things in the memory given to it by __morecore) with HUGETLB_MORECORE. But, as David said, I'd try it out. Thanks, Nish -- Nishanth Aravamudan <[EMAIL PROTECTED]> IBM Linux Technology Center ------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/ _______________________________________________ Libhugetlbfs-devel mailing list Libhugetlbfs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libhugetlbfs-devel
