On 05.10.2007 [10:12:34 +1000], David Gibson wrote: > On Thu, Oct 04, 2007 at 05:06:15PM -0700, Nishanth Aravamudan wrote: > > On 05.10.2007 [09:37:06 +1000], David Gibson wrote: > > > On Wed, Oct 03, 2007 at 09:43:39AM -0500, Mike Kistler wrote: > > > > Thanks for the info. Unfortunately, I already have the code to > > > > open and mmap a file in the hugetlbfs. What I am looking for is an > > > > allocator that will go on top of this with an explicit interface, > > > > like huge_malloc(bytes). > > > > > > Yeah, for that you'd need to look at some allocator library with a > > > changable backend. Heap Layers > > > (http://www.cs.umass.edu/~emery/heaplayers.html) or Hoard > > > (http://www.cs.umass.edu/~emery/hoard/index.html) perhaps? > > > > Indeed, see below, but we specifically do not provide this > > functionality (an interface into hugepage malloc). > > > > > > My concern with sending *all* my mallocs to huge pages is that my > > > > app has many mallocs will have no benefit from being satisfied from > > > > large pages, and these could wind up fragmenting the large page > > > > storage area to the point that the structures that really do need to > > > > be in large pages won't fit there. > > > > > > That doesn't sound all that likely, if put enough hugepages into the > > > pool. Worth a shot, surely, before deciding it won't work.. > > > > Just to clarify, libhugetlbfs doesn't technically override malloc() > > itself. Or even provide an alternative malloc(). It simply notifies > > glibc (via a function pointer) that we provide a source of memory for > > dynamic allocations. So rather than going into arenas and all that, > > glibc simply asks us for memory. From what I've seen, at least, this > > means that as long as the so-far-allocated dynamic area is large enough, > > glibc won't ask us for more hugepages to use. When the area needs to be > > grown, we grow the hugetlb file that backs the heap. malloc() and brk() > > transparently work the same way as before just the source of memory is > > different. > > Careful, malloc() will transparently be hugepage backed, brk() is a > kernel call and will not. brk() will either return normal pages, or > simply fail, depending on linker layout and existing hugepage mappings > (and arhcitecture).
I knew once I wrote a paragraph, I'd make a mistake. You're very right, David. Thanks, Nish -- Nishanth Aravamudan <[EMAIL PROTECTED]> IBM Linux Technology Center ------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/ _______________________________________________ Libhugetlbfs-devel mailing list Libhugetlbfs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libhugetlbfs-devel
