On May 6, 2013, at 6:46 AM, Abhishek Singh wrote:
> On Sat, May 4, 2013 at 10:14 PM, Jason Evans <[email protected]> wrote:
> On Apr 28, 2013, at 9:03 PM, Abhishek Singh <[email protected]>
> wrote:
> > Our setups start typically with 96 GB of RAM and up. We have observed that
> > using jemalloc the virtual memory usage of our process rises up to around
> > 75GB. While the resident memory stays low and it is not a problem as such,
> > when we try to fork a process from within here, it fails as the kernel
> > assumes there is not enough memory to copy the VM space. Perhaps a vfork
> > would be better but we can't use that for now.
> >
> > So we have made some modifications to jemalloc so that all huge memory
> > allocations are forced to unmap their memory on freeing up. Non huge memory
> > allocations and freeing up remain the same. This seems to help us. I have
> > attached the patch here which is against jemalloc-3.3.1. Please review and
> > suggest if there is a better way to handle this.
>
> Does --enable-munmap give you similar results? Ideally, munmap() would
> always be enabled, but Linux has some unfortunate VM map fragmentation issues
> (it doesn't consistently reuse holes left by munmap()). I don't think
> there's much benefit to using munmap() only selectively, because avoiding the
> VM map fragmentation issue is an all-or-nothing proposition.
>
> Yes, results with --enable-munmap are similar but some preliminary benchmarks
> for the kind of allocations we do, seem to indicate degradation in
> performance as compared to unmap disabled. While you are right and ideally it
> would be great to unmap all the memory as and when it is not needed, if there
> is a performance tradeoff then it may not be feasible. Our aim as of now is
> to ensure that we cause minimal memory fragmentation (thereby reducing our
> virtual memory footprint) and still get as good a performance as we can.
> Selectively unmapping only huge allocations seemed to be a compromise between
> the two extremes.
Does your application use fork() just to call exec()? If so, you can pre-fork
one or more helper processes (and communicate with them via pipes) early during
execution that act as fork()/exec() helpers. By creating them early on, you
assure that they have a much smaller memory footprint than the huge main
process, which will improve both speed and reliability.
Jason
_______________________________________________
jemalloc-discuss mailing list
[email protected]
http://www.canonware.com/mailman/listinfo/jemalloc-discuss
