#698: GHC's internal memory allocator never releases memory back to the OS
---------------------------------+------------------------------------------
Reporter: guest | Owner: igloo
Type: bug | Status: new
Priority: low | Milestone: 6.12 branch
Component: Runtime System | Version: 6.4.1
Severity: normal | Resolution:
Keywords: | Difficulty: Moderate (1 day)
Testcase: N/A | Os: Unknown/Multiple
Architecture: Unknown/Multiple |
---------------------------------+------------------------------------------
Comment (by simonmar):
Replying to [comment:17 crutcher]:
> I want to work on this.
>
> It seems that there's little agreement on what the 'right' behavior is,
because there are many different execution models for which different
behaviors are preferable. It seems we need a means of setting a memory
reclamation policy, and plugging in some number of implementations of that
policy, with flags to set it.
>
> Off the top of my head, I see a few obvious ones:
> * Never return free memory (the current behavior)
> * Immediately return free memory (the notional behavior)
> * Return outstanding free memory on 'flush' events (nice for the dll
case?)
> * Fixed Buffer - return free memory over X, for some buffer size X.
> * Ratio Buffer - return free memory over R, for some ratio of used
memory.
>
> And there's this one, which I'd like to be able to play with, but has
numerous knobs.
> * Derivative Ratio Buffer - at time t, estimate the derivative D(t)
of memory use, and return free memory over R*D(t+h) for some ration R and
time step h.
Bear in mind that the actual memory requirements fluctuate over time due
to GC activity. When the copying GC is being used, at a major GC we
require F*L0+L1 memory, where L0 is the amount of live data at the last
GC, L1 is the current live data, and F is the value set by `+RTS -F`
(default 2). In practice we'll need a little bit more than this, because
we GC the cycle after the limit has been reached.
So I suggest that after a major GC we
* estimate the amount of memory required at the next GC, assuming live
data
remains constant, call this M, and add a constant C
* release any whole megablocks over this limit
Provide a way to set C, and/or define it as a fraction of M. I imagine
that C == M would be a reasonable default: keep double the current
requirements around just in case. People who want to be frugal with
memory could set C == M/3. Programs with wildly varying memory
requirements will suffer a performance hit if C is too low.
Memory could be released between GCs, but the live data value can only be
calculated at a major GC, so it makes most sense to release memory at a
major GC. Programs compiled with `-threaded` get an automatic major GC
when they're idle (idle time set by `+RTS -I`), programs compiled without
`-threaded` will have to call `System.Mem.performGC` to release memory if
they intend to go idle.
--
Ticket URL: <http://hackage.haskell.org/trac/ghc/ticket/698#comment:22>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler_______________________________________________
Glasgow-haskell-bugs mailing list
[email protected]
http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs
