On 23/03/2011 02:32, Tim Docker wrote:
On Mon, Mar 21, 2011 at 9:59 AM, I wrote:
My question on the ghc heap profiler on stack overflow:
http://stackoverflow.com/questions/5306717/how-should-i-interpret-the-output-of-the-ghc-heap-profiler
remains unanswered :-( Perhaps that's not the best forum. Is there
someone
here prepared to explain how the memory usage in the heap profiler
relates
to the "Live Bytes" count shown in the garbage collection statistics?
I've made a little progress on this. I've simplified my program down to
a simple executable that loads a bunch of data into an in-memory map,
and then writes it out again. I've added calls to `seq` to ensure that
laziness is not causing excessing memory consumption. When I run this on
my sample data set, it takes ~7 cpu seconds, and uses ~120 MB of vm An
equivalent python script, takes ~2 secs and ~19MB of vm :-(.
The code is below. I'm mostly concerned with the memory usage rather
than performance at this stage. What is interesting, is that when I turn
on garbage collection statistics (+RTS -s), I see this:
10,089,324,996 bytes allocated in the heap
201,018,116 bytes copied during GC
12,153,592 bytes maximum residency (8 sample(s))
59,325,408 bytes maximum slop
114 MB total memory in use (1 MB lost due to fragmentation)
Generation 0: 19226 collections, 0 parallel, 1.59s, 1.64selapsed
Generation 1: 8 collections, 0 parallel, 0.04s, 0.04selapsed
INIT time 0.00s ( 0.00s elapsed)
MUT time 5.84s ( 5.96s elapsed)
GC time 1.63s ( 1.68s elapsed)
EXIT time 0.00s ( 0.00s elapsed)
Total time 7.47s ( 7.64s elapsed)
%GC time 21.8% (22.0% elapsed)
Alloc rate 1,726,702,840 bytes per MUT second
Productivity 78.2% of total user, 76.5% of total elapsed
This seems strange. The maximum residency of 12MB sounds about correct
for my data. But what's with the 59MB of "slop"? According to the ghc docs:
| The "bytes maximum slop" tells you the most space that is ever wasted
| due to the way GHC allocates memory in blocks. Slop is memory at the
| end of a block that was wasted. There's no way to control this; we
| just like to see how much memory is being lost this way.
There's this page also:
http://hackage.haskell.org/trac/ghc/wiki/Commentary/Rts/Storage/Slop
but it doesn't really make things clearer for me.
I think the slop figure might be inaccurate when there are lots of
ByteStrings floating around, due to the way the garbage collector
handles pinned objects (which ByteStrings are). I'll take a look at
this sometime.
Cheers,
Simon
_______________________________________________
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
