Send Beginners mailing list submissions to
[email protected]
To subscribe or unsubscribe via the World Wide Web, visit
http://www.haskell.org/mailman/listinfo/beginners
or, via email, send a message with subject or body 'help' to
[email protected]
You can reach the person managing the list at
[email protected]
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Beginners digest..."
Today's Topics:
1. Re: A few really short beginners questions (Klaus Gy)
2. Flying Dutchman sailing blind (Jeroen van Maanen)
3. Re: Flying Dutchman sailing blind (Daniel Fischer)
4. Re: Flying Dutchman sailing blind (Stephen Tetley)
----------------------------------------------------------------------
Message: 1
Date: Mon, 4 Oct 2010 22:38:01 +0200
From: Klaus Gy <[email protected]>
Subject: Re: [Haskell-beginners] A few really short beginners
questions
To: beginners <[email protected]>
Message-ID:
<[email protected]>
Content-Type: text/plain; charset=ISO-8859-1
Thanks again for all the accurate answers! -fweth
------------------------------
Message: 2
Date: Tue, 05 Oct 2010 10:06:03 +0200
From: Jeroen van Maanen <[email protected]>
Subject: [Haskell-beginners] Flying Dutchman sailing blind
To: [email protected]
Message-ID: <[email protected]>
Content-Type: text/plain; charset=ISO-8859-1
A few weeks ago I was greatly helped by members of this list to expose an error
that was hiding in my, by now quite extensive, Haskell program. I should still
thank Daniel Fisher for his lucid explanation of how an overdose of strictness
can prevent a try-expression from wrapping an exception.
Now that the error is gone, I face a new challenge: when I let my program run
for a while (it is a computationally intensive task that never stops by
design), after fifteen minutes or so the CPU usage drops from nearly 100% to
around 15% and a few minutes later the process dies with the message:
"lexau: memory allocation failed (requested 2097152 bytes)"
The whole thing is a tail biting snail pit of threads that are communicating
through MVars. Every thread runs a tail recursive function. (I read in another
post that it is not a good idea to use explicit recursion, but when I compared
alternatives the fold variants produced even worse stack/heap problems.) The
thread that is likely to cause the problem is an optimizer that tries many
possible improvements of a complex data structure and incrementally applies the
successful ones. I use a strict foldl style in an attempt to limit the memory
used by the optimizer. Frankly, however, I have no idea what eats up so much
heap space.
Now I have always proudly labeled myself a 'printf programmer', but I am afraid
that I am going to need some profiling tool to determine where the problem is.
Any suggestions where I should start?
Cheers, Jeroen
P.S. For an idea of what is living in the snail pit, have a look at:
http://lexau.svn.sourceforge.net/viewvc/lexau/branches/totem/src/LExAu/Pipeline/Concurrent.hs?view=markup
http://lexau.svn.sourceforge.net/viewvc/lexau/branches/totem/src/LExAu/Model/HistoryTree.hs?view=markup
(I know, HistoryTree.hs badly needs to be split up into smaller modules.)
------------------------------
Message: 3
Date: Tue, 5 Oct 2010 12:07:56 +0200
From: Daniel Fischer <[email protected]>
Subject: Re: [Haskell-beginners] Flying Dutchman sailing blind
To: [email protected]
Cc: Jeroen van Maanen <[email protected]>
Message-ID: <[email protected]>
Content-Type: text/plain; charset="iso-8859-1"
On Tuesday 05 October 2010 10:06:03, Jeroen van Maanen wrote:
> A few weeks ago I was greatly helped by members of this list to expose
> an error that was hiding in my, by now quite extensive, Haskell program.
> I should still thank Daniel Fisher for his lucid explanation of how an
> overdose of strictness can prevent a try-expression from wrapping an
> exception.
>
> Now that the error is gone, I face a new challenge: when I let my
> program run for a while (it is a computationally intensive task that
> never stops by design), after fifteen minutes or so the CPU usage drops
> from nearly 100% to around 15% and a few minutes later the process dies
> with the message:
>
> "lexau: memory allocation failed (requested 2097152 bytes)"
>
> The whole thing is a tail biting snail pit of threads that are
> communicating through MVars. Every thread runs a tail recursive
> function. (I read in another post that it is not a good idea to use
> explicit recursion, but when I compared alternatives the fold variants
> produced even worse stack/heap problems.)
That depends. Using a combinator (folds etc.) gives you more elegant, more
general and usually more readable code.
Using a low-level direct recursion however gives you more control over
strictness, speed and allocation behaviour.
Sometimes you have to stick your hands in the low-level details to get
adequate performance.
> The thread that is likely to
> cause the problem is an optimizer that tries many possible improvements
> of a complex data structure and incrementally applies the successful
> ones. I use a strict foldl style in an attempt to limit the memory used
> by the optimizer.
May be not strict enough (or too strict). What's the suspect function?
> Frankly, however, I have no idea what eats up so much heap space.
>
> Now I have always proudly labeled myself a 'printf programmer', but I am
> afraid that I am going to need some profiling tool to determine where
> the problem is. Any suggestions where I should start?
As a very first measure, run your programme with the "-hT" RTS-option (
$ ./lexau +RTS -hT -RTS args
-- wait until it dies or ctrl-C it when CPU usage has dropped
$ hp2ps -c lexau.hp
-- look at the graph in lexau.ps
).
If that doesn't reveal the source of the problem, compile for profiling,
$ ghc -O2 -prof -auto-all -osuf p_o -hisuf p_hi --make lexau -o profLexau
and run with some heap profiling options,
http://darcs.haskell.org/download/docs/6.12.3/html/users_guide/profiling.html
explains how.
>
> Cheers, Jeroen
>
> P.S. For an idea of what is living in the snail pit, have a look at:
>
>
> http://lexau.svn.sourceforge.net/viewvc/lexau/branches/totem/src/LExAu/P
>ipeline/Concurrent.hs?view=markup
> http://lexau.svn.sourceforge.net/viewvc/lexau/branches/totem/src/LExAu/M
>odel/HistoryTree.hs?view=markup
Ewww.
>
> (I know, HistoryTree.hs badly needs to be split up into smaller
> modules.)
+ 5
------------------------------
Message: 4
Date: Tue, 5 Oct 2010 12:46:28 +0100
From: Stephen Tetley <[email protected]>
Subject: Re: [Haskell-beginners] Flying Dutchman sailing blind
Cc: [email protected]
Message-ID:
<[email protected]>
Content-Type: text/plain; charset=ISO-8859-1
On 5 October 2010 09:06, Jeroen van Maanen <[email protected]> wrote:
>
> P.S. For an idea of what is living in the snail pit, have a look at:
>
> http://lexau.svn.sourceforge.net/viewvc/lexau/branches/totem/src/LExAu/Pipeline/Concurrent.hs?view=markup
> http://lexau.svn.sourceforge.net/viewvc/lexau/branches/totem/src/LExAu/Model/HistoryTree.hs?view=markup
>
> (I know, HistoryTree.hs badly needs to be split up into smaller modules.)
A minor style tip...
You are using ShowS family functions (showString, showChar) to
generate strings which is good as it avoids (++). However, to make the
code clearer you might want to code up a little helper library with
ShowS versions of the usual pretty print combinators.
e.g. all the functions like parens:
parens :: ShowS -> ShowS
parens s = showChar '(' . s . showChar ')'
Having a library of these functions usually pays off (I think there is
one on Hackage but I can't remember its name). You could use a pretty
printing library, but they are somewhat less efficient as the have to
do work measuring line lengths for fitting lines to screen width.
Best wishes
Stephen
------------------------------
_______________________________________________
Beginners mailing list
[email protected]
http://www.haskell.org/mailman/listinfo/beginners
End of Beginners Digest, Vol 28, Issue 14
*****************************************
