Re: [Haskell-cafe] Is it possible to represent such polymorphism?
On 04/10/2011 07:08 AM, Dominique Devriese wrote: All, In case anyone is interested, I just want to point out an interesting article about the relation between Haskell type classes and C++ (overloading + concepts): http://sms.cs.chalmers.se/publications/papers/2008-WGP.pdf Dominique Thanks for that. A very interesting read... ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Haddocok in Haskell Platform 2011.2.0.1?
Hi, I just installed a new computer, downloaded ghc 7.0.3 and HP 2011.2.0.1 from http://hackage.haskell.org/platform/linux.html as proposed there, and I can not install haddock. It says it requires ghc =7.2 and =7.4. Is that intended? Cheers, Christian ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Haddocok in Haskell Platform 2011.2.0.1?
On 5 October 2011 18:09, . ch.go...@googlemail.com wrote: Hi, I just installed a new computer, downloaded ghc 7.0.3 and HP 2011.2.0.1 from http://hackage.haskell.org/platform/linux.html as proposed there, and I can not install haddock. It says it requires ghc =7.2 and =7.4. Is that intended? I believe Haddock uses GHC to do the parsing of Haskell source files, so you'll probably need to get an older version of Haddock (as newer ones seem to be aimed explicitly at GHC-7.2); try 2.9.3 or 2.9.2. -- Ivan Lazar Miljenovic ivan.miljeno...@gmail.com IvanMiljenovic.wordpress.com ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Haddocok in Haskell Platform 2011.2.0.1?
Thanks for the hint! It works with 2.9.2, I was just wondering .. if someone does a cabal update and e.g. a cabal install leksah, it wants to install haddock 2.9.4 (at least that was the case on my computer). However, that does not work, so unless you install 2.9.2 by hand first, you can not use the package that depends on haddock (again, e.g. leksah). On Wed, 2011-10-05 at 19:00 +1100, Ivan Lazar Miljenovic wrote: On 5 October 2011 18:09, . ch.go...@googlemail.com wrote: Hi, I just installed a new computer, downloaded ghc 7.0.3 and HP 2011.2.0.1 from http://hackage.haskell.org/platform/linux.html as proposed there, and I can not install haddock. It says it requires ghc =7.2 and =7.4. Is that intended? I believe Haddock uses GHC to do the parsing of Haskell source files, so you'll probably need to get an older version of Haddock (as newer ones seem to be aimed explicitly at GHC-7.2); try 2.9.3 or 2.9.2. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Is it possible to represent such polymorphism?
If a newbie considers this as something natural, this is another reason for syntactic sugaring of HList: http://www.haskell.org/pipermail/haskell-cafe/2011-April/090986.html 2011/10/2 Du Xi sdiy...@sjtu.edu.cn --I tried to write such polymorphic function: expand (x,y,z) = (x,y,z) expand (x,y) = (x,y,1) --And it didn't compile. Then I added a type signature: expand::a-b expand (x,y,z) = (x,y,z) expand (x,y) = (x,y,1) --It still didn't compile. I think the reason is that the following is disallowed: f::a-b f x = x --Is it possible to get around this and write the expand function? Of course, x and y may be of different types __**_ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/**mailman/listinfo/haskell-cafehttp://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Is it possible to represent such polymorphism?
On Wed, Oct 5, 2011 at 8:45 AM, Alberto G. Corona agocor...@gmail.com wrote: If a newbie considers this as something natural, this is another reason for syntactic sugaring of HList: http://www.haskell.org/pipermail/haskell-cafe/2011-April/090986.html Exposing newbies to HList seems like a recipe for disaster for me =). -- Felipe. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Munich Haskell Meeting
Dear all, last Thursday's get-together for Haskell users in Munich was a success. About 12 people meet at Cafe Puck where we spent a nice evening. The overall opinion was, that gatherings of this kind should be held on a more regular basis. Therefore, I'd like to announce the following page: http://www.haskell-munich.de If you can make it to Munich, please feel invited to join us at one of the given dates. We would highly appreciate your company. If you have suggestions about the page or anything else, please contact me. Have a nice day, Heinrich -- -- kont...@funktional.info www.funktional.info -- ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Fwd: Is it possible to represent such polymorphism?
if Hlist is sugarized as variable length tuples, then the initial code would compile without noticing the use of HList... 2011/10/5 Felipe Almeida Lessa felipe.le...@gmail.com On Wed, Oct 5, 2011 at 8:45 AM, Alberto G. Corona agocor...@gmail.com wrote: If a newbie considers this as something natural, this is another reason for syntactic sugaring of HList: http://www.haskell.org/pipermail/haskell-cafe/2011-April/090986.html Exposing newbies to HList seems like a recipe for disaster for me =). -- Felipe. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] categories in Haskell and module names
Hello. IMHO there are 2 ways to define categories in Haskell: W0) the category of types and functions (base.Data.Functor and company belongs to it); W1) the class base.Control.Category.Category. (Defining a category where the class of objects is a type seems impossible in Haskell.) But these ways are mingled in base. E.g. Control.Applicative (W0) and Control.Category (W1) are neighbors. Oddly, Data.Functor (W0) is under Data. I believe that the whole library category-extras belongs to W1. I suggest to split these ways under distinct names, e.g.: W0) Category.Function (@hom == (-)@); W1) Category.Hom (hom is an abstract type constructor). Maybe both should be under Mathematics. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Really Simple explanation of Continuations Needed
On the general notion of continuations, I believe Matt Might's blog explains
it quite well using Javascript.
http://matt.might.net/articles/by-example-continuation-passing-style/
In the way of a simple example, he suggests that instead of writing
function id(x) {
return x ;
}
a CPS version might write:
function id(x,ret) {
ret(x) ;
}
etc...
IMO things appear confusing to newbies (it happened to me once too) when
people dont use intuitive names for obvious things like continuations
On Fri, Sep 30, 2011 at 11:42 PM, Mark Spezzano
mark.spezz...@chariot.net.au wrote:
Hi,
Can someone please give me a _lucid_ and _simple_ explanation of exactly
how continuations can be used in Haskell?
I've already had a look at most of the tutorials and explanations on the
web, but I'm still confused. Continuations and CPS have me baffled. (I have
most of the Haskell textbooks and even these are sketchy on Continuations)
I don't understand the notion of the Cont monad and how it can be used for
multitasking, backtracking and interrupting computations. I understand that
functions take in a (continuation) function that represents the work
remaining to do, but al of the explanations on the web and in technical
papers seems to trip over themselves in explaining the fundamentals to a
CPS-newbie.
If anyone could explain such concepts to me in unambiguous, clear English
then this would be very helpful.
Thanks in advance for your help,
Mark
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] what happens to ()'s from Core?
Hello,
I'm working on a small EDSL, and I think I've finally managed to get
GHC to compile it to good core. Basically, it allows for the creation
of expressions like:
g = 0.5*x + 0.1*y
which is then compiled to a tuple (related work: CCA, stream fusion)
exists s. (s, s - Double - (s,Double))
I also have a function 'mapAccumL :: (V.Unbox a, V.Unbox b) = (s - a
- (s,b)) - s - V.Vector a - V.Vector b'. Basic usage would be
similar to:
import qualified Data.Vector.Unboxed as V
main = do
let (gs, gf) = $(compile [] g)
ys = mapAccumL gf gs $ V.enumFromTo (1::Double) 5
print ys
For 'g' as above, I currently get 's :: (((), ()), Double)', which is
expected. GHC produces the following core for the inner loop, which
looks pretty good to me:
letrec {
$s$wa_s2OL [Occ=LoopBreaker]
:: ()
- ()
- GHC.Prim.Double#
- GHC.Prim.Int#
- GHC.Prim.State# (Control.Monad.Primitive.R:PrimStateST s_a1Y9)
- (# GHC.Prim.State# s_a1Y9, () #)
[LclId, Arity=5, Str=DmdType L]
$s$wa_s2OL =
\ _
_
(sc2_s2Oq :: GHC.Prim.Double#)
(sc3_s2Or :: GHC.Prim.Int#)
(sc4_s2Os
:: GHC.Prim.State#
(Control.Monad.Primitive.R:PrimStateST s_a1Y9)) -
case GHC.Prim.# sc3_s2Or rb1_a2EV of _ {
GHC.Types.False - (# sc4_s2Os, GHC.Unit.() #);
GHC.Types.True -
let {
x#_a2aI [Dmd=Just L] :: GHC.Prim.Double#
[LclId, Str=DmdType]
x#_a2aI =
GHC.Prim.+##
(GHC.Prim.*##
(GHC.Prim.indexDoubleArray#
rb2_a2EW (GHC.Prim.+# rb_a2EU sc3_s2Or))
0.5)
(GHC.Prim.*## sc2_s2Oq 0.1) } in
$s$wa_s2OL
GHC.Unit.()
GHC.Unit.()
x#_a2aI
(GHC.Prim.+# sc3_s2Or 1)
((GHC.Prim.writeDoubleArray#
@ (Control.Monad.Primitive.PrimState (GHC.ST.ST s_a1Y9))
arr#_a29n
sc3_s2Or
x#_a2aI
(sc4_s2Os
`cast` (GHC.Prim.State#
(Sym
(Control.Monad.Primitive.TFCo:R:PrimStateST s_a1Y9))
:: GHC.Prim.State#
(Control.Monad.Primitive.R:PrimStateST s_a1Y9)
~
GHC.Prim.State#
(Control.Monad.Primitive.PrimState
(GHC.ST.ST s_a1Y9)
`cast` (GHC.Prim.State#
(Control.Monad.Primitive.TFCo:R:PrimStateST s_a1Y9)
:: GHC.Prim.State#
(Control.Monad.Primitive.PrimState
(GHC.ST.ST s_a1Y9))
~
GHC.Prim.State#
(Control.Monad.Primitive.R:PrimStateST s_a1Y9)))
}; } in
So my question is, what happens to the ()'s after this stage? Since
they're not used, and also expressed as literals in core (both in the
recursive case and the original call site of $s$wa_s2OL, is the
backend smart enough to get rid of them completely?
Thanks for any advice,
John L.
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Cabal rebuilding all of the C++ code for wxHaskell
On 30 September 2011 03:02, Claude Heiland-Allen cla...@goto10.org wrote: On 30/09/11 02:45, DukeDave wrote: 1. Is there some reason (other than 'safety') that cabal install cleans everything up? As far as I've experienced and understand it, it doesn't - it's more that GHC can detect when Haskell modules don't need recompiling while the same is not true for C or C++ sources. For example, I change one module and see GHC report only that module and its dependents being recompiled, while the other compiled modules are reused from previous 'cabal install' runs. The C-sources: are recompiled every time even if unchanged, which I too find it somewhat annoying even with my small projects. Excellent, that is consistent with what I'm seeing, and I'm glad I'm not the only one who finds it annoying. I have no familiarity with how cabal and GHC handle C-sources, but I presume that the job of building them is handed off to a local C/C++ compiler (I presume g++ in my case). Given this I can only assume that cabal is doing something: 1. Deleting the object files before calling the C compiler (and so everything must be rebuilt)? 2. Touching the source C files in some way, before calling the C compiler? 3. Passing some argument to the compiler which is telling it to rebuild everything? 4. Something else? 2. Why does setting cleanHook to return () not have any effect? I think becausae the clean hook is probably not called by 'cabal install', but by 'cabal clean'. Ah yes, that does make sense, my bad. Claude ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] SMP parallelism increasing GC time dramatically
I am having some strange performance issues when using SMP parallelism, that I think may be something to do with GC. Apologies for the large readouts below but I'm not familiar enough to know what is and isn't relevant! I have a pure function that is mapped over a list of around 10 values, and this happens several times for each iteration of my program. It does some fairly intensive calculations using hmatrix, generating intermediate matrices along the way. The computation is significantly more complex for some values, so the work done by each call is not spread equally. I did some profiling and it seems like the program is spending about 50% of its time in that function. First of all, without any attempts at parallelism, I see this from ./Main +RTS -s 67,142,126,336 bytes allocated in the heap 147,759,264 bytes copied during GC 109,384 bytes maximum residency (58 sample(s)) 354,408 bytes maximum slop 3 MB total memory in use (0 MB lost due to fragmentation) Generation 0: 104551 collections, 0 parallel, 1.13s, 1.11s elapsed Generation 1:58 collections, 0 parallel, 0.01s, 0.01s elapsed Parallel GC work balance: -nan (0 / 0, ideal 1) MUT time (elapsed) GC time (elapsed) Task 0 (worker) :0.00s( 67.06s) 0.00s( 0.00s) Task 1 (worker) :0.00s( 67.09s) 0.00s( 0.00s) Task 2 (bound) : 66.95s( 67.09s) 1.14s( 1.12s) SPARKS: 0 (0 converted, 0 pruned) INIT time0.00s ( 0.00s elapsed) MUT time 66.95s ( 67.09s elapsed) GCtime1.14s ( 1.12s elapsed) EXIT time0.00s ( 0.00s elapsed) Total time 68.09s ( 68.21s elapsed) %GC time 1.7% (1.6% elapsed) Alloc rate1,002,835,517 bytes per MUT second Productivity 98.3% of total user, 98.2% of total elapsed gc_alloc_block_sync: 0 whitehole_spin: 0 gen[0].sync_large_objects: 0 gen[1].sync_large_objects: 0 This looks ok to me... Then if I try to use Control.Parallel to parallelise my code, simpy replacing a map with parMap (rdeepseq), on a 12 core machine using +RTS -N12 -s I get this: 66,065,148,144 bytes allocated in the heap 197,202,056 bytes copied during GC 181,312 bytes maximum residency (251 sample(s)) 387,240 bytes maximum slop 12 MB total memory in use (3 MB lost due to fragmentation) Generation 0: 37592 collections, 37591 parallel, 245.32s, 26.67s elapsed Generation 1: 251 collections, 251 parallel, 3.12s, 0.33s elapsed Parallel GC work balance: 2.41 (24219609 / 10058220, ideal 12) MUT time (elapsed) GC time (elapsed) Task 0 (worker) :0.00s( 0.00s) 0.00s( 0.00s) Task 1 (worker) :0.00s( 0.00s) 0.00s( 0.00s) Task 2 (worker) :0.00s( 17.97s) 0.00s( 0.00s) Task 3 (worker) :0.00s( 19.35s) 0.00s( 0.00s) Task 4 (worker) :0.00s( 40.28s) 0.00s( 0.00s) Task 5 (worker) :0.00s( 45.08s) 0.00s( 0.00s) Task 6 (worker) :0.00s( 47.06s) 0.00s( 0.00s) Task 7 (worker) : 18.30s( 49.73s) 16.24s( 1.71s) Task 8 (worker) :0.00s( 51.22s) 0.00s( 0.00s) Task 9 (worker) :0.00s( 53.75s) 0.00s( 0.00s) Task 10 (worker) :0.00s( 54.17s) 0.00s( 0.00s) Task 11 (worker) :5.65s( 54.30s) 0.70s( 0.08s) Task 12 (worker) :0.00s( 54.41s) 0.41s( 0.04s) Task 13 (worker) :4.34s( 54.58s) 4.50s( 0.48s) Task 14 (worker) :5.82s( 54.76s) 5.91s( 0.64s) Task 15 (worker) :6.50s( 55.01s) 3.37s( 0.38s) Task 16 (worker) :7.60s( 55.21s) 8.56s( 0.94s) Task 17 (worker) : 11.05s( 55.21s) 9.04s( 0.96s) Task 18 (worker) : 11.75s( 55.21s) 12.94s( 1.43s) Task 19 (worker) : 16.02s( 55.21s) 13.32s( 1.43s) Task 20 (worker) : 26.98s( 55.23s) 7.43s( 0.77s) Task 21 (worker) :7.36s( 55.23s) 7.47s( 0.83s) Task 22 (worker) : 16.08s( 55.23s) 10.25s( 1.12s) Task 23 (worker) :7.04s( 55.23s) 4.99s( 0.57s) Task 24 (worker) : 28.47s( 55.23s) 8.78s( 0.94s) Task 25 (worker) :7.43s( 55.23s) 1.62s( 0.18s) Task 26 (worker) :6.33s( 55.23s) 11.42s( 1.23s) Task 27 (worker) :9.80s( 55.23s) 8.72s( 0.95s) Task 28 (worker) :4.88s( 55.26s) 8.92s( 0.99s) Task 29 (worker) :0.00s( 55.26s) 0.00s( 0.00s) Task 30 (bound) :5.59s( 55.26s) 0.59s( 0.06s) Task 31 (worker) : 41.16s( 55.26s) 3.48s( 0.38s) Task 32 (worker) : 17.03s( 55.26s) 3.90s( 0.42s) Task 33 (worker) : 14.89s( 55.26s)
Re: [Haskell-cafe] SMP parallelism increasing GC time dramatically
Hi Tom, I think debugging this sort of problem is exactly what we need to be doing (and making easier). Have you tried Duncan's newest version of Threadscope by the way? It looks like -- completely aside from the GC time -- this program is not scaling. The mutator time itself, disregarding GC, isn't going down much with parallelism (with the total mutator time increasing drastically). Either this is completely memory bottlenecked or there is some other kind of bad interaction (e.g. false sharing, contention on a hot lock, etc). My inclination would be to figure this out first before worrying about the GC time. Is this code that you would be able to share for debugging? I think we need to get together some general documentation on how to debug this kind of problem. For example, you can get some hints as to the memory behavior by running valgrind/cachegrind on the program. Also, what does top say, by the way? Is the process using 1200% CPU? Cheers, -Ryan On Wed, Oct 5, 2011 at 2:15 PM, Tom Thorne thomas.thorn...@gmail.comwrote: I am having some strange performance issues when using SMP parallelism, that I think may be something to do with GC. Apologies for the large readouts below but I'm not familiar enough to know what is and isn't relevant! I have a pure function that is mapped over a list of around 10 values, and this happens several times for each iteration of my program. It does some fairly intensive calculations using hmatrix, generating intermediate matrices along the way. The computation is significantly more complex for some values, so the work done by each call is not spread equally. I did some profiling and it seems like the program is spending about 50% of its time in that function. First of all, without any attempts at parallelism, I see this from ./Main +RTS -s 67,142,126,336 bytes allocated in the heap 147,759,264 bytes copied during GC 109,384 bytes maximum residency (58 sample(s)) 354,408 bytes maximum slop 3 MB total memory in use (0 MB lost due to fragmentation) Generation 0: 104551 collections, 0 parallel, 1.13s, 1.11s elapsed Generation 1:58 collections, 0 parallel, 0.01s, 0.01s elapsed Parallel GC work balance: -nan (0 / 0, ideal 1) MUT time (elapsed) GC time (elapsed) Task 0 (worker) :0.00s( 67.06s) 0.00s( 0.00s) Task 1 (worker) :0.00s( 67.09s) 0.00s( 0.00s) Task 2 (bound) : 66.95s( 67.09s) 1.14s( 1.12s) SPARKS: 0 (0 converted, 0 pruned) INIT time0.00s ( 0.00s elapsed) MUT time 66.95s ( 67.09s elapsed) GCtime1.14s ( 1.12s elapsed) EXIT time0.00s ( 0.00s elapsed) Total time 68.09s ( 68.21s elapsed) %GC time 1.7% (1.6% elapsed) Alloc rate1,002,835,517 bytes per MUT second Productivity 98.3% of total user, 98.2% of total elapsed gc_alloc_block_sync: 0 whitehole_spin: 0 gen[0].sync_large_objects: 0 gen[1].sync_large_objects: 0 This looks ok to me... Then if I try to use Control.Parallel to parallelise my code, simpy replacing a map with parMap (rdeepseq), on a 12 core machine using +RTS -N12 -s I get this: 66,065,148,144 bytes allocated in the heap 197,202,056 bytes copied during GC 181,312 bytes maximum residency (251 sample(s)) 387,240 bytes maximum slop 12 MB total memory in use (3 MB lost due to fragmentation) Generation 0: 37592 collections, 37591 parallel, 245.32s, 26.67s elapsed Generation 1: 251 collections, 251 parallel, 3.12s, 0.33s elapsed Parallel GC work balance: 2.41 (24219609 / 10058220, ideal 12) MUT time (elapsed) GC time (elapsed) Task 0 (worker) :0.00s( 0.00s) 0.00s( 0.00s) Task 1 (worker) :0.00s( 0.00s) 0.00s( 0.00s) Task 2 (worker) :0.00s( 17.97s) 0.00s( 0.00s) Task 3 (worker) :0.00s( 19.35s) 0.00s( 0.00s) Task 4 (worker) :0.00s( 40.28s) 0.00s( 0.00s) Task 5 (worker) :0.00s( 45.08s) 0.00s( 0.00s) Task 6 (worker) :0.00s( 47.06s) 0.00s( 0.00s) Task 7 (worker) : 18.30s( 49.73s) 16.24s( 1.71s) Task 8 (worker) :0.00s( 51.22s) 0.00s( 0.00s) Task 9 (worker) :0.00s( 53.75s) 0.00s( 0.00s) Task 10 (worker) :0.00s( 54.17s) 0.00s( 0.00s) Task 11 (worker) :5.65s( 54.30s) 0.70s( 0.08s) Task 12 (worker) :0.00s( 54.41s) 0.41s( 0.04s) Task 13 (worker) :4.34s( 54.58s) 4.50s( 0.48s) Task 14 (worker) :5.82s( 54.76s) 5.91s( 0.64s) Task 15 (worker) :6.50s( 55.01s) 3.37s( 0.38s) Task 16 (worker) :7.60s( 55.21s) 8.56s( 0.94s) Task
Re: [Haskell-cafe] SMP parallelism increasing GC time dramatically
I don't know if this is relevant to your problems, but I'm currently struggling to get some performance out of a parallel - or rather, concurrent - program. Basically, the initial thread parses some data into an IntMap, and then multiple threads access this read-only to do the Real Work. Now, there appears to be a lot of overhead incurred when using multiple threads, and I suspect that this is caused by the map storing unevaluated thunks, which then are forced by accesses by the worker threads. Ideally, the evaluation should be performed in parallel, but perhaps there are issues (of synchronization, say) that makes this less performant? -k -- If I haven't seen further, it is by standing in the footprints of giants ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] ANN: hit a reimplementation of git storage in haskell.
On 10/04/2011 11:07 PM, Jason Dagit wrote: On Tue, Oct 4, 2011 at 2:15 PM, Vincent Hanquezt...@snarc.org wrote: Any comments welcome, Nice! Have you looked at Petr Rockai's hashed-storage? http://hackage.haskell.org/package/hashed-storage-0.5.8 i heard about it before, but i don't know much more than that. I had a quick look at the hackage documentation: it's interesting and fairly similar in some aspects. thanks, -- Vincent ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] SMP parallelism increasing GC time dramatically
Ketil, For your particular problem, unevaluated thunks should be easy to check: dump a heap profile and look for a decreasing allocation of thunks. That being said, IntMap is spine strict, so that will all be evaluated, and if your threads are accessing disjoint keys there should be no contention. If there is, yes, threads will be blocking on evaluation, I don't have a good sense for how slow that tends to be. (Cache effects may be swamping you.) You may be interested in repa, if your maps are dense. Edward Excerpts from Ketil Malde's message of Wed Oct 05 17:00:11 -0400 2011: I don't know if this is relevant to your problems, but I'm currently struggling to get some performance out of a parallel - or rather, concurrent - program. Basically, the initial thread parses some data into an IntMap, and then multiple threads access this read-only to do the Real Work. Now, there appears to be a lot of overhead incurred when using multiple threads, and I suspect that this is caused by the map storing unevaluated thunks, which then are forced by accesses by the worker threads. Ideally, the evaluation should be performed in parallel, but perhaps there are issues (of synchronization, say) that makes this less performant? -k ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] ANN: hit a reimplementation of git storage in haskell.
On 10/05/2011 01:58 AM, Conrad Parker wrote: Hi Vincent, great stuff! I've also got an in-progress toy git clone called ght: http://github.com/kfish/ght. It only reads, no write support and no revspec parsing. I tried to keep close to the git design, using mmap and Ptr-based binary search to read pack indices etc. Doing so seems fairly un-Haskelly but turned out surprisingly neat, what with Haskell being the world's finest imperative programming language and all. Conrad. Hi Conrad, Look like i'm not the first one to kind-of reimplement git ;-) In my use case, creating objects was the most important bit to get this project running. Design wise, i made sure i can switch to mmap later too, which is the reason of the filereader abstraction. I'm actually getting a hold of mmap in haskell in a work project right now, and will probably add a mmap mode along with the handle mode to hit's filereader too. -- Vincent ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] SMP parallelism increasing GC time dramatically
Thanks for the reply, I haven't actually tried threadscope yet, I will have a look at that tomorrow at some point. I also had no idea you could use valgrind on haskell programs, so I will look into that as well. I think the program certainly does have problems scaling, since I made a very basic attempt at parallelism that wasn't dividing the work equally (although this was because in an early version of the program, this seemed to work better than dividing the work more finely and evenly, possibly because of the issue with parallel GC I mention below...) I just changed the code to split the work into smaller chunks and more evenly, using Control.Parallel, and it seems much better -- conditional on one thing. If I switch off the parallel GC by setting -qg, my code suddenly runs much faster and outperforms the sequential version, coming in at around 20 seconds walltime, and spends only 1% of its time on GC. top seems to be sitting at around 800%. I can post the output from +RTS -s as well if that would help. The only problem is that now I am getting random occasional segmentation faults that I was not been getting before, and once got a message saying: Main: schedule: re-entered unsafely Perhaps a 'foreign import unsafe' should be 'safe'? I think this may be something to do with creating a lot of sparks though, since this occurs whether I have the parallel GC on or not. But it also seems that turning on the parallel GC makes the fine grained parallel code, which is creating around 1,000,000 sparks, and outperforms the sequential version if I turn off parallel GC, run much slower (260s walltime) than the coarse grained parallel code (50,000 sparks, 82s walltime). I think this is what led me to originally use the coarse grained parallel version that divides the work unequally. I imagine there is still some room for improvement, since 20s is only about 3 times faster than the sequential version, but I think that is more an issue of me being something of a novice haskell programmer! It does seem to me like there is something very strange going on with the GC though. Unfortunately the code is for a paper (nothing to do with computer science or haskell incidentally), and although I'm planning on open sourcing it, I can't do that until the paper is published. thanks again Tom On Wed, Oct 5, 2011 at 9:36 PM, Ryan Newton rrnew...@gmail.com wrote: Hi Tom, I think debugging this sort of problem is exactly what we need to be doing (and making easier). Have you tried Duncan's newest version of Threadscope by the way? It looks like -- completely aside from the GC time -- this program is not scaling. The mutator time itself, disregarding GC, isn't going down much with parallelism (with the total mutator time increasing drastically). Either this is completely memory bottlenecked or there is some other kind of bad interaction (e.g. false sharing, contention on a hot lock, etc). My inclination would be to figure this out first before worrying about the GC time. Is this code that you would be able to share for debugging? I think we need to get together some general documentation on how to debug this kind of problem. For example, you can get some hints as to the memory behavior by running valgrind/cachegrind on the program. Also, what does top say, by the way? Is the process using 1200% CPU? Cheers, -Ryan On Wed, Oct 5, 2011 at 2:15 PM, Tom Thorne thomas.thorn...@gmail.comwrote: I am having some strange performance issues when using SMP parallelism, that I think may be something to do with GC. Apologies for the large readouts below but I'm not familiar enough to know what is and isn't relevant! I have a pure function that is mapped over a list of around 10 values, and this happens several times for each iteration of my program. It does some fairly intensive calculations using hmatrix, generating intermediate matrices along the way. The computation is significantly more complex for some values, so the work done by each call is not spread equally. I did some profiling and it seems like the program is spending about 50% of its time in that function. First of all, without any attempts at parallelism, I see this from ./Main +RTS -s 67,142,126,336 bytes allocated in the heap 147,759,264 bytes copied during GC 109,384 bytes maximum residency (58 sample(s)) 354,408 bytes maximum slop 3 MB total memory in use (0 MB lost due to fragmentation) Generation 0: 104551 collections, 0 parallel, 1.13s, 1.11s elapsed Generation 1:58 collections, 0 parallel, 0.01s, 0.01s elapsed Parallel GC work balance: -nan (0 / 0, ideal 1) MUT time (elapsed) GC time (elapsed) Task 0 (worker) :0.00s( 67.06s) 0.00s( 0.00s) Task 1 (worker) :0.00s( 67.09s) 0.00s( 0.00s) Task 2 (bound) : 66.95s( 67.09s) 1.14s( 1.12s) SPARKS: 0 (0
Re: [Haskell-cafe] SMP parallelism increasing GC time dramatically
On Wed, Oct 5, 2011 at 2:37 PM, Tom Thorne thomas.thorn...@gmail.comwrote: The only problem is that now I am getting random occasional segmentation faults that I was not been getting before, and once got a message saying: Main: schedule: re-entered unsafely Perhaps a 'foreign import unsafe' should be 'safe'? I think this may be something to do with creating a lot of sparks though, since this occurs whether I have the parallel GC on or not. Unless you (or some library you're using) is doing what the error message says then you should file a GHC bug here: http://hackage.haskell.org/trac/ghc/ -- Johan ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Haskell Weekly News: Issue 202
Welcome to issue 202 of the HWN, a newsletter covering developments in the Haskell community. This release covers the week of September 25 to October 1, 2011. New and Updated Projects * ParserFunction (Enzo Fabrizio; 0.0.5) Provides utilities for parsing and evaluating mathematical expressions. [1] http://goo.gl/xNOe1 * flexiwrap (Iain Alexander; 0.1.0) Also updates to related packages: data-type, function-combine, and flexiwrap-smallcheck. [2] http://goo.gl/dN0ew * crypto-api-tests (Thomas DuBuisson) These tests have been split out into their own package. Tests now use the test-framework package. [3] http://goo.gl/7kUIV * hledger (Simon Michael; 0.16) Stability/bugfix/polish release. [4] http://goo.gl/hW085 * GA (Kenneth Hoste; 1.0) Major version bump to this genetic algorithms package. [5] http://goo.gl/NoAl4 Quotes of the Week * glguy: lazy evaluation is best 50% of time because there are 2 cases, the cases where it is the best and the cases where it is not the best * Cale: Computing lengths is admitting defeat * syntaxfree: Many people think the core ideas behind Haskell [snip] In the Deep Complex Universe some Gnomes were having Tangling problems. Wadler friends solved their problems and as a reward the Tangled Gnomes gave them the Secret of Haskell * monochrom: How do I extract the IO out of IO String? Top Reddit Stories * GHC-specific alias analysis for LLVM Domain: blog.omega-prime.co.uk, Score: 74, Comments: 4 On Reddit: [6] http://goo.gl/kNXp9 Original: [7] http://goo.gl/851qj * GHC Projects for All Domain: dterei.blogspot.com, Score: 45, Comments: 2 On Reddit: [8] http://goo.gl/X3jpJ Original: [9] http://goo.gl/FpnUP * The Evolution of a Haskell Programmer Domain: willamette.edu, Score: 41, Comments: 17 On Reddit: [10] http://goo.gl/4WQyK Original: [11] http://goo.gl/gbNsH * Stanford CS240h, Functional Systems in Haskell course materials available on GitHub Domain: github.com, Score: 29, Comments: 3 On Reddit: [12] http://goo.gl/5aL1X Original: [13] http://goo.gl/EEflc * Introduction to programming with shift and reset (Tutorial by Oleg) Domain: okmij.org, Score: 29, Comments: 1 On Reddit: [14] http://goo.gl/Ogsjo Original: [15] http://goo.gl/xvVmQ * A few Haskell notes from the Strange Loop conference Domain: serpentine.com, Score: 27, Comments: 2 On Reddit: [16] http://goo.gl/qx4qn Original: [17] http://goo.gl/nplIs * Par Monad Slides from CUFP Domain: community.haskell.org, Score: 27, Comments: 0 On Reddit: [18] http://goo.gl/pFkgw Original: [19] http://goo.gl/Ax4bW * Slides from the CUFP 2011 Snap Framework Tutorial Domain: gregorycollins.net, Score: 23, Comments: 0 On Reddit: [20] http://goo.gl/uVrlv Original: [21] http://goo.gl/myRYa * Haskell Job Opportunity at Sensor Sense Domain: self.haskell, Score: 22, Comments: 1 On Reddit: [22] http://goo.gl/PpSdW Original: [23] http://goo.gl/PpSdW * Slides from my Haskell Implementors Workshop talk Domain: blog.johantibell.com, Score: 20, Comments: 3 On Reddit: [24] http://goo.gl/sgmzC Original: [25] http://goo.gl/BT4H0 Top StackOverflow Questions * Learning Haskell with a view to learning Scala votes: 23, answers: 4 Read on SO: [26] http://goo.gl/8BnAv * Why are the functions making Vector an instance of Functor, Monad, Applicative, Alternative, Foldable and Traversable slow? votes: 11, answers: 2 Read on SO: [27] http://goo.gl/F8sph * Strategies in Scala votes: 10, answers: 1 Read on SO: [28] http://goo.gl/NzUiU * What's the smoothest way to update haskell platform to latest? votes: 8, answers: 2 Read on SO: [29] http://goo.gl/EnG7p * Does Haskell have an equivalent to Sage? votes: 8, answers: 3 Read on SO: [30] http://goo.gl/v7oE8 * Haskell syntax for 'or' in case expressions votes: 7, answers: 3 Read on SO: [31] http://goo.gl/Akl9D * Is operator strict in Haskell? votes: 6, answers: 3 Read on SO: [32] http://goo.gl/TLe86 Until next time, Daniel Santa Cruz References 1. http://permalink.gmane.org/gmane.comp.lang.haskell.general/18942 2. http://permalink.gmane.org/gmane.comp.lang.haskell.general/18943 3. http://permalink.gmane.org/gmane.comp.lang.haskell.cafe/92700 4. http://permalink.gmane.org/gmane.comp.finance.ledger.hledger/521 5. http://permalink.gmane.org/gmane.comp.lang.haskell.cafe/92746 6. http://blog.omega-prime.co.uk/?p=135 7. http://www.reddit.com/r/haskell/comments/kwaee/ghcspecific_alias_analysis_for_llvm/ 8. http://dterei.blogspot.com/2011/09/ghc-project-for-all.html 9. http://www.reddit.com/r/haskell/comments/kwm0g/ghc_projects_for_all/ 10.
