I think I can clarify a bit on what does have an impact on performance.
The reason that forcing things through the base monad or using `liftIO`
slows things down is not because `IO` or the base monad are
intrinsically slow, but rather because they interfere with compiler
optimizations. The compiler cannot optimize across these monad layers,
which causes problems in really tight loops.
This is why `pipes` internally uses a representation that minimizes the
use of the base monad. That's also why `FreeT` slows things down
because it does not do this sort of internal optimization.
On 8/6/2015 5:43 PM, Tran Ma wrote:
Hi Michael,
I read the conduit/vector-builder post. The approach is essentially
the same (using mutable vectors to chunk incoming single values), but
they seemed to go through a lot of trouble to stay in the base IO
monad. I'm not too sure I'm 100% convinced that "avoiding liftIO calls
has a *profound* performance impact" -- perhaps it does. I'd be
interested to see why these pipes/foldl versions outperform the
conduit one.
From inspecting the core (with "ghc -c Bar.hs -O2 -ddump-prep
-dppr-case-as-let -dsuppress-all"), there doesn't seem to be much
indirection in using "next" anyway. If I'm reading it correctly, ghc
manages to reduce the uses of "next" in "addToChunk" to simple casing
on the internal proxy constructors.
Even with the efficient core, Data.List still outperforms my chunked
streams, which makes me very sad =(
Thanks for your replies!
Tran
sidenote: it would be good to add an internal "chunk" operation to the
pipes library, I think -- does this sound useful to you, Gabriel?
On Friday, 7 August 2015 06:38:07 UTC+10, Michael Thompson wrote:
Dear Tran, forget that bit about conduit. Maybe I'm missing
something but it
seems the conduit version of our program using 'vectorBuilder' is
main =
cnums C.=$= C.vectorBuilder 1000 C.mapM_ C.=$= CL.map
Chunk C.$$ ii
where
cnums = CL.sourceList $ L.replicate 10000001 (1 :: Int)
ii = C.awaitForever $ \(Chunk v) -> V.mapM_ (($!) const
(return ())) v
and this takes about twice as much time as the 'idiomatic'
approach using the existing pipes material,
real0m4.090s vs real0m1.917s
--
You received this message because you are subscribed to the Google
Groups "Haskell Pipes" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to haskell-pipes+unsubscr...@googlegroups.com
<mailto:haskell-pipes+unsubscr...@googlegroups.com>.
To post to this group, send email to haskell-pipes@googlegroups.com
<mailto:haskell-pipes@googlegroups.com>.
--
You received this message because you are subscribed to the Google Groups "Haskell
Pipes" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to haskell-pipes+unsubscr...@googlegroups.com.
To post to this group, send email to haskell-pipes@googlegroups.com.
