Hi Michael & Gabriel,
Thanks for the replies, they are very helpful.
I had a good ol' C-style solution with IORefs (yuck), here is how it
compares with pipes-group's "chunksOf" and Michael's faster "chunksOf":
idiomatic way: 0.97 real
avoid not going down to the Proxy monad: 0.75 real
C style: 0.66 real
Is the speedup from your alternative "chunksOf" gained from not going down
to the Proxy monad, Michael? I didn't know FreeT was that expensive.
Here is the C-style chunking code, it produces some tight loops in Core, so
that's good. It's pretty awful though:
chunkify :: Int -> Producer a IO () -> Producer (Chunk a) IO ()
chunkify size p
= do x <- liftIO $ startChunk size p
case x of
Nothing
-> return ()
Just (ref, p')
-> do p'' <- liftIO (addToChunk size ref p')
chunk <- liftIO (readIORef ref >>= freezeChunk)
yield chunk
chunkify size p''
freezeChunk (i, v)
= V.unsafeFreeze (VM.unsafeSlice 0 i v) >>= return . Chunk
startChunk size p
= do x <- next p
case x of
Left _
-> return Nothing
Right (a, p')
-> do chunk <- VM.new size
VM.unsafeWrite chunk 0 a
ref <- liftIO $ newIORef (1,chunk)
return $ Just (ref, p')
addToChunk size ref p
= do x <- next p
case x of
Left _
-> return p
Right (a, p')
-> do (i, chunk) <- readIORef ref
VM.unsafeWrite chunk i a
writeIORef ref (i+1, chunk)
if i + 1 >= size
then return p'
else addToChunk size ref p'
Even this code is not as fast as I expected it to be, which is strange to
me =/
Thanks again,
Tran
On Thursday, 6 August 2015 14:35:20 UTC+10, Michael Thompson wrote:
>
> I noticed that `chunksOf` involves a very complicated double use of `next`
> (it uses
> `splitAt` which is also defined with `next`). `next` basically forces each
> step into the
> base monad.
>
> It's not a miracle, but if I reimplement `chunksOf` using Pipes.Internal
> explicitly,
> the 'idiomatic' program goes from
>
> real 0m0.209s
>
> to
>
> real 0m0.097s
>
> (I was chunking at 10 for some reason) Not too bad. Does this work better
> for you Tran?
>
> import qualified Pipes.Internal as I
>
> chunksOf
> :: Monad m => Int
> -> Lens (Producer a' m x) (Producer a m x)
> (FreeT (Producer a' m) m x) (FreeT (Producer a m) m x)
> chunksOf n0 k p0 = fmap concats (k (_chunksOf_ p0))
> where
> _chunksOf_ p = case p of
> I.Pure r -> return r
> I.Request v _ -> I.closed v
> I.M m -> FreeT $ m >>= runFreeT . _chunksOf_
> x -> FreeT $ return $ Free $ fmap _chunksOf_ (to n0
> x)
> to n p =
> if n <= 0
> then I.Pure p
> else case p of
> I.Pure r -> return (return r)
> I.Request v _ -> I.closed v
> I.M m -> I.M $ liftM (to n) m
> I.Respond a f -> I.Respond a (to (n-1) . f)
>
> `to n` there is `view (splitAt n)` which would presumably be a little
> better,
> when used by itself, if it were written without `next`.
>
>
>
>
>
--
You received this message because you are subscribed to the Google Groups
"Haskell Pipes" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to haskell-pipes+unsubscr...@googlegroups.com.
To post to this group, send email to haskell-pipes@googlegroups.com.
