Re: [haskell-pipes] Re: Parallelizing fold of Producer

[Michael Thompson]

 

Here's the module I was working with fwiw.  Is `chunkEval` below completely 
beside the point? It does speed up the calculation of the sum a bit, with 
`+RTS -N2`


    import Control.Foldl (list, purely)

    import Control.Parallel

    import Control.Monad.Par

    import Lens.Family.State.Strict (zoom)

    import Lens.Family

    import Pipes 

    import Pipes.Parse 

    import Prelude hiding (splitAt)

    import qualified Pipes.Prelude as P

    import Control.Monad (replicateM_)

    import Pipes.Group

    import Control.Exception

    import Control.Concurrent.Async

    fib :: Integer -> Integer

    fib 0 = 1

    fib 1 = 1

    fib n = fib (n-1) + fib (n-2)


    parallelize :: Monad m => Int -> Producer a m r -> Producer a m r

    parallelize n p = do

        let parser = zoom (splitAt n) drawAll

        (as, p') <- lift (runStateT parser p)

        as `par` (each as >> parallelize n p')

    


    batched n p = purely folds list (view (chunksOf n) p)


    chunkEval :: MonadIO m => Int -> Producer a m r -> Producer a m r

    chunkEval n p = purely folds list (view (chunksOf n) p) 

                    >-> P.mapM (liftIO . mapConcurrently evaluate) 

                    >-> P.concat


    fibproducer n = replicateM_ n p >-> P.map fib


    p = each [15..28::Integer]


    main = do 

    --  n <- P.sum $ parallelize 5 (fibproducer 7)

    --  n <- P.sum $ fibproducer 7

      n <- P.sum $ chunkEval 10 $ fibproducer 7 

      print n




On Sunday, November 8, 2015 at 1:29:57 PM UTC-5, Michael Thompson wrote:
>
> `parallelize` seems to diverge, but I can't tell why yet.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Haskell Pipes" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to haskell-pipes+unsubscr...@googlegroups.com.
To post to this group, send email to haskell-pipes@googlegroups.com.