Michael, I guess it is not so easy to translate the example to use your streaming library due to the parsing bit ?
On Sun, Nov 8, 2015 at 10:06 PM, Gabriel Gonzalez <gabriel...@gmail.com> wrote: > Yeah, actually `chunkEval` is a good approach, too. If that works then you > should go with that. > > > On 11/8/2015 10:43 AM, Michael Thompson wrote: > > Here's the module I was working with fwiw. Is `chunkEval` below completely > beside the point? It does speed up the calculation of the sum a bit, with > `+RTS -N2` > > > import Control.Foldl (list, purely) > > import Control.Parallel > > import Control.Monad.Par > > import Lens.Family.State.Strict (zoom) > > import Lens.Family > > import Pipes > > import Pipes.Parse > > import Prelude hiding (splitAt) > > import qualified Pipes.Prelude as P > > import Control.Monad (replicateM_) > > import Pipes.Group > > import Control.Exception > > import Control.Concurrent.Async > > fib :: Integer -> Integer > > fib 0 = 1 > > fib 1 = 1 > > fib n = fib (n-1) + fib (n-2) > > > parallelize :: Monad m => Int -> Producer a m r -> Producer a m r > > parallelize n p = do > > let parser = zoom (splitAt n) drawAll > > (as, p') <- lift (runStateT parser p) > > as `par` (each as >> parallelize n p') > > > > > batched n p = purely folds list (view (chunksOf n) p) > > > chunkEval :: MonadIO m => Int -> Producer a m r -> Producer a m r > > chunkEval n p = purely folds list (view (chunksOf n) p) > > >-> P.mapM (liftIO . mapConcurrently evaluate) > > >-> P.concat > > > fibproducer n = replicateM_ n p >-> P.map fib > > > p = each [15..28::Integer] > > > main = do > > -- n <- P.sum $ parallelize 5 (fibproducer 7) > > -- n <- P.sum $ fibproducer 7 > > n <- P.sum $ chunkEval 10 $ fibproducer 7 > > print n > > > > > On Sunday, November 8, 2015 at 1:29:57 PM UTC-5, Michael Thompson wrote: >> >> `parallelize` seems to diverge, but I can't tell why yet. > > -- > You received this message because you are subscribed to the Google Groups > "Haskell Pipes" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to haskell-pipes+unsubscr...@googlegroups.com. > To post to this group, send email to haskell-pipes@googlegroups.com. > > > -- > You received this message because you are subscribed to the Google Groups > "Haskell Pipes" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to haskell-pipes+unsubscr...@googlegroups.com. > To post to this group, send email to haskell-pipes@googlegroups.com. -- Pierre -- You received this message because you are subscribed to the Google Groups "Haskell Pipes" group. To unsubscribe from this group and stop receiving emails from it, send an email to haskell-pipes+unsubscr...@googlegroups.com. To post to this group, send email to haskell-pipes@googlegroups.com.
