No, it doesn't accumulate the whole file, I am pretty sure because the file I work with wouldn't fit into memory, and the OS suggests that the process only uses 3-4MB of memory while streaming. So the streaming part is actually good, the difference in the execution time is what puzzles me...
On Friday, August 21, 2015 at 3:38:53 PM UTC+10, John Wiegley wrote: > > >>>>> Alexey Raga <alexe...@gmail.com <javascript:>> writes: > > > while this one runs much slower (>2 minutes to execute): > > bslines :: (MonadIO m) => Producer ByteString m () > > bslines = purely folds mconcat . view BS.lines $ BS.stdin > > > main :: IO () > > main = runEffect $ bslines >-> BS.stdout > > > Why does it happen? > > I'm pretty sure this is accumulating the entire file into memory, calling > mconcat N types, which will pay the cost of appending N times on segments > of > ever growing length. > > I think that using 'concats' will achieve what you want without degrading > performance, if I had to guess > > John > -- You received this message because you are subscribed to the Google Groups "Haskell Pipes" group. To unsubscribe from this group and stop receiving emails from it, send an email to haskell-pipes+unsubscr...@googlegroups.com. To post to this group, send email to haskell-pipes@googlegroups.com.
