I have a pipe which consumes data from storage, folding it with foldl package. It's all great and wonderful. But now, since I have multiple tasks which stream the same data basically, but fold them with different folds into different report types, I want to gather them into a single piped stream, which gets folded into multiple reports at once. On one hand I'd like to have a guarantee that data gets stored in a constant memory, on the other hand, I'd like reports to be computed in different threads.
Which are the best practices to achieve this? Is there a way to make a "tee" on a pipe, but then make sure it somehow gets blocked when consumer of one channel's copy gets "too fast", e.g. when slower report is 2 times slower and faster report consumes 2000 items while slower one only consumed 1000, I'd like to block faster report to wait until slower one catches up to not have more than 1000 elems in memory. Any ideas or experiences are welcome. Thank you! -- You received this message because you are subscribed to the Google Groups "Haskell Pipes" group. To unsubscribe from this group and stop receiving emails from it, send an email to haskell-pipes+unsubscr...@googlegroups.com. To post to this group, send email to haskell-pipes@googlegroups.com.
