Hi David. I've looked at pipes-concurrency before and it was a bit slow, but I'll try it out as one of the options.
Thank you. On Tuesday, March 29, 2016 at 7:09:37 PM UTC+3, David McBride wrote: > > Have you looked at the pipes-concurrency package? It allows you to spawn > threads from consumers and producers that pass messages between each other > and you can bound the buffers to allow you to achive good memory usage. It > has a nice tutorial as well. > > On Tue, Mar 29, 2016 at 7:13 AM, Kostiantyn Rybnikov <k-...@k-bx.com > <javascript:>> wrote: > >> I have a pipe which consumes data from storage, folding it with foldl >> package. It's all great and wonderful. But now, since I have multiple tasks >> which stream the same data basically, but fold them with different folds >> into different report types, I want to gather them into a single piped >> stream, which gets folded into multiple reports at once. On one hand I'd >> like to have a guarantee that data gets stored in a constant memory, on the >> other hand, I'd like reports to be computed in different threads. >> >> Which are the best practices to achieve this? Is there a way to make a >> "tee" on a pipe, but then make sure it somehow gets blocked when consumer >> of one channel's copy gets "too fast", e.g. when slower report is 2 times >> slower and faster report consumes 2000 items while slower one only consumed >> 1000, I'd like to block faster report to wait until slower one catches up >> to not have more than 1000 elems in memory. >> >> Any ideas or experiences are welcome. Thank you! >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Haskell Pipes" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to haskell-pipe...@googlegroups.com <javascript:>. >> To post to this group, send email to haskel...@googlegroups.com >> <javascript:>. >> > > -- You received this message because you are subscribed to the Google Groups "Haskell Pipes" group. To unsubscribe from this group and stop receiving emails from it, send an email to haskell-pipes+unsubscr...@googlegroups.com. To post to this group, send email to haskell-pipes@googlegroups.com.
