W dniu wtorek, 3 stycznia 2017 19:54:14 UTC+1 użytkownik Marcin Pękalski
napisał:
>
>
> That is why I think foreach{thing => doStuff(thing)} is not a good option
> for me. I need to be able to use backpressure if computations take too long
> so I do not run out of RAM. Also the matrix with weights will probably be
> big (tens of GB), so it would be good if I did not have to store too many
> rows from file in the memory.
>
That's a valid concern, but Akka streams take care of that. foreach{thing
=> doStuff(thing)} pulls next "thing" from upstream only after "doStuff"
completes processing previous "thing", so back pressure is maintained. Of
course this means that the "things" are processed sequentially. You can
process them in parallel if you wish, by changing doStuff(thing) to return
Future[Unit] instead of Unit and then doing for example
mapAsyncUnordered(4)(doStuff).to(Sink.ignore) Akka Streams would ensure
that at most 4 invocations of "doStuff" are running at any given time.
cheers,
Rafał
--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ:
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
