Hi Evan, first off, we describe Reactive Streams as being dynamic push–pull: due to the batching, a fast consumer can keep the demand at the producer always big enough such that the system behaves exactly like a push-model.
Concerning merge strategies we offer a rather limited set at the moment plus the ability to write your own, and each of the input ports of the merging stage will have a (small) buffer that it tries to keep filled by signaling demand to its Publisher. As such you can implement any scheme that you like, but currently we do not support dynamic merging in the sense of allowing streams to come and go. I do agree that this is a must-have, though. Regards, Roland > 13 feb 2015 kl. 08:29 skrev Evan Chan <[email protected]>: > > Dear friends, > > I've read and grok the back-pressure story around Akka streams, which is > pull-based. I have also seen examples of this with fixed topologies that > tend to be relatively linear or are fan-out: for example, a coordinator > farming work out to worker nodes. For these examples, I can see that it > would work well. Each worker issues a pull request (haha, no pun intended) > for more work from one master. > > Now, consider the case where I may have multiple flows or pipelines feeding > in to one I/O sink, which does something like a mapAsync (feeds to futures > doing I/O). This is much more of a fan-in pattern. > The multiplicity of the pipelines changes dynamically. > > For example, let's say that each pipeline ingests one customer's data, and it > is simpler to keep one pipeline per customer. As a new customer comes in, > then we want to start a pipeline for them, and it should feed into the same > I/O sink. If there are more customers/pipelines, one would expect > backpressure so that each flow slows down, which is expected. > > Implementing a pull backpressure in this case seems much more complicated. > Does Akka streams handle dynamic registration / materialization? In order > for the sink to know about each pipeline, each pipeline must somehow register > itself with the sink, and the sink must then send pull requests to each > pipeline. Or, would the sink only send pull requests to some random subset > of pipelines, if it is more heavily loaded? > > I can think of a way to have push, but still have backpressure, even over > networks. > > -Evan > > -- > >>>>>>>>>> Read the docs: http://akka.io/docs/ <http://akka.io/docs/> > >>>>>>>>>> Check the FAQ: > >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html > >>>>>>>>>> <http://doc.akka.io/docs/akka/current/additional/faq.html> > >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user > >>>>>>>>>> <https://groups.google.com/group/akka-user> > --- > You received this message because you are subscribed to the Google Groups > "Akka User List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] > <mailto:[email protected]>. > To post to this group, send email to [email protected] > <mailto:[email protected]>. > Visit this group at http://groups.google.com/group/akka-user > <http://groups.google.com/group/akka-user>. > For more options, visit https://groups.google.com/d/optout > <https://groups.google.com/d/optout>. Dr. Roland Kuhn Akka Tech Lead Typesafe <http://typesafe.com/> – Reactive apps on the JVM. twitter: @rolandkuhn <http://twitter.com/#!/rolandkuhn> -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
