Hello, Thanks for detailing . It clarified my thoughts a lot. This conversation has put more lights on reactive streams. Also, how reactive streams and pull mode better fit in design/architecture.
On Thursday, 7 August 2014 18:07:31 UTC+5:30, Konrad Malawski wrote: > > Hello again, > replies in-line: > > [-- cut --] >> > In Pull based approach, if producer is creating more work, we can >> implement a logic to add more worker actors to the system. These additional >> workers will take the surge of work created by fast producer. This >> maintain's balance or capability in the system to handle sudden load. >> > Sure, that's standard tactics for scaling on-demand. > Also, it assumes you are able to do this fast enough, which simply may not > be true. > > > >> Nothing against akka streams but it's back pressure will force the >> producer to push the work at slow rate. >> > No, because the protocol used by reactive streams can adapt and in the > case of faster subscribers than publishers, they can emit way larger demand > than the publisher is producing at. > We call it "dynamic push pull", which shows how the protocol behaves quite > well. This means that they can signal demand quite rarely, as in this > example: > > We can signal much more demand, if the subscriber is fast or has large > buffers: > S: DEMAND 10.000 > P: SEND > P: SEND > P: SEND > ... # 100 MORE SENDS > S: DEMAND 10.000 > P: SEND # SEND NO. 104 > > This effectively looks like push "for a while" (P can push 10000 elements > without waiting for any more demand). > Signalling demand can be interleaved between signalling data, which can > lead to the publisher never having to wait with publishing. > > If demand is depleted, it looks like pull again, because the publisher > can't publish to this subscriber. > > vs. the naive implementation (which we avoid, but uses the same protocol) > (which would such, because of the overheads, but perhaps that's exacly what > a subscriber needs – because it can't cope with more elements than 1 (has > no buffer, is slow, whatever)): > S: DEMAND 1 > P: SEND > S: DEMAND 1 > P: SEND > S: DEMAND 1 > P: SEND > > >> What are your thoughts on taking the approach of pull based processing >> with capability of adding more worker's or worker nodes on the fly? >> > Of course, that's one of our most often recommended patterns ever :-) > Like I said before, it still assumes the "work dispatcher" is able to keep > up with incoming work requests which it then delegates to these worker > nodes – this usually holds, but is not guaranteed. > > In order to guarantee things like this proper congestion control must be > implemented within the system - such as the reactive streams protocol (or > TCP or your home grown congestion control – there's plenty of these). > > -- > Cheers, > Konrad 'ktoso' Malawski > hAkker @ Typesafe > > <http://typesafe.com> > -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
