On Fri, May 22, 2015 at 4:01 PM, Vadim Bobrov <[email protected]> wrote:
> > > On Friday, May 22, 2015 at 9:37:28 AM UTC-4, Patrik Nordwall wrote: >> >> If you are using a bounded mailbox you should use zero >> mailbox-push-timeout-time, which means that messages may be dropped. >> Otherwise you will block threads, which is probably not desired. >> > > If I simply allow messages to be dropped (what exactly dropped means - > routed to dead letters?) then there is no back pressure. Producers will > happily continue sending messages only for them to be dropped, not knowing > the consumer (W) is overwhelmed. I am not sure I understand why blocking > threads is bad in this particular case > It would potentially mean blocking one thread for each P. Make sure that you run these on a dedicated dispatcher if you go that route. > > >> >> It should be possible to solve this with pull pattern that is not awkward. >> Let W request a bunch a items from C. C does not push more than that >> number of items to P1-Pn-W until it gets a new request from W. >> > > isn't going to work. Like I said P1-Pn produce a lot of derived stuff. > Even a single message from C may trigger generation of too many derived > messages for W to process and will result in out-of-memory and crash. So > the back pressure must propagate all the way from W through P1-Pn to C and > this becomes too complicated. > Thanks for clarifying. /Patrik > > >> >> I don't fully understand what you mean by timeouts and blocking. >> >> Regards, >> Patrik >> >> On Thu, May 21, 2015 at 8:32 PM, Vadim Bobrov <[email protected]> wrote: >> >>> Endre, >>> >>> I did. However this is a multiple-thousand lines app and a complete >>> rewriting of its backbone is out of question in the nearest future. However >>> a proper and transparent backpressure handling will certainly help moving >>> toward streams. Currently what I have is a rather awkward pull pattern >>> >>> On Thursday, May 21, 2015 at 2:25:50 PM UTC-4, drewhk wrote: >>>> >>>> Hi Vadim, >>>> >>>> Why don't you look at Akka Streams first? It is all about handling >>>> backpressure in a simple way. >>>> >>>> -Endre >>>> >>>> On Thu, May 21, 2015 at 7:39 PM, Vadim Bobrov <[email protected]> >>>> wrote: >>>> >>>>> Hello all! >>>>> >>>>> I am thinking of using bounded mailbox as a simple solution to >>>>> backpressure implementation. My app basically consumes messages from a >>>>> queue (kafka), unpacks and processes them and sends to HBase storage. The >>>>> birds-eye overview of the principal actors is: >>>>> >>>>> W - writes to Hbase >>>>> C - consumes from Kafka and sends to W and a number of processors >>>>> P1...Pn >>>>> P1-Pn - do various processing, generating a lot of derived stuff and >>>>> send all to W >>>>> >>>>> so I want to propagate backpressure all the way from W (through P1-Pn, >>>>> because they are the primary generators of load) to C. Should I or should >>>>> I >>>>> not use message timeouts? I don't like the idea that I would have to >>>>> figure >>>>> out exactly what message and from what actor timed out - it'll make things >>>>> too complicated. On the other hand not using timeouts will block actors >>>>> which is a bad idea. Is it really in this case? I can't see why if the >>>>> actor has to wait anyway >>>>> >>>>> Thanks >>>>> Vadim >>>>> >>>>> -- >>>>> >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>> >>>>>>>>>> Check the FAQ: >>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>> >>>>>>>>>> Search the archives: >>>>> https://groups.google.com/group/akka-user >>>>> --- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Akka User List" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To post to this group, send email to [email protected]. >>>>> Visit this group at http://groups.google.com/group/akka-user. >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> -- >>> >>>>>>>>>> Read the docs: http://akka.io/docs/ >>> >>>>>>>>>> Check the FAQ: >>> http://doc.akka.io/docs/akka/current/additional/faq.html >>> >>>>>>>>>> Search the archives: >>> https://groups.google.com/group/akka-user >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "Akka User List" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To post to this group, send email to [email protected]. >>> Visit this group at http://groups.google.com/group/akka-user. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> >> >> -- >> >> Patrik Nordwall >> Typesafe <http://typesafe.com/> - Reactive apps on the JVM >> Twitter: @patriknw >> >> -- > >>>>>>>>>> Read the docs: http://akka.io/docs/ > >>>>>>>>>> Check the FAQ: > http://doc.akka.io/docs/akka/current/additional/faq.html > >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user > --- > You received this message because you are subscribed to the Google Groups > "Akka User List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/akka-user. > For more options, visit https://groups.google.com/d/optout. > -- Patrik Nordwall Typesafe <http://typesafe.com/> - Reactive apps on the JVM Twitter: @patriknw -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
