On Wed, Jun 11, 2014 at 1:01 AM, Marek Prochera <[email protected]> wrote:
> Hi Patrik, > > Thanks for your quick response > > My question was more conceptual, especially regarding sharing the same > routing rules across the cluster. But definitely it easier to talk about > real life example. > > Typical use case will be following, N-number of location transparent > producers will will put into system message via edge actors (rest services) > or from internal component actors. Message will be marked by special topic > which in some manner will define processing channel for message - one of > them will be ordered processing per topic. But beside the channels topic > will also define how the processing could be load balanced - eg. we will > gonna try to load balance topics per nodes. > > Most of the messages will be "tell" and beside fault handling I do not see > the risk of back pressure. > I used too few words. I mean the problem with producing more messages than can be consumed by the workers, or sending too many messages to a slow worker. That is when the work pulling is nice solution. > > The simplest idea would be to use the concept of master which would > distribute messages to the workers, but it means that we will have > bottleneck from the beginning. > I agree, that you should not be your first choice. > > That's why I started thinking about typical hash-based load balancer (the > same which is used for web load balancing) or just about the > ConsistentHashingRoutingLogic which seems to be straightforward and based > on the same architecture. But my questions is how to achieve it not having > another singleton bottleneck - actor playing the role of the router - > or even how to simplify coexistence of the same routing rules on all nodes. > If you need your own routing logic I think you should implement that as an ordinary actor that you run on each (edge) node. That is easier than implementing a custom router and you will probably not notice the performance difference. If you need a dynamic way to register the workers so that the other (edge) nodes know about them it can be convenient to use DistributedPubSub. You can plugin your routing logic to DistributedPubSub, and that will be used when sending messages to a logical path. Alternatively apply the routing logic before sending the message via DistributedPubSub. Cheers, Patrik > > > thanks, > > Marek > > > W dniu wtorek, 10 czerwca 2014 23:20:32 UTC+2 użytkownik Patrik Nordwall > napisał: >> >> Please elaborate on how the messages flow in the system? What is >> producing the messages? What are the routing rules? How will you handle >> back pressure? >> >> Regards, >> Patrik >> >> >> >> >> On Tue, Jun 10, 2014 at 2:30 PM, Marek Prochera <[email protected]> >> wrote: >> >>> Hi, I'm trying to choose the best pattern to implement cluster wide >>> distribution of massages/work with ordered processing requirements - of >>> course with all attitudes of Akka location, transparency, fault tolerance >>> and so on. >>> >>> In fact I'm going to migrate previous implementation based on extended >>> version of disruptor pattern which looks like this >>> >>> >>> >>> <https://lh3.googleusercontent.com/-gkdePmRMDYI/U5b3qnR6plI/AAAAAAAAick/lxTTI3lLZYk/s1600/disruptor-multi-ring.jpg> >>> >>> I see two potential implementation pattern which I could use >>> >>> - Singleton actor with work distribution similar to one presented in >>> this article (http://letitcrash.com/post/29044669086/balancing- >>> workload-across-nodes-with-akka-2) >>> - Extension of the routing logic, something similar to akka.routing. >>> ConsistentHashingRoutingLogic >>> >>> Second option would be more natural and flexible, but I'm not sure if >>> the cluster wide configuration of the router will work ? Does anyone know >>> if this particular commit introduce this type functionality ? (+con #3669 >>> Possibility to define routing logic in DistributedPubSub - >>> https://github.com/akka/akka/pull/1778) or I'm mixing different stuff ? >>> >>> Diagram of the design two design strategies below >>> >>> >>> >>> <https://lh5.googleusercontent.com/-FQfKxp3n4Ms/U5b5oaxFfOI/AAAAAAAAic0/Sj0rpCAiX_I/s1600/akka-topic.png> >>> >>> many thanks in advance >>> I guarantee good blog post about this topic ;) >>> >>> >>> Marek Prochera >>> >>> >>> -- >>> >>>>>>>>>> Read the docs: http://akka.io/docs/ >>> >>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/ >>> current/additional/faq.html >>> >>>>>>>>>> Search the archives: https://groups.google.com/ >>> group/akka-user >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "Akka User List" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To post to this group, send email to [email protected]. >>> >>> Visit this group at http://groups.google.com/group/akka-user. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> >> >> -- >> >> Patrik Nordwall >> Typesafe <http://typesafe.com/> - Reactive apps on the JVM >> Twitter: @patriknw >> >> <http://www.scaladays.org/> >> >> -- > >>>>>>>>>> Read the docs: http://akka.io/docs/ > >>>>>>>>>> Check the FAQ: > http://doc.akka.io/docs/akka/current/additional/faq.html > >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user > --- > You received this message because you are subscribed to the Google Groups > "Akka User List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/akka-user. > For more options, visit https://groups.google.com/d/optout. > -- Patrik Nordwall Typesafe <http://typesafe.com/> - Reactive apps on the JVM Twitter: @patriknw <http://www.scaladays.org/> -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
