Adam, We have some operations that run very quickly and on a limited circumstance and a fixed pool would be no problem. However, we have other operations that take a domain object, find the ids of related objects and do an operation on all of them. Throughout this thread I have been talking Customer and Order as common example, but we are really not processing that kind of data. I am just under some disclosure restraints. But imagine a customer has 200,000 orders and you get the picture. The domain object A is related to hundreds of thousands of Bs. Now I can push them all through a fixed pool but then I risk running out of bandwidth in the pool. You talked about an an actor per request but that becomes problematic when the data is processed on several different nodes.
The 200,000 ids are located and split up according to where they live in the cluster, in a 4 node system roughly 1/4 will go to each node. Then on that node they will respond to the message. This is important because of the balancing of the memory cache and to avoid unnecessary serialization in high volume situations. Finally, they will all send back responses to the caller. To send the messages to the node, I use a router. Its possible there is a way to have a router create actor per request, but I don't know how that could be done. If so that would be fine in a way but not in another way. There is a limited resource also there, the database, which can run only so many transactions at once so when we update all 200,000 objects we can't do it all at once with only 1000 db connections. What we have to do is funnel them through a pool so only so many are running concurrently. The fear I have is a deadlock from something Calling actor A from actor B and thats why I posted this thread. A variable pool, I was thinking, could realize all the actors are busy somehow and create a new set of actors that handle the recursive throughput. This is the problem I am trying to handle and I want to build it in such a way that its kid proof, that a dev with little to no akka experience can't deadlock the system waiting on a pool of a certain number of actors (currently a certain executor) while executing an operation. -- Robert On Friday, August 28, 2015 at 8:31:05 AM UTC-5, Adam wrote: > > Does a variable sized pool really have any advantage? > Or maybe you meant the size varies according to number of nodes. > Within the same JVM I would assume that a fixed size pool is good enough > as messages can be buffered and back pressure can be applied when the load > increases beyond what this pool of workers can handle. > > Regarding routing - I'm not familiar enough with Akka's clustering support > to give a reliable opinion - I've only played with it for a bit out of > interest. We use an internal implementation for that purpose. As a rule of > thumb, though, I'd assume it's best to keep the number of components that > have to handle clustering logic to a minimum. Each such component has to be > managed, so it increases the operational cost. So I'd say that if what you > described can handle the various issues that can (or actually will) arise > in a distributed system, I'd go for it. It at least sounds simple enough. > > On Thu, Aug 27, 2015 at 4:44 PM, kraythe <[email protected] <javascript:>> > wrote: > >> A variable sized pool could work. Because of the nature of the way our >> memory cache (Hazelcast) distributes map data and the essential need to run >> the operations local to that data, its essential we route the message to an >> actor on the node co-resident with the data, hence the router. Im not usign >> Akka clusterign but rather Akka remoting where the remote reference is >> stored in a map in hazelcast. Essentially the router says "is this key on >> my node? If not figure out what node its on and send it to that node, if so >> then use whatever local routing logic (round robin, etc) the user wanted in >> the router. I have never used the akka clustering features. Can you think >> of an easier way to do this ? >> >> Also I think I might have to implement a variable sized pool but if so >> thats fine so long as it is possible. >> >> Very helpful adam, thanks a bunch. >> >> -- Robert >> >> -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
