Hi Ryon,
> I am a new Akka user; having read the documentation, I am having > difficulties understanding a few things, especially as they relate to my > project. I have a few specific questions regarding the capabilities of Akka > as it relates to our project goals, as well as a few general questions > regarding Akka itself. > > I am building out a Java-based system to handle events for user accounts. > We are consuming Avro-encoded messages from a Kafka queue, and these > messages are of different types (each message type uses a different Avro > schema). We do various tasks, largely Database-based with the contents of > each message. > > The consistent hashing router seems ideal for our needs as each hash would > map to an individual actor. Currently, our thinking is as follows: > > * We look up the proper Avro schema to decode the message, then > deserialize it into the correct type. > * We implement an Actor dedicated to handling each message type. That is, > there would be a 1-1 correspondence between Avro message types and Akka > Actor types. > * Each message, regardless of event type has a user's unique and > consistent user ID associated with it (intuitively, this means that each > message for a given user would hash identically). > * The most important point: we need to handle messages for a given user > SERIALLY; these are events for a user account and are strictly ordered. > Messages for different users, however, we may handled in parallel. > > Our primary concern with this project is that race conditions don't exist > between a single user's messages. > > The rub comes when I try to figure out the best way to implement this > idea. Namely, I haven't been able to find or figure out: > > 1. Whether or not a single Consistent Hashing Router can handle this. Can > we have a single router that can have instances of heterogeneous actors > that handle each message type? > The difference between Pool and Group is that a Pool creates its own routees, therefore it expects a factory for actors (a.k.a Props). By nature, Pools therefore use the same actor implementation for all routees. Groups on the other hand expect an externally injected list of paths to send messages to. It does not know what kind of actor sits at those paths, therefore it can be heterogeneous. The problem is with Group in your case that it accepts a static list of paths which might or might not be OK for your case. Also note, that the consistent-hashing is a dynamic operation, i.e. once one actor is dropped from a consistent hashing Pool (because it is cluster-aware for example and the corresponding machine has crashed) then it *rebalances* keys to the other routees. I.e. it does not guarantee that all messages of a given hash will *always* go the the same actor. It is important to mention though, that Routers are just a performance optimization, conceptually they are not different from an actor that has the only job to dispatch to its children or other actors. You can easily create an ordinary actor that fits your use case, there is no need to reach for routers. In fact, it is a very common beginners mistake to use routers when it is not necessary. I also recommend to take a look at sharding which might fit your case much better: http://doc.akka.io/docs/akka/2.4.0/scala/cluster-sharding.html > 2. If 1. DOES hold, are we still guaranteed that each user's messages will > be handled serially? (example: an instance of message type X comes off the > queue, then and instance of message type Y; both would be events for the > same user's account, so they would have the same hash value. Would message > X still complete processing before message Y? That is, does the consistent > hash mapper block on the calculated hash, or on the availability of an > actor that handles a specific type with that hash? > Yes, it is guaranteed for a consistent-hashing router *as long as the set of routees do not change*. If the routee set changes, the keys are uniformly rebalanced. > > In terms of general questions: > > 1. The documentation is quite fuzzy when it comes to Dispatchers vs. > Routers. On occasion, the documentation seems to imply that a Router can > replace a dispatcher completely, whereas in other places the documentation > describes configuring a dispatcher for a specific router. this makes it > difficult for me to understand when this actually needs to happen (in > particular, the Consistent Hashing router documentation in particular omits > explicit mention of a dispatcher). Is a dispatcher necessary or > configurable with the Consistent Hashing router? > They are completely different beasts. A Router is nothing more than an optimized actor. A Dispatcher is a carefully managed CPU resource, usually a pool of threads and as such it lives *below* the actor model. The only router that has any coupling with a specific dispatcher is the BalancingRouter (another mostly over- and abused tool) which has a very intimate relationship with a specialized Dispatcher specifically created to make this kind of router very performant. Please note that this is an exception though, routers and dispatchers live on different layers. > 2. I am unclear as to the role of Mailboxes in this scheme. With the > consistent hash router, is it one mailbox per calculated hash, or one > mailbox per calculated hash per type (that is, for a calculated hash of > "abc123" and messages of type X and Y, do the actors that handle type X and > Y have their own mailboxes? > No, this is a confusion. Every actor has a mailbox. In fact an actor is nothing more than a mailbox+state+behavior. In 99% of the cases you don't need to do anything with Mailboxes (as in changing configuration of them, etc.). They are basically queues and they don't care of your message types at all (they are all Objects to them). > 3. I read the following Slideshare: > http://www.slideshare.net/shinolajla/effective-actors-jaxconf2012-14472247 > and it seems to indicate that Actors should be non-blocking. Unfortunately, > I haven't been able to find examples of "proper" actor implementation in a > language other than Scala. Is there a way to implement database > reads/writes that are serial for a given user *without* a blocking actor? > The reason why you should not call blocking calls in an actor is because a set of actors share a common threadpool (the Dispatcher). Imagine that you have 8 threads in a thread-pool and 1000 of actors serving some users. Now if 8 of those actors call blocking calls, they basically starve the other 992 actors by holding those threads while the blocking call lasts. The proper solution is usually to instead of calling the blocking calls directly from those 1000 actors, you dedicate a pool of, let's say 16 actors (probably behind a router) and dedicate them a dispatcher (preferably a pinned which will make sure that 16 actors have each one dedicated thread, 16 in total). In this way, those actors are safe to block, since they don't starve the 1000 user actors from threads since they don't share the same thread pool. > > Thank you for your patience and any help that anyone could offer in > helping me puzzle through these issues. > Please consider buying Typesafe training and support (or even consulting) so we can help with these kind of questions more in-depth ( https://www.typesafe.com/services/expert-support). Cheers. -Endre > > -- > >>>>>>>>>> Read the docs: http://akka.io/docs/ > >>>>>>>>>> Check the FAQ: > http://doc.akka.io/docs/akka/current/additional/faq.html > >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user > --- > You received this message because you are subscribed to the Google Groups > "Akka User List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/akka-user. > For more options, visit https://groups.google.com/d/optout. > -- Akka Team Typesafe - Reactive apps on the JVM Blog: letitcrash.com Twitter: @akkateam -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
