Hi Folks-
I've got an application that uses cluster sharding to distribute work across
a cluster and provide failover. But because of a change in scope I need to
change how the work is done to maintain system performance while also
allowing for failover and cluster rebalancing.
At present the system has several actors that are created at application
startup via the ClusterSharding extension. The same actors with the same
behavior and configuration are started across all the nodes in the cluster.
These actors have no state, so they don't use akka persistence. A
singleton exists in the cluster whose purpose is to trigger the sharded
actors to perform work on a schedule(It's a homebrew integration with
Spring's TaskScheduler). Every day, that singleton triggers the actors
that were configured via ClusterSharding, and the sharded actors pull their
work from a DBMS, process each entity, save the results and wait until they
are triggered again.
Now the processing we have to do on each entity is grown in complexity and
time required, so I want to delegate that work to some actor that is
dedicated to each entity.
The simple answer to splitting up this work further seems like it would be
to maintain a list of actors that the sharded actor can delegate work to
then trigger them when the entities they represent need to be processed:
//inside the sharded actors receive:
listOfEntitiesFromDBQuery.each{ entity ->
//create the child actor if it doesn't exist
if(!context.getChild(entity.id)){
context.actorOf(Props.create(EntityProcessor.class, entity.id),
entity.id))
}
context.getChild(entity.id).tell(DOWORK)
}
But this design still looks like it would have problems.
1) While the sharded actor would move around the cluster because of
rebalancing, what about the children? They'll be stopped as the parent is
stopped and moved, correct? Is there any chance a child could exist on
both the old node and new node at the same time?
2) The children are processor intensive-so we'd want them to be
distributed across the cluster too and it seems that this won't happen with
this simple design.
It seems to me what I'd *like* to do is chard the processors/children as
well-not just start them as a child of the sharded actor on the actorsystem
it's currently living on. But I'm not sure how to do that since they
aren't known at startup.
Is this a problem anyone else has solved? If this is a common pattern, is
there anything built into akka-contrib that pertains to this?
thanks,
Mike Kohout
--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ:
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
