I have a 2-node akka cluster. Whenever a deployment happens, I send a akka-cluster (the JMX tool) "leave" command to each of the nodes and restart them one by one. This was going well for a while, but I noticed a strange pattern recently.
Sometimes, I get an infinite loop as shown in the following log, which requires both the nodes to be restarted. 2016-03-28 08:19:13,351 WARN [actorSystem-akka.actor.default-dispatcher-4] akka.cluster.sharding.ShardRegion - Trying to register to coordinator at [Some(ActorSelection[Anchor(akka://actorSystem/), Path(/system/sharding/SubrCoordinator/singleton/coordinator)])], but no acknowledgement. Total [36] buffered messages. Is the delivery failure of the TakeOverFromMe message the cause of this? Maybe it's unlikely as the name of the coordinator actor is different. 2016-03-28 09:22:13,857 INFO [actorSystem-akka.actor.default-dispatcher-15] akka.actor.EmptyLocalActorRef - Message [akka.cluster.singleton.ClusterSingletonManager$Internal$TakeOverFromMe$] from Actor[akka.tcp://actorSystem@app02:2551/system/sharding/sentinel-workerCoordinator#-489394297] to Actor[akka://actorSystem/system/sharding/sentinel-workerCoordinator] was not delivered. [3] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. I have also observed that this doesn't go into a loop when I run the two nodes on the same machine. What can be done to prevent this issue? Please help. -- Thanks Joseph -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
