I'm having some difficulty with akka clustering and I'm hoping someone here has solved this problem before.
I'm running an 30 node cluster in AWS and quite often when several nodes restart the cluster state refuses to converge and I get logs that looks like this: New incarnation of existing member [Member(address = > akka.tcp://[email protected]:2552, status = Down)] is trying to > join. Existing will be removed from the cluster and then new member will be > allowed to join. > Followed by: > Leader can currently not perform its duties, reachability status: ... More complete logs here: https://gist.github.com/hiloboy0119/40b2ed7683b0472b630a11717af42b61 I've tried all manner of tuning in the akka.remote and akka.cluster sections of application.conf but nothing really seems to help. I'm currently running akka 2.4.6 (I saw there was a fix related to this in 2.4.3) Relevant snippet of my application.conf remote { > log-remote-lifecycle-events = on > netty.tcp { > hostname = ${?LOCAL_IP} > bind-hostname = 0.0.0.0 > port = ${?AKKA_PORT} > } > use-dispatcher = "remote-dispatcher" > retry-gate-closed-for = 5 s > } > cluster { > seed-nodes = [${?SEED_NODE1}, ${?SEED_NODE2}, ${?SEED_NODE3}] > use-dispatcher = "cluster-dispatcher" > roles = ["backend"] > auto-down-unreachable-after = 10s > allow-weakly-up-members = on > role { > # Minimum required number of members of a certain role before the > leader > # changes member status of 'Joining' members to 'Up'. Typically used > together > # with 'Cluster.registerOnMemberUp' to defer some action, such as > starting > # actors, until the cluster has reached a certain size. > backend.min-nr-of-members = 20 > } > } -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
