I run a very small, 3 node cluster on EC2 and I observerve constant disassociacions.
Heartbeats are exchanged, nothing is lost: 05:52:06.702 [sgActors-akka.actor.default-dispatcher-62] DEBUG a.c.ClusterHeartbeatSender: Cluster Node [akka.tcp://[email protected]:2552] - Heartbeat to [akka.tcp://[email protected]:2555] 05:52:06.703 [sgActors-akka.actor.default-dispatcher-62] DEBUG a.c.ClusterHeartbeatSender: Cluster Node [akka.tcp://[email protected]:2552] - Heartbeat response from [akka.tcp://[email protected]:2555] 05:52:07.702 [sgActors-akka.actor.default-dispatcher-42] DEBUG a.c.ClusterHeartbeatSender: Cluster Node [akka.tcp://[email protected]:2552] - Heartbeat to [akka.tcp://[email protected]:2555] 05:52:07.704 [sgActors-akka.actor.default-dispatcher-42] DEBUG a.c.ClusterHeartbeatSender: Cluster Node [akka.tcp://[email protected]:2552] - Heartbeat response from [akka.tcp://[email protected]:2555] 05:52:08.412 [sgActors-akka.actor.default-dispatcher-42] DEBUG a.r.EndpointWriter: Disassociated [akka.tcp://[email protected]:2552] -> [akka.tcp://[email protected]:2555] 05:52:08.413 [sgActors-akka.actor.default-dispatcher-42] WARN a.r.ReliableDeliverySupervisor: Association with remote system [akka.tcp://[email protected]:2555] has failed, address is now gated for [5000] ms. Reason is: [Disassociated]. 05:52:08.413 [sgActors-akka.actor.default-dispatcher-42] DEBUG a.r.EndpointWriter: Disassociated [akka.tcp://[email protected]:2552] -> [akka.tcp://[email protected]:2555] 05:52:08.701 [sgActors-akka.actor.default-dispatcher-63] DEBUG a.c.ClusterHeartbeatSender: Cluster Node [akka.tcp://[email protected]:2552] - Heartbeat to [akka.tcp://[email protected]:2555] 05:52:09.702 [sgActors-akka.actor.default-dispatcher-40] DEBUG a.c.ClusterHeartbeatSender: Cluster Node [akka.tcp://[email protected]:2552] - Heartbeat to [akka.tcp://[email protected]:2555] 05:52:10.702 [sgActors-akka.actor.default-dispatcher-40] DEBUG a.c.ClusterHeartbeatSender: Cluster Node [akka.tcp://[email protected]:2552] - Heartbeat to [akka.tcp://[email protected]:2555] 05:52:11.701 [sgActors-akka.actor.default-dispatcher-63] DEBUG a.c.ClusterHeartbeatSender: Cluster Node [akka.tcp://[email protected]:2552] - Heartbeat to [akka.tcp://[email protected]:2555] 05:52:12.702 [sgActors-akka.actor.default-dispatcher-63] DEBUG a.c.ClusterHeartbeatSender: Cluster Node [akka.tcp://[email protected]:2552] - Heartbeat to [akka.tcp://[email protected]:2555] 05:52:13.521 [sgActors-akka.actor.default-dispatcher-63] DEBUG a.r.EndpointWriter: Associated [akka.tcp://[email protected]:2552] -> [akka.tcp://[email protected]:2555] 05:52:13.701 [sgActors-akka.actor.default-dispatcher-63] DEBUG a.c.ClusterHeartbeatSender: Cluster Node [akka.tcp://[email protected]:2552] - Heartbeat to [akka.tcp://[email protected]:2555] 05:52:13.703 [sgActors-akka.actor.default-dispatcher-63] DEBUG a.c.ClusterHeartbeatSender: Cluster Node [akka.tcp://[email protected]:2552] - Heartbeat response from [akka.tcp://[email protected]:2555] But somehow EndpointWriter disassosiates regularly, every couple of minutes: 05:46:50.856 [sgActors-akka.actor.default-dispatcher-62] DEBUG a.r.EndpointWriter: Disassociated [akka.tcp://[email protected]:2552] -> [akka.tcp://[email protected]:2555] 05:46:50.857 [sgActors-akka.actor.default-dispatcher-62] WARN a.r.ReliableDeliverySupervisor: Association with remote system [akka.tcp://[email protected]:2555] has failed, address is now gated for [5000] ms. Reason is: [Disassociated]. 05:46:50.857 [sgActors-akka.actor.default-dispatcher-62] DEBUG a.r.EndpointWriter: Disassociated [akka.tcp://[email protected]:2552] -> [akka.tcp://[email protected]:2555] 05:49:31.221 [sgActors-akka.actor.default-dispatcher-3] DEBUG a.r.EndpointWriter: Disassociated [akka.tcp://[email protected]:2552] -> [akka.tcp://[email protected]:2555] 05:49:31.222 [sgActors-akka.actor.default-dispatcher-3] WARN a.r.ReliableDeliverySupervisor: Association with remote system [akka.tcp://[email protected]:2555] has failed, address is now gated for [5000] ms. Reason is: [Disassociated]. 05:49:31.222 [sgActors-akka.actor.default-dispatcher-3] DEBUG a.r.EndpointWriter: Disassociated [akka.tcp://[email protected]:2552] -> [akka.tcp://[email protected]:2555] 05:52:08.412 [sgActors-akka.actor.default-dispatcher-42] DEBUG a.r.EndpointWriter: Disassociated [akka.tcp://[email protected]:2552] -> [akka.tcp://[email protected]:2555] 05:52:08.413 [sgActors-akka.actor.default-dispatcher-42] WARN a.r.ReliableDeliverySupervisor: Association with remote system [akka.tcp://[email protected]:2555] has failed, address is now gated for [5000] ms. Reason is: [Disassociated]. 05:52:08.413 [sgActors-akka.actor.default-dispatcher-42] DEBUG a.r.EndpointWriter: Disassociated [akka.tcp://[email protected]:2552] -> [akka.tcp://[email protected]:2555] Today I've seen 440 Disassociacions during 6 hours. Is there a way to track disassociacion cause? Remote counterpart (10.79.1.14:2555) does not have any of disassociacion events in logs, so it seems it's from one side only. Jvm is not loaded, no gc pauses -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
