Guido, I agree with your advice. I just wanted to mention the wrong conf property as additional info. /Patrik mån 24 aug 2015 kl. 17:23 skrev Guido Medina <[email protected]>:
> And more information about the application itself to see memory usage > patterns. > > > On Monday, August 24, 2015 at 4:22:10 PM UTC+1, Guido Medina wrote: > >> Hi Patrik, >> >> I'm quite sure his problem has anything to do with Akka but lack of >> proper JVM parameters, that 32 seconds pause shown in his logs would be >> enough to cause nodes de-association and connection failures/timeouts of >> Akka remote (without mentioning Akka cluster and other extensions) hence I >> suggested to switch to newest garbage collector, of course the ideal thing >> to do is for OP to post JVM parameters and more information about the JVM >> in use to aid better. >> >> Hope that helps, >> >> Guido. >> >> On Monday, August 24, 2015 at 2:41:53 PM UTC+1, Patrik Nordwall wrote: >> >> Note that there is no: akka.cluster.transport-failure-detector >> >> you probably mean: akka.remote.transport-failure-detector >> >> The Disassociated might be triggered by that failure detector and you >> can try to increase >> >> akka.remote.transport-failure-detector.acceptable-heartbeat-pause >> >> /Patrik >> >> On Fri, Aug 21, 2015 at 11:29 AM, Guido Medina <[email protected]> wrote: >> >> Try the JVM option -XX:+UseG1GC, assuming you are using at least JVM >> 7uLatest where latest should be 79 or 80. The new GC algorithm doesn't stop >> world, it works more often and can make your application a bit slower (on a >> very hyper micro-level) but GC will now be consistent. >> >> >> On Friday, August 21, 2015 at 6:36:11 AM UTC+1, Ara Tatous wrote: >> >> I have a configuration of Akka cluster with 4 nodes. Let's call them >> Engine and Warehouse. >> >> In Engine: >> >> >> >> *Everything is normal* >> >> [DEBUG] [08/20/2015 07:32:25.082] >> [EngineWarehouseActorSystem-akka.actor.default-dispatcher-80] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2552/system/cluster/core/daemon/heartbeatSender] >> Cluster Node [akka.tcp://EngineWarehouseActorSystem@compprod2:2552] - >> Heartbeat to [akka.tcp://EngineWarehouseActorSystem@compprod1:2551] >> >> [DEBUG] [08/20/2015 07:32:25.082] >> [EngineWarehouseActorSystem-akka.actor.default-dispatcher-80] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2552/system/cluster/core/daemon/heartbeatSender] >> Cluster Node [akka.tcp://EngineWarehouseActorSystem@compprod2:2552] - >> Heartbeat to [akka.tcp://EngineWarehouseActorSystem@compprod1:2552] >> >> [DEBUG] [08/20/2015 07:32:25.082] >> [EngineWarehouseActorSystem-akka.actor.default-dispatcher-80] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2552/system/cluster/core/daemon/heartbeatSender] >> Cluster Node [akka.tcp://EngineWarehouseActorSystem@compprod2:2552] - >> Heartbeat to [akka.tcp://EngineWarehouseActorSystem@compprod2:2551] >> >> [DEBUG] [08/20/2015 07:32:25.084] >> [EngineWarehouseActorSystem-akka.actor.default-dispatcher-98] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2552/system/cluster/core/daemon/heartbeatSender] >> Cluster Node [akka.tcp://EngineWarehouseActorSystem@compprod2:2552] - >> Heartbeat response from [akka.tcp://EngineWarehouseActorSystem@compprod2 >> :2551] >> >> [DEBUG] [08/20/2015 07:32:25.084] >> [EngineWarehouseActorSystem-akka.actor.default-dispatcher-98] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2552/system/cluster/core/daemon/heartbeatSender] >> Cluster Node [akka.tcp://EngineWarehouseActorSystem@compprod2:2552] - >> Heartbeat response from [akka.tcp://EngineWarehouseActorSystem@compprod1 >> :2551] >> >> [DEBUG] [08/20/2015 07:32:25.084] >> [EngineWarehouseActorSystem-akka.actor.default-dispatcher-98] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2552/system/cluster/core/daemon/heartbeatSender] >> Cluster Node [akka.tcp://EngineWarehouseActorSystem@compprod2:2552] - >> Heartbeat response from [akka.tcp://EngineWarehouseActorSystem@compprod1 >> :2552] >> >> >> >> >> >> *We have GC pause here* >> >> 2015-08-20T07:32:31.959-0700: [GC2015-08-20T07:32:31.960-0700: [ParNew >> (promotion failed): 172544K->170208K(172544K), 0.8968931 >> secs]2015-08-20T07:32:32.857-0700: [CMS2015-08-20T07:32:39.682-0700: >> [CMS-concurrent-sweep: 7.003/7.909 secs] [Times: user=8.78 sys=0.40, >> real=7.91 secs] >> >> (concurrent mode failure): 5760566K->2755481K(6099776K), 30.9310318 >> secs] 5924201K->2755481K(6272320K), [CMS Perm : 142090K->141971K(237016K)], >> 31.8286597 secs] [Times: user=32.22 sys=0.38, real=31.83 secs] >> >> >> >> *We have almost 32 seconds GC pause. The following heartbeat request are >> buffered and played right after GC pause in one single second!. All the >> connection is being disassociated with other cluster members.* >> >> >> >> [DEBUG] [08/20/2015 07:33:03.794] >> [EngineWarehouseActorSystem-akka.actor.default-dispatcher-23] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2552/system/cluster/core/daemon/heartbeatSender] >> Cluster Node [akka.tcp://EngineWarehouseActorSystem@compprod2:2552] - >> Heartbeat to [akka.tcp://EngineWarehouseActorSystem@compprod1:2551] >> >> [DEBUG] [08/20/2015 07:33:03.794] >> [EngineWarehouseActorSystem-akka.actor.default-dispatcher-23] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2552/system/cluster/core/daemon/heartbeatSender] >> Cluster Node [akka.tcp://EngineWarehouseActorSystem@compprod2:2552] - >> Heartbeat to [akka.tcp://EngineWarehouseActorSystem@compprod1:2552] >> >> [DEBUG] [08/20/2015 07:33:03.794] >> [EngineWarehouseActorSystem-akka.actor.default-dispatcher-23] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2552/system/cluster/core/daemon/heartbeatSender] >> Cluster Node [akka.tcp://EngineWarehouseActorSystem@compprod2:2552] - >> Heartbeat to [akka.tcp://EngineWarehouseActorSystem@compprod2:2551] >> >> [DEBUG] [08/20/2015 07:33:03.808] >> [EngineWarehouseActorSystem-akka.actor.default-dispatcher-81] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2552/system/cluster/core/daemon/heartbeatSender] >> Cluster Node [akka.tcp://EngineWarehouseActorSystem@compprod2:2552] - >> Heartbeat to [akka.tcp://EngineWarehouseActorSystem@compprod1:2551] >> >> [DEBUG] [08/20/2015 07:33:03.808] >> [EngineWarehouseActorSystem-akka.actor.default-dispatcher-81] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2552/system/cluster/core/daemon/heartbeatSender] >> Cluster Node [akka.tcp://EngineWarehouseActorSystem@compprod2:2552] - >> Heartbeat to [akka.tcp://EngineWarehouseActorSystem@compprod1:2552] >> >> [DEBUG] [08/20/2015 07:33:03.808] >> [EngineWarehouseActorSystem-akka.actor.default-dispatcher-81] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2552/system/cluster/core/daemon/heartbeatSender] >> Cluster Node [akka.tcp://EngineWarehouseActorSystem@compprod2:2552] - >> Heartbeat to [akka.tcp://EngineWarehouseActorSystem@compprod2:2551] >> >> [DEBUG] [08/20/2015 07:33:03.808] >> [EngineWarehouseActorSystem-akka.actor.default-dispatcher-97] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2552/system/cluster/core/daemon/heartbeatSender] >> Cluster Node [akka.tcp://EngineWarehouseActorSystem@compprod2:2552] - >> Heartbeat to [akka.tcp://EngineWarehouseActorSystem@compprod1:2551] >> >> [DEBUG] [08/20/2015 07:33:03.808] >> [EngineWarehouseActorSystem-akka.actor.default-dispatcher-97] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2552/system/cluster/core/daemon/heartbeatSender] >> Cluster Node [akka.tcp://EngineWarehouseActorSystem@compprod2:2552] - >> Heartbeat to [akka.tcp://EngineWarehouseActorSystem@compprod1:2552] >> >> [DEBUG] [08/20/2015 07:33:03.808] >> [EngineWarehouseActorSystem-akka.actor.default-dispatcher-97] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2552/system/cluster/core/daemon/heartbeatSender] >> Cluster Node [akka.tcp://EngineWarehouseActorSystem@compprod2:2552] - >> Heartbeat to [akka.tcp://EngineWarehouseActorSystem@compprod2:2551] >> >> >> >> *After GC pause we have these disassociations* >> >> [DEBUG] [08/20/2015 07:33:03.823] >> [EngineWarehouseActorSystem-akka.remote.default-remote-dispatcher-79] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FEngineWarehouseActorSystem%40compprod1%3A2551-0/endpointWriter] >> Disassociated [akka.tcp://EngineWarehouseActorSystem@compprod2:2552] -> >> [akka.tcp://EngineWarehouseActorSystem@compprod1:2551] >> >> [DEBUG] [08/20/2015 07:33:03.823] >> [EngineWarehouseActorSystem-akka.remote.default-remote-dispatcher-79] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FEngineWarehouseActorSystem%40compprod1%3A2552-2/endpointWriter] >> Disassociated [akka.tcp://EngineWarehouseActorSystem@compprod2:2552] -> >> [akka.tcp://EngineWarehouseActorSystem@compprod1:2552] >> >> [WARN] [08/20/2015 07:33:03.823] >> [EngineWarehouseActorSystem-akka.remote.default-remote-dispatcher-79] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FEngineWarehouseActorSystem%40compprod1%3A2551-0] >> Association with remote system >> [akka.tcp://EngineWarehouseActorSystem@compprod1:2551] has failed, >> address is now gated for [5000] ms. Reason: [Disassociated] >> >> [WARN] [08/20/2015 07:33:03.827] >> [EngineWarehouseActorSystem-akka.remote.default-remote-dispatcher-147] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FEngineWarehouseActorSystem%40compprod1%3A2552-2] >> Association with remote system >> [akka.tcp://EngineWarehouseActorSystem@compprod1:2552] has failed, >> address is now gated for [5000] ms. Reason: [Disassociated] >> >> [DEBUG] [08/20/2015 07:33:03.838] >> [EngineWarehouseActorSystem-akka.remote.default-remote-dispatcher-147] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FEngineWarehouseActorSystem%40compprod1%3A2552-2/endpointWriter] >> Disassociated [akka.tcp://EngineWarehouseActorSystem@compprod2:2552] -> >> [akka.tcp://EngineWarehouseActorSystem@compprod1:2552] >> >> [DEBUG] [08/20/2015 07:33:03.839] >> [EngineWarehouseActorSystem-akka.remote.default-remote-dispatcher-147] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FEngineWarehouseActorSystem%40compprod2%3A2551-8/endpointWriter] >> Disassociated [akka.tcp://EngineWarehouseActorSystem@compprod2:2552] -> >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2551] >> >> [DEBUG] [08/20/2015 07:33:03.839] >> [EngineWarehouseActorSystem-akka.remote.default-remote-dispatcher-79] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FEngineWarehouseActorSystem%40compprod1%3A2551-0/endpointWriter] >> Disassociated [akka.tcp://EngineWarehouseActorSystem@compprod2:2552] -> >> [akka.tcp://EngineWarehouseActorSystem@compprod1:2551] >> >> [WARN] [08/20/2015 07:33:03.839] >> [EngineWarehouseActorSystem-akka.remote.default-remote-dispatcher-147] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FEngineWarehouseActorSystem%40compprod2%3A2551-8] >> Association with remote system >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2551] has failed, >> address is now gated for [5000] ms. Reason: [Disassociated] >> >> [DEBUG] [08/20/2015 07:33:03.851] >> [EngineWarehouseActorSystem-akka.remote.default-remote-dispatcher-79] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FEngineWarehouseActorSystem%40compprod2%3A2551-8/endpointWriter] >> Disassociated [akka.tcp://EngineWarehouseActorSystem@compprod2:2552] -> >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2551] >> >> [DEBUG] [08/20/2015 07:33:03.851] >> [EngineWarehouseActorSystem-akka.remote.default-remote-dispatcher-147] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2552/system/endpointManager/endpointWriter-akka.tcp%3A%2F%2FEngineWarehouseActorSystem%40compprod1%3A2552-13] >> Disassociated [akka.tcp://EngineWarehouseActorSystem@compprod2:2552] <- >> [akka.tcp://EngineWarehouseActorSystem@compprod1:2552] >> >> [DEBUG] [08/20/2015 07:33:03.852] >> [EngineWarehouseActorSystem-akka.remote.default-remote-dispatcher-79] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2552/system/endpointManager/endpointWriter-akka.tcp%3A%2F%2FEngineWarehouseActorSystem%40compprod1%3A2552-13] >> Disassociated [akka.tcp://EngineWarehouseActorSystem@compprod2:2552] <- >> [akka.tcp://EngineWarehouseActorSystem@compprod1:2552] >> >> >> >> >> >> *But, after gaining back the associating, we have buffer drainage.* >> >> [DEBUG] [08/20/2015 07:33:03.855] >> [EngineWarehouseActorSystem-akka.remote.default-remote-dispatcher-147] >> [Remoting] Associated [akka.tcp://EngineWarehouseActorSystem@compprod2:2552] >> <- [akka.tcp://EngineWarehouseActorSystem@compprod1:2552] >> >> [DEBUG] [08/20/2015 07:33:03.856] >> [EngineWarehouseActorSystem-akka.remote.default-remote-dispatcher-147] >> [Remoting] Associated [akka.tcp://EngineWarehouseActorSystem@compprod2:2552] >> <- [akka.tcp://EngineWarehouseActorSystem@compprod1:2551] >> >> [DEBUG] [08/20/2015 07:33:03.889] >> [EngineWarehouseActorSystem-akka.remote.default-remote-dispatcher-79] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FEngineWarehouseActorSystem%40compprod1%3A2551-0/endpointWriter] >> Associated [akka.tcp://EngineWarehouseActorSystem@compprod2:2552] -> >> [akka.tcp://EngineWarehouseActorSystem@compprod1:2551] >> >> [DEBUG] [08/20/2015 07:33:03.890] >> [EngineWarehouseActorSystem-akka.remote.default-remote-dispatcher-79] >> [akka.tcp://EngineWarehouseActorSystem@compprod2 >> :2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FEngineWarehouseActorSystem%40compprod1%3A2551-0/endpointWriter] >> Drained >> buffer with maxWriteCount: 50, fullBackoffCount: 1, smallBackoffCount: >> 0, noBackoffCount: 0 , adaptiveBackoff: 1000 >> >> [DEBUG] [08/20/2015 07:33:03.891] >> [EngineWarehouseActorSystem-akka.remote.default-remote-dispatcher-147] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FEngineWarehouseActorSystem%40compprod1%3A2552-2/endpointWriter] >> Associated [akka.tcp://EngineWarehouseActorSystem@compprod2:2552] -> >> [akka.tcp://EngineWarehouseActorSystem@compprod1:2552] >> >> [DEBUG] [08/20/2015 07:33:03.892] >> [EngineWarehouseActorSystem-akka.remote.default-remote-dispatcher-147] >> [akka.tcp://EngineWarehouseActorSystem@compprod2 >> :2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FEngineWarehouseActorSystem%40compprod1%3A2552-2/endpointWriter] >> Drained >> buffer with maxWriteCount: 50, fullBackoffCount: 1, smallBackoffCount: >> 0, noBackoffCount: 0 , adaptiveBackoff: 1000 >> >> [DEBUG] [08/20/2015 07:33:03.912] >> [EngineWarehouseActorSystem-akka.remote.default-remote-dispatcher-79] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FEngineWarehouseActorSystem%40compprod2%3A2551-8/endpointWriter] >> Associated [akka.tcp://EngineWarehouseActorSystem@compprod2:2552] -> >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2551] >> >> [DEBUG] [08/20/2015 07:33:03.912] >> [EngineWarehouseActorSystem-akka.remote.default-remote-dispatcher-79] >> [akka.tcp://EngineWarehouseActorSystem@compprod2 >> :2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FEngineWarehouseActorSystem%40compprod2%3A2551-8/endpointWriter] >> Drained >> buffer with maxWriteCount: 50, fullBackoffCount: 1, smallBackoffCount: >> 0, noBackoffCount: 0 , adaptiveBackoff: 1000 >> >> [DEBUG] [08/20/2015 07:33:03.938] >> [EngineWarehouseActorSystem-akka.remote.default-remote-dispatcher-147] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2552/system/endpointManager/endpointWriter-akka.tcp%3A%2F%2FEngineWarehouseActorSystem%40compprod1%3A2551-10] >> Disassociated [akka.tcp://EngineWarehouseActorSystem@compprod2:2552] <- >> [akka.tcp://EngineWarehouseActorSystem@compprod1:2551] >> >> [DEBUG] [08/20/2015 07:33:03.938] >> [EngineWarehouseActorSystem-akka.remote.default-remote-dispatcher-79] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2552/system/endpointManager/endpointWriter-akka.tcp%3A%2F%2FEngineWarehouseActorSystem%40compprod1%3A2551-10] >> Disassociated [akka.tcp://EngineWarehouseActorSystem@compprod2:2552] <- >> [akka.tcp://EngineWarehouseActorSystem@compprod1:2551] >> >> >> >> >> >> *After that Everything is normal again* >> >> [DEBUG] [08/20/2015 07:33:09.054] >> [EngineWarehouseActorSystem-akka.actor.default-dispatcher-87] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2552/system/cluster/core/daemon/heartbeatSender] >> Cluster Node [akka.tcp://EngineWarehouseActorSystem@compprod2:2552] - >> Heartbeat to [akka.tcp://EngineWarehouseActorSystem@compprod1:2551] >> >> [DEBUG] [08/20/2015 07:33:09.054] >> [EngineWarehouseActorSystem-akka.actor.default-dispatcher-87] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2552/system/cluster/core/daemon/heartbeatSender] >> Cluster Node [akka.tcp://EngineWarehouseActorSystem@compprod2:2552] - >> Heartbeat to [akka.tcp://EngineWarehouseActorSystem@compprod1:2552] >> >> [DEBUG] [08/20/2015 07:33:09.054] >> [EngineWarehouseActorSystem-akka.actor.default-dispatcher-87] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2552/system/cluster/core/daemon/heartbeatSender] >> Cluster Node [akka.tcp://EngineWarehouseActorSystem@compprod2:2552] - >> Heartbeat to [akka.tcp://EngineWarehouseActorSystem@compprod2:2551] >> >> [DEBUG] [08/20/2015 07:33:09.056] >> [EngineWarehouseActorSystem-akka.actor.default-dispatcher-89] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2552/system/cluster/core/daemon/heartbeatSender] >> Cluster Node [akka.tcp://EngineWarehouseActorSystem@compprod2:2552] - >> Heartbeat response from [akka.tcp://EngineWarehouseActorSystem@compprod1 >> :2551] >> >> [DEBUG] [08/20/2015 07:33:09.056] >> [EngineWarehouseActorSystem-akka.actor.default-dispatcher-89] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2552/system/cluster/core/daemon/heartbeatSender] >> Cluster Node [akka.tcp://EngineWarehouseActorSystem@compprod2:2552] - >> Heartbeat response from [akka.tcp://EngineWarehouseActorSystem@compprod2 >> :2551] >> >> [DEBUG] [08/20/2015 07:33:09.056] >> [EngineWarehouseActorSystem-akka.actor.default-dispatcher-87] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2552/system/cluster/core/daemon/heartbeatSender] >> Cluster Node [akka.tcp://EngineWarehouseActorSystem@compprod2:2552] - >> Heartbeat response from [akka.tcp://EngineWarehouseActorSystem@compprod1 >> :2552] >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> *Meanwhile in Warehouse:* >> >> >> >> *Everything is normal so far until* >> >> [DEBUG] [08/20/2015 07:32:29.023] >> [EngineWarehouseActorSystem-akka.actor.default-dispatcher-25] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2551/system/cluster/core/daemon/heartbeatSender] >> Cluster Node [akka.tcp://EngineWarehouseActorSystem@compprod2:2551] - >> Heartbeat to [akka.tcp://EngineWarehouseActorSystem@compprod2:2552] >> >> [DEBUG] [08/20/2015 07:32:29.024] >> [EngineWarehouseActorSystem-akka.actor.default-dispatcher-25] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2551/system/cluster/core/daemon/heartbeatSender] >> Cluster Node [akka.tcp://EngineWarehouseActorSystem@compprod2:2551] - >> Heartbeat to [akka.tcp://EngineWarehouseActorSystem@compprod1:2551] >> >> [DEBUG] [08/20/2015 07:32:29.024] >> [EngineWarehouseActorSystem-akka.actor.default-dispatcher-25] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2551/system/cluster/core/daemon/heartbeatSender] >> Cluster Node [akka.tcp://EngineWarehouseActorSystem@compprod2:2551] - >> Heartbeat to [akka.tcp://EngineWarehouseActorSystem@compprod1:2552] >> >> [DEBUG] [08/20/2015 07:32:29.025] >> [EngineWarehouseActorSystem-akka.actor.default-dispatcher-15] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2551/system/cluster/core/daemon/heartbeatSender] >> Cluster Node [akka.tcp://EngineWarehouseActorSystem@compprod2:2551] - >> Heartbeat response from [akka.tcp://EngineWarehouseActorSystem@compprod2 >> :2552] >> >> [DEBUG] [08/20/2015 07:32:29.025] >> [EngineWarehouseActorSystem-akka.actor.default-dispatcher-15] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2551/system/cluster/core/daemon/heartbeatSender] >> Cluster Node [akka.tcp://EngineWarehouseActorSystem@compprod2:2551] - >> Heartbeat response from [akka.tcp://EngineWarehouseActorSystem@compprod1 >> :2551] >> >> [DEBUG] [08/20/2015 07:32:29.025] >> [EngineWarehouseActorSystem-akka.actor.default-dispatcher-25] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2551/system/cluster/core/daemon/heartbeatSender] >> Cluster Node [akka.tcp://EngineWarehouseActorSystem@compprod2:2551] - >> Heartbeat response from [akka.tcp://EngineWarehouseActorSystem@compprod1 >> :2552] >> >> >> >> *Engine is running 32 seconds GC pause now* >> >> [DEBUG] [08/20/2015 07:32:40.035] >> [EngineWarehouseActorSystem-akka.actor.default-dispatcher-21] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2551/system/cluster/core/daemon/heartbeatSender] >> Cluster Node [akka.tcp://EngineWarehouseActorSystem@compprod2:2551] - >> Heartbeat to [akka.tcp://EngineWarehouseActorSystem@compprod2:2552] >> <-First heartbeat req w/o reply >> >> [DEBUG] [08/20/2015 07:32:40.035] >> [EngineWarehouseActorSystem-akka.actor.default-dispatcher-21] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2551/system/cluster/core/daemon/heartbeatSender] >> Cluster Node [akka.tcp://EngineWarehouseActorSystem@compprod2:2551] - >> Heartbeat to [akka.tcp://EngineWarehouseActorSystem@compprod1:2551] >> >> [DEBUG] [08/20/2015 07:32:40.035] >> [EngineWarehouseActorSystem-akka.actor.default-dispatcher-21] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2551/system/cluster/core/daemon/heartbeatSender] >> Cluster Node [akka.tcp://EngineWarehouseActorSystem@compprod2:2551] - >> Heartbeat to [akka.tcp://EngineWarehouseActorSystem@compprod1:2552] >> >> [DEBUG] [08/20/2015 07:32:40.036] >> [EngineWarehouseActorSystem-akka.actor.default-dispatcher-20] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2551/system/cluster/core/daemon/heartbeatSender] >> Cluster Node [akka.tcp://EngineWarehouseActorSystem@compprod2:2551] - >> Heartbeat response from [akka.tcp://EngineWarehouseActorSystem@compprod1 >> :2551] >> >> [DEBUG] [08/20/2015 07:32:40.037] >> [EngineWarehouseActorSystem-akka.actor.default-dispatcher-21] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2551/system/cluster/core/daemon/heartbeatSender] >> Cluster Node [akka.tcp://EngineWarehouseActorSystem@compprod2:2551] - >> Heartbeat response from [akka.tcp://EngineWarehouseActorSystem@compprod1 >> :2552] >> >> >> >> [DEBUG] [08/20/2015 07:32:51.033] >> [EngineWarehouseActorSystem-akka.actor.default-dispatcher-30] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2551/system/cluster/core/daemon/heartbeatSender] >> Cluster Node [akka.tcp://EngineWarehouseActorSystem@compprod2:2551] - >> Heartbeat to [akka.tcp://EngineWarehouseActorSystem@compprod2:2552] >> <-2’nd heartbeat req w/o reply >> >> [DEBUG] [08/20/2015 07:32:51.033] >> [EngineWarehouseActorSystem-akka.actor.default-dispatcher-30] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2551/system/cluster/core/daemon/heartbeatSender] >> Cluster Node [akka.tcp://EngineWarehouseActorSystem@compprod2:2551] - >> Heartbeat to [akka.tcp://EngineWarehouseActorSystem@compprod1:2551] >> >> [DEBUG] [08/20/2015 07:32:51.033] >> [EngineWarehouseActorSystem-akka.actor.default-dispatcher-30] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2551/system/cluster/core/daemon/heartbeatSender] >> Cluster Node [akka.tcp://EngineWarehouseActorSystem@compprod2:2551] - >> Heartbeat to [akka.tcp://EngineWarehouseActorSystem@compprod1:2552] >> >> [DEBUG] [08/20/2015 07:32:51.034] >> [EngineWarehouseActorSystem-akka.actor.default-dispatcher-20] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2551/system/cluster/core/daemon/heartbeatSender] >> Cluster Node [akka.tcp://EngineWarehouseActorSystem@compprod2:2551] - >> Heartbeat response from [akka.tcp://EngineWarehouseActorSystem@compprod1 >> :2551] >> >> [DEBUG] [08/20/2015 07:32:51.035] >> [EngineWarehouseActorSystem-akka.actor.default-dispatcher-30] >> [akka.tcp://EngineWarehouseActorSystem@compprod2:2551/system/cluster/core/daemon/heartbeatSender] >> Cluster Node [akka.tcp://EngineWarehouseActorSystem@compprod2:2551] - >> Heartbeat response from [akka.tcp://Eng >> >> ... > > -- > >>>>>>>>>> Read the docs: http://akka.io/docs/ > >>>>>>>>>> Check the FAQ: > http://doc.akka.io/docs/akka/current/additional/faq.html > >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user > --- > You received this message because you are subscribed to the Google Groups > "Akka User List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/akka-user. > For more options, visit https://groups.google.com/d/optout. > -- /Patrik -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
