Hi Patrik, Thank you very much for your reply.
I enabled the Akka remote logging and ran both of my nodes. Attached here the logs for your perusal. Any findings/pointers are highly appreciated. On Tuesday, 21 June 2016 12:36:10 UTC-7, Patrik Nordwall wrote: > > Try to run with the remote debug logging enabled. Described in logging > section of documentation. > > I'm not sure where the port number 54947 comes from but we might be able > to see that if we look at full debug logs. > Today I got a different port number at this line of the log file: WARN | 2016-06-22 20:00:03 | [ClusterSystem-akka.actor.default-dispatcher-17] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-8, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:43209/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A42282-207, akkaTimestamp=20:00:03.446UTC] - Association with remote system [akka.tcp://ClusterSystem@hostname1:42282] has failed, address is now gated for [5000] ms. Reason: [Association failed with [akka.tcp://ClusterSystem@hostname1:42282]] Caused by: [Connection refused: hostname1/xxx.xxx.xxx.xxx:42282] > /Patrik > tis 21 juni 2016 kl. 21:11 skrev Manoj Santhakumaran <[email protected] > <javascript:>>: > >> >> Today's update: >> >> I tried setting the bind-hostname also: >> >> akka.remote.netty.tcp.bind-hostname = 0.0.0.0 >> >> But no luck !! >> >> I verified the connectivity between two docker containers, from inside >> container. >> >> telnet hostname1 2551 (From hostname2 works) >> telnet hostname2 2552 (From hostname1 works) >> host hostname1 (From hostname2 works) >> host hostname2 (From hostname1 works) >> >> I'm using Akka 2.4.4 version. >> >> Appreciate if anyone have any idea/pointers what could be going wrong >> here? >> >> >> Thank you very much. >> Manoj >> >> >> On Tuesday, 21 June 2016 01:36:02 UTC-7, Manoj Santhakumaran wrote: >>> >>> Hi, >>> >>> I'm working on a Play/Akka application with cluster system of two nodes, >>> and each node is deployed in docker container in two different hosts. >>> >>> My log files are filled up with following message, which comes every >>> 5/10 milli seconds. >>> >>> WARN | 2016-06-21 01:00:03 | >>> [ClusterSystem-akka.actor.default-dispatcher-21] ? - [?] - Association with >>> remote system [akka.tcp://ClusterSystem@hostname2:54797] has failed, >>> address is now gated for [5000] ms. Reason: [Association failed with >>> [akka.tcp://ClusterSystem@hostname2:54797]] Caused by: [Connection refused: >>> hostname2/xxx.xxx.xxx.xxx:54797] >>> >>> Can anyone help me to understand and fix this issue? >>> >>> My application.conf is as follows: >>> >>> akka { >>> actor.provider = "akka.cluster.ClusterActorRefProvider" >>> ... >>> cluster { >>> seed-nodes = [ >>> "akka.tcp://ClusterSystem@hostname1:2551", >>> "akka.tcp://ClusterSystem@hostname2:2552"] >>> } >>> ... >>> } >>> >>> And I'm passing port and host name as JVM args for both of the instances >>> (Eg: -Dakka.remote.netty.tcp.port=2551/2552 >>> -Dakka.remote.netty.tcp.hostname=hostname1/hostname2) >>> >>> I verified the network connectivity between these two hosts using "ping" >>> command inside docker container. There was no packet loss and looked fine. >>> >>> Thank you very much. >>> Manoj >>> >>> -- >> >>>>>>>>>> Read the docs: http://akka.io/docs/ >> >>>>>>>>>> Check the FAQ: >> http://doc.akka.io/docs/akka/current/additional/faq.html >> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user >> --- >> You received this message because you are subscribed to the Google Groups >> "Akka User List" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> To post to this group, send email to [email protected] >> <javascript:>. >> Visit this group at https://groups.google.com/group/akka-user. >> For more options, visit https://groups.google.com/d/optout. >> > Thank you very much. Manoj -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
DEBUG | 2016-06-22 20:00:00 | [ClusterSystem-akka.actor.default-dispatcher-17] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-6, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:00.019UTC] - received local message RemoteMessage: [ActorSelectionMessage(GossipStatus(UniqueAddress(akka.tcp://ClusterSystem@hostname1:2551,598411574),VectorClock(0674e80a1aa6eaa2990d3dab1b9beedc -> 1, b7a6f622639ec0187ab601337796f207 -> 7, bec77a033021fc1ab6448cb0c1af53fd -> 1, e191b0013b1bc705772a35b0e9cdbca3 -> 1)),Vector(system, cluster, core, daemon),false)] to [Actor[akka://ClusterSystem/]]<+[akka://ClusterSystem/] from [Actor[akka.tcp://ClusterSystem@hostname1:2551/system/cluster/core/daemon#1532363050]()] DEBUG | 2016-06-22 20:00:00 | [ClusterSystem-akka.actor.default-dispatcher-17] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-5, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:00.073UTC] - received local message RemoteMessage: [ActorSelectionMessage(Status(Map(akka.tcp://ClusterSystem@hostname1:2551 -> 1466624412169)),Vector(system, distributedPubSubMediator),false)] to [Actor[akka://ClusterSystem/]]<+[akka://ClusterSystem/] from [Actor[akka.tcp://ClusterSystem@hostname1:2551/system/distributedPubSubMediator#-379718827]()] DEBUG | 2016-06-22 20:00:00 | [ClusterSystem-akka.actor.default-dispatcher-29] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-9, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:43209/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:00.089UTC] - received local message RemoteMessage: [ActorSelectionMessage(Heartbeat(akka.tcp://ClusterSystem@hostname1:2551),Vector(system, cluster, heartbeatReceiver),false)] to [Actor[akka://ClusterSystem/]]<+[akka://ClusterSystem/] from [Actor[akka.tcp://ClusterSystem@hostname1:2551/system/cluster/core/daemon/heartbeatSender#-1740097823]()] DEBUG | 2016-06-22 20:00:00 | [ClusterSystem-akka.actor.default-dispatcher-14] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-5, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:00.089UTC] - received local message RemoteMessage: [ActorSelectionMessage(Heartbeat(akka.tcp://ClusterSystem@hostname1:2551),Vector(system, cluster, heartbeatReceiver),false)] to [Actor[akka://ClusterSystem/]]<+[akka://ClusterSystem/] from [Actor[akka.tcp://ClusterSystem@hostname1:2551/system/cluster/core/daemon/heartbeatSender#-1740097823]()] DEBUG | 2016-06-22 20:00:00 | [ClusterSystem-akka.actor.default-dispatcher-29] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-9, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:43209/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:00.089UTC] - sending message RemoteMessage: [HeartbeatRsp(UniqueAddress(akka.tcp://ClusterSystem@hostname2:43209,-951268098))] to [Actor[akka.tcp://ClusterSystem@hostname1:2551/system/cluster/core/daemon/heartbeatSender#-1740097823]]<+[akka.tcp://ClusterSystem@hostname1:2551/system/cluster/core/daemon/heartbeatSender] from [Actor[akka://ClusterSystem/system/cluster/heartbeatReceiver#-787172203]] DEBUG | 2016-06-22 20:00:00 | [ClusterSystem-akka.actor.default-dispatcher-16] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-5, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:00.089UTC] - sending message RemoteMessage: [HeartbeatRsp(UniqueAddress(akka.tcp://ClusterSystem@hostname2:2552,235402593))] to [Actor[akka.tcp://ClusterSystem@hostname1:2551/system/cluster/core/daemon/heartbeatSender#-1740097823]]<+[akka.tcp://ClusterSystem@hostname1:2551/system/cluster/core/daemon/heartbeatSender] from [Actor[akka://ClusterSystem/system/cluster/heartbeatReceiver#-670753192]] DEBUG | 2016-06-22 20:00:00 | [ClusterSystem-akka.actor.default-dispatcher-20] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-8, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:43209/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:00.384UTC] - sending message RemoteMessage: [ActorSelectionMessage(akka.cluster.GossipEnvelope@7ea6e228,Vector(system, cluster, core, daemon),false)] to [Actor[akka.tcp://ClusterSystem@hostname1:2551/]]<+[akka.tcp://ClusterSystem@hostname1:2551/] from [Actor[akka://ClusterSystem/system/cluster/core/daemon#-1375582141]] DEBUG | 2016-06-22 20:00:00 | [ClusterSystem-akka.actor.default-dispatcher-20] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-8, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:43209/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:00.384UTC] - sending message RemoteMessage: [ActorSelectionMessage(Status(Map(akka.tcp://ClusterSystem@hostname1:2551 -> 1466624412169)),Vector(system, distributedPubSubMediator),false)] to [Actor[akka.tcp://ClusterSystem@hostname1:2551/]]<+[akka.tcp://ClusterSystem@hostname1:2551/] from [Actor[akka://ClusterSystem/system/distributedPubSubMediator#78424083]] DEBUG | 2016-06-22 20:00:00 | [ClusterSystem-akka.actor.default-dispatcher-16] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-5, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname2%3A43209-3/endpointWriter, akkaTimestamp=20:00:00.396UTC] - sending message RemoteMessage: [ActorSelectionMessage(Heartbeat(akka.tcp://ClusterSystem@hostname2:2552),Vector(system, cluster, heartbeatReceiver),false)] to [Actor[akka.tcp://ClusterSystem@hostname2:43209/]]<+[akka.tcp://ClusterSystem@hostname2:43209/] from [Actor[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender#-132561496]] DEBUG | 2016-06-22 20:00:00 | [ClusterSystem-akka.actor.default-dispatcher-16] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-6, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:00.396UTC] - sending message RemoteMessage: [ActorSelectionMessage(Heartbeat(akka.tcp://ClusterSystem@hostname2:2552),Vector(system, cluster, heartbeatReceiver),false)] to [Actor[akka.tcp://ClusterSystem@hostname1:2551/]]<+[akka.tcp://ClusterSystem@hostname1:2551/] from [Actor[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender#-132561496]] DEBUG | 2016-06-22 20:00:00 | [ClusterSystem-akka.actor.default-dispatcher-16] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-5, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A42282-4/endpointWriter, akkaTimestamp=20:00:00.396UTC] - sending message RemoteMessage: [ActorSelectionMessage(Heartbeat(akka.tcp://ClusterSystem@hostname2:2552),Vector(system, cluster, heartbeatReceiver),false)] to [Actor[akka.tcp://ClusterSystem@hostname1:42282/]]<+[akka.tcp://ClusterSystem@hostname1:42282/] from [Actor[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender#-132561496]] DEBUG | 2016-06-22 20:00:00 | [ClusterSystem-akka.actor.default-dispatcher-20] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-8, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:43209/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname2%3A2552-1/endpointWriter, akkaTimestamp=20:00:00.396UTC] - received local message RemoteMessage: [ActorSelectionMessage(Heartbeat(akka.tcp://ClusterSystem@hostname2:2552),Vector(system, cluster, heartbeatReceiver),false)] to [Actor[akka://ClusterSystem/]]<+[akka://ClusterSystem/] from [Actor[akka.tcp://ClusterSystem@hostname2:2552/system/cluster/core/daemon/heartbeatSender#-132561496]()] DEBUG | 2016-06-22 20:00:00 | [ClusterSystem-akka.actor.default-dispatcher-20] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-8, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:43209/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname2%3A2552-1/endpointWriter, akkaTimestamp=20:00:00.396UTC] - sending message RemoteMessage: [HeartbeatRsp(UniqueAddress(akka.tcp://ClusterSystem@hostname2:43209,-951268098))] to [Actor[akka.tcp://ClusterSystem@hostname2:2552/system/cluster/core/daemon/heartbeatSender#-132561496]]<+[akka.tcp://ClusterSystem@hostname2:2552/system/cluster/core/daemon/heartbeatSender] from [Actor[akka://ClusterSystem/system/cluster/heartbeatReceiver#-787172203]] DEBUG | 2016-06-22 20:00:00 | [ClusterSystem-akka.actor.default-dispatcher-16] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-5, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname2%3A43209-3/endpointWriter, akkaTimestamp=20:00:00.397UTC] - received local message RemoteMessage: [HeartbeatRsp(UniqueAddress(akka.tcp://ClusterSystem@hostname2:43209,-951268098))] to [Actor[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender#-132561496]]<+[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender] from [Actor[akka.tcp://ClusterSystem@hostname2:43209/system/cluster/heartbeatReceiver#-787172203]()] DEBUG | 2016-06-22 20:00:00 | [ClusterSystem-akka.actor.default-dispatcher-16] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-5, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A42282-4/endpointWriter, akkaTimestamp=20:00:00.403UTC] - received local message RemoteMessage: [HeartbeatRsp(UniqueAddress(akka.tcp://ClusterSystem@hostname1:42282,1853147177))] to [Actor[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender#-132561496]]<+[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender] from [Actor[akka.tcp://ClusterSystem@hostname1:42282/system/cluster/heartbeatReceiver#1799437957]()] DEBUG | 2016-06-22 20:00:00 | [ClusterSystem-akka.actor.default-dispatcher-16] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-6, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:00.403UTC] - received local message RemoteMessage: [HeartbeatRsp(UniqueAddress(akka.tcp://ClusterSystem@hostname1:2551,598411574))] to [Actor[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender#-132561496]]<+[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender] from [Actor[akka.tcp://ClusterSystem@hostname1:2551/system/cluster/heartbeatReceiver#-2035426665]()] DEBUG | 2016-06-22 20:00:00 | [ClusterSystem-akka.actor.default-dispatcher-20] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-9, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:43209/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname2%3A2552-1/endpointWriter, akkaTimestamp=20:00:00.443UTC] - sending message RemoteMessage: [ActorSelectionMessage(Heartbeat(akka.tcp://ClusterSystem@hostname2:43209),Vector(system, cluster, heartbeatReceiver),false)] to [Actor[akka.tcp://ClusterSystem@hostname2:2552/]]<+[akka.tcp://ClusterSystem@hostname2:2552/] from [Actor[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender#-271688942]] DEBUG | 2016-06-22 20:00:00 | [ClusterSystem-akka.actor.default-dispatcher-20] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-8, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:43209/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:00.443UTC] - sending message RemoteMessage: [ActorSelectionMessage(Heartbeat(akka.tcp://ClusterSystem@hostname2:43209),Vector(system, cluster, heartbeatReceiver),false)] to [Actor[akka.tcp://ClusterSystem@hostname1:2551/]]<+[akka.tcp://ClusterSystem@hostname1:2551/] from [Actor[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender#-271688942]] DEBUG | 2016-06-22 20:00:00 | [ClusterSystem-akka.actor.default-dispatcher-16] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-5, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname2%3A43209-3/endpointWriter, akkaTimestamp=20:00:00.443UTC] - received local message RemoteMessage: [ActorSelectionMessage(Heartbeat(akka.tcp://ClusterSystem@hostname2:43209),Vector(system, cluster, heartbeatReceiver),false)] to [Actor[akka://ClusterSystem/]]<+[akka://ClusterSystem/] from [Actor[akka.tcp://ClusterSystem@hostname2:43209/system/cluster/core/daemon/heartbeatSender#-271688942]()] DEBUG | 2016-06-22 20:00:00 | [ClusterSystem-akka.actor.default-dispatcher-16] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-5, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname2%3A43209-3/endpointWriter, akkaTimestamp=20:00:00.443UTC] - sending message RemoteMessage: [HeartbeatRsp(UniqueAddress(akka.tcp://ClusterSystem@hostname2:2552,235402593))] to [Actor[akka.tcp://ClusterSystem@hostname2:43209/system/cluster/core/daemon/heartbeatSender#-271688942]]<+[akka.tcp://ClusterSystem@hostname2:43209/system/cluster/core/daemon/heartbeatSender] from [Actor[akka://ClusterSystem/system/cluster/heartbeatReceiver#-670753192]] DEBUG | 2016-06-22 20:00:00 | [ClusterSystem-akka.actor.default-dispatcher-20] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-9, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:43209/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname2%3A2552-1/endpointWriter, akkaTimestamp=20:00:00.443UTC] - received local message RemoteMessage: [HeartbeatRsp(UniqueAddress(akka.tcp://ClusterSystem@hostname2:2552,235402593))] to [Actor[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender#-271688942]]<+[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender] from [Actor[akka.tcp://ClusterSystem@hostname2:2552/system/cluster/heartbeatReceiver#-670753192]()] DEBUG | 2016-06-22 20:00:00 | [ClusterSystem-akka.actor.default-dispatcher-20] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-9, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:43209/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:00.446UTC] - received local message RemoteMessage: [HeartbeatRsp(UniqueAddress(akka.tcp://ClusterSystem@hostname1:2551,598411574))] to [Actor[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender#-271688942]]<+[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender] from [Actor[akka.tcp://ClusterSystem@hostname1:2551/system/cluster/heartbeatReceiver#-2035426665]()] DEBUG | 2016-06-22 20:00:00 | [ClusterSystem-akka.actor.default-dispatcher-16] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-5, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A42282-4/endpointWriter, akkaTimestamp=20:00:00.505UTC] - received local message RemoteMessage: [ActorSelectionMessage(akka.cluster.GossipEnvelope@6e211d7b,Vector(system, cluster, core, daemon),false)] to [Actor[akka://ClusterSystem/]]<+[akka://ClusterSystem/] from [Actor[akka.tcp://ClusterSystem@hostname1:42282/system/cluster/core/daemon#-1814189187]()] DEBUG | 2016-06-22 20:00:00 | [ClusterSystem-akka.actor.default-dispatcher-16] ? - [sourceThread=ClusterSystem-akka.actor.default-dispatcher-14, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/cluster/core/daemon, akkaTimestamp=20:00:00.505UTC] - Cluster Node [akka.tcp://ClusterSystem@hostname2:2552] - Receiving gossip from [UniqueAddress(akka.tcp://ClusterSystem@hostname1:42282,1853147177)] DEBUG | 2016-06-22 20:00:00 | [ClusterSystem-akka.actor.default-dispatcher-17] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-6, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:00.636UTC] - sending message RemoteMessage: [ActorSelectionMessage(GossipStatus(UniqueAddress(akka.tcp://ClusterSystem@hostname2:2552,235402593),VectorClock(0674e80a1aa6eaa2990d3dab1b9beedc -> 1, b7a6f622639ec0187ab601337796f207 -> 7, bec77a033021fc1ab6448cb0c1af53fd -> 1, e191b0013b1bc705772a35b0e9cdbca3 -> 1)),Vector(system, cluster, core, daemon),false)] to [Actor[akka.tcp://ClusterSystem@hostname1:2551/]]<+[akka.tcp://ClusterSystem@hostname1:2551/] from [Actor[akka://ClusterSystem/system/cluster/core/daemon#-150228074]] DEBUG | 2016-06-22 20:00:00 | [ClusterSystem-akka.actor.default-dispatcher-17] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-6, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A42282-4/endpointWriter, akkaTimestamp=20:00:00.696UTC] - sending message RemoteMessage: [ActorSelectionMessage(Status(Map(akka.tcp://ClusterSystem@hostname1:2551 -> 1466624412169)),Vector(system, distributedPubSubMediator),false)] to [Actor[akka.tcp://ClusterSystem@hostname1:42282/]]<+[akka.tcp://ClusterSystem@hostname1:42282/] from [Actor[akka://ClusterSystem/system/distributedPubSubMediator#638476632]] DEBUG | 2016-06-22 20:00:00 | [ClusterSystem-akka.actor.default-dispatcher-17] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-5, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2Fsnmp-WorkerSystem%40127.0.0.1%3A44183-13/endpointWriter, akkaTimestamp=20:00:00.821UTC] - received local message RemoteMessage: [Heartbeat] to [Actor[akka://ClusterSystem/system/receptionist#-1830908093]]<+[akka://ClusterSystem/system/receptionist] from [Actor[akka.tcp://[email protected]:44183/user/clusterClient#1180082629]()] DEBUG | 2016-06-22 20:00:00 | [ClusterSystem-akka.actor.default-dispatcher-17] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-5, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2Fsnmp-WorkerSystem%40127.0.0.1%3A44183-13/endpointWriter, akkaTimestamp=20:00:00.821UTC] - sending message RemoteMessage: [HeartbeatRsp] to [Actor[akka.tcp://[email protected]:44183/user/clusterClient#1180082629]]<+[akka.tcp://[email protected]:44183/user/clusterClient] from [Actor[akka://ClusterSystem/system/receptionist#-1830908093]] DEBUG | 2016-06-22 20:00:00 | [ClusterSystem-akka.actor.default-dispatcher-17] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-6, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A42282-4/endpointWriter, akkaTimestamp=20:00:00.864UTC] - received local message RemoteMessage: [ActorSelectionMessage(Heartbeat(akka.tcp://ClusterSystem@hostname1:42282),Vector(system, cluster, heartbeatReceiver),false)] to [Actor[akka://ClusterSystem/]]<+[akka://ClusterSystem/] from [Actor[akka.tcp://ClusterSystem@hostname1:42282/system/cluster/core/daemon/heartbeatSender#1072477085]()] DEBUG | 2016-06-22 20:00:00 | [ClusterSystem-akka.actor.default-dispatcher-17] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-6, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A42282-4/endpointWriter, akkaTimestamp=20:00:00.864UTC] - sending message RemoteMessage: [HeartbeatRsp(UniqueAddress(akka.tcp://ClusterSystem@hostname2:2552,235402593))] to [Actor[akka.tcp://ClusterSystem@hostname1:42282/system/cluster/core/daemon/heartbeatSender#1072477085]]<+[akka.tcp://ClusterSystem@hostname1:42282/system/cluster/core/daemon/heartbeatSender] from [Actor[akka://ClusterSystem/system/cluster/heartbeatReceiver#-670753192]] DEBUG | 2016-06-22 20:00:00 | [ClusterSystem-akka.actor.default-dispatcher-2] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-5, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:00.897UTC] - sending message RemoteMessage: [ActorSelectionMessage(MetricsGossipEnvelope(akka.tcp://ClusterSystem@hostname2:2552,MetricsGossip(Set(NodeMetrics(akka.tcp://ClusterSystem@hostname1:2551,1466625595253,Set(Metric(heap-memory-used,666528488,Some(EWMA(6.66528488E8,0.15910417746033534))), Metric(heap-memory-max,7460618240,None), Metric(processors,8,None), Metric(system-load-average,0.00732421875,None), Metric(heap-memory-committed,1200619520,Some(EWMA(1.20061952E9,0.15910417746033534))))), NodeMetrics(akka.tcp://ClusterSystem@hostname2:2552,1466625600897,Set(Metric(heap-memory-used,154272488,Some(EWMA(1.54272488E8,0.15910417746033534))), Metric(heap-memory-max,7460618240,None), Metric(processors,8,None), Metric(system-load-average,0.0068359375,None), Metric(heap-memory-committed,1372585984,Some(EWMA(1.372585984E9,0.15910417746033534))))))),false),Vector(system, cluster, metrics),false)] to [Actor[akka.tcp://ClusterSystem@hostname1:2551/]]<+[akka.tcp://ClusterSystem@hostname1:2551/] from [Actor[akka://ClusterSystem/system/cluster/metrics#1040761959]] DEBUG | 2016-06-22 20:00:00 | [ClusterSystem-akka.actor.default-dispatcher-2] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-5, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:00.898UTC] - received local message RemoteMessage: [ActorSelectionMessage(MetricsGossipEnvelope(akka.tcp://ClusterSystem@hostname1:2551,MetricsGossip(Set(NodeMetrics(akka.tcp://ClusterSystem@hostname1:2551,1466625598253,Set(Metric(heap-memory-used,671845048,Some(EWMA(6.71845048E8,0.15910417746033534))), Metric(heap-memory-max,7460618240,None), Metric(processors,8,None), Metric(system-load-average,0.00732421875,None), Metric(heap-memory-committed,1200619520,Some(EWMA(1.20061952E9,0.15910417746033534))))), NodeMetrics(akka.tcp://ClusterSystem@hostname2:2552,1466625600897,Set(Metric(heap-memory-used,154272488,Some(EWMA(1.54272488E8,0.15910417746033534))), Metric(heap-memory-max,7460618240,None), Metric(processors,8,None), Metric(system-load-average,0.0068359375,None), Metric(heap-memory-committed,1372585984,Some(EWMA(1.372585984E9,0.15910417746033534))))))),true),Vector(system, cluster, metrics),false)] to [Actor[akka://ClusterSystem/]]<+[akka://ClusterSystem/] from [Actor[akka.tcp://ClusterSystem@hostname1:2551/system/cluster/metrics#616453818]()] DEBUG | 2016-06-22 20:00:01 | [ClusterSystem-akka.actor.default-dispatcher-17] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-5, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:01.017UTC] - received local message RemoteMessage: [ActorSelectionMessage(GossipStatus(UniqueAddress(akka.tcp://ClusterSystem@hostname1:2551,598411574),VectorClock(0674e80a1aa6eaa2990d3dab1b9beedc -> 1, b7a6f622639ec0187ab601337796f207 -> 7, bec77a033021fc1ab6448cb0c1af53fd -> 1, e191b0013b1bc705772a35b0e9cdbca3 -> 1)),Vector(system, cluster, core, daemon),false)] to [Actor[akka://ClusterSystem/]]<+[akka://ClusterSystem/] from [Actor[akka.tcp://ClusterSystem@hostname1:2551/system/cluster/core/daemon#1532363050]()] DEBUG | 2016-06-22 20:00:01 | [ClusterSystem-akka.actor.default-dispatcher-17] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-6, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:01.067UTC] - received local message RemoteMessage: [ActorSelectionMessage(Status(Map(akka.tcp://ClusterSystem@hostname1:2551 -> 1466624412169)),Vector(system, distributedPubSubMediator),false)] to [Actor[akka://ClusterSystem/]]<+[akka://ClusterSystem/] from [Actor[akka.tcp://ClusterSystem@hostname1:2551/system/distributedPubSubMediator#-379718827]()] DEBUG | 2016-06-22 20:00:01 | [ClusterSystem-akka.actor.default-dispatcher-17] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-5, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:01.087UTC] - received local message RemoteMessage: [ActorSelectionMessage(Heartbeat(akka.tcp://ClusterSystem@hostname1:2551),Vector(system, cluster, heartbeatReceiver),false)] to [Actor[akka://ClusterSystem/]]<+[akka://ClusterSystem/] from [Actor[akka.tcp://ClusterSystem@hostname1:2551/system/cluster/core/daemon/heartbeatSender#-1740097823]()] DEBUG | 2016-06-22 20:00:01 | [ClusterSystem-akka.actor.default-dispatcher-17] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-5, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:01.088UTC] - sending message RemoteMessage: [HeartbeatRsp(UniqueAddress(akka.tcp://ClusterSystem@hostname2:2552,235402593))] to [Actor[akka.tcp://ClusterSystem@hostname1:2551/system/cluster/core/daemon/heartbeatSender#-1740097823]]<+[akka.tcp://ClusterSystem@hostname1:2551/system/cluster/core/daemon/heartbeatSender] from [Actor[akka://ClusterSystem/system/cluster/heartbeatReceiver#-670753192]] DEBUG | 2016-06-22 20:00:01 | [ClusterSystem-akka.actor.default-dispatcher-20] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-9, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:43209/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:01.088UTC] - received local message RemoteMessage: [ActorSelectionMessage(Heartbeat(akka.tcp://ClusterSystem@hostname1:2551),Vector(system, cluster, heartbeatReceiver),false)] to [Actor[akka://ClusterSystem/]]<+[akka://ClusterSystem/] from [Actor[akka.tcp://ClusterSystem@hostname1:2551/system/cluster/core/daemon/heartbeatSender#-1740097823]()] DEBUG | 2016-06-22 20:00:01 | [ClusterSystem-akka.actor.default-dispatcher-20] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-9, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:43209/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:01.088UTC] - sending message RemoteMessage: [HeartbeatRsp(UniqueAddress(akka.tcp://ClusterSystem@hostname2:43209,-951268098))] to [Actor[akka.tcp://ClusterSystem@hostname1:2551/system/cluster/core/daemon/heartbeatSender#-1740097823]]<+[akka.tcp://ClusterSystem@hostname1:2551/system/cluster/core/daemon/heartbeatSender] from [Actor[akka://ClusterSystem/system/cluster/heartbeatReceiver#-787172203]] DEBUG | 2016-06-22 20:00:01 | [ClusterSystem-akka.actor.default-dispatcher-17] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-5, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:01.257UTC] - received local message RemoteMessage: [ActorSelectionMessage(MetricsGossipEnvelope(akka.tcp://ClusterSystem@hostname1:2551,MetricsGossip(Set(NodeMetrics(akka.tcp://ClusterSystem@hostname1:2551,1466625598253,Set(Metric(heap-memory-used,671845048,Some(EWMA(6.71845048E8,0.15910417746033534))), Metric(heap-memory-max,7460618240,None), Metric(processors,8,None), Metric(system-load-average,0.00732421875,None), Metric(heap-memory-committed,1200619520,Some(EWMA(1.20061952E9,0.15910417746033534))))), NodeMetrics(akka.tcp://ClusterSystem@hostname2:2552,1466625600897,Set(Metric(heap-memory-used,154272488,Some(EWMA(1.54272488E8,0.15910417746033534))), Metric(heap-memory-max,7460618240,None), Metric(processors,8,None), Metric(system-load-average,0.0068359375,None), Metric(heap-memory-committed,1372585984,Some(EWMA(1.372585984E9,0.15910417746033534))))))),false),Vector(system, cluster, metrics),false)] to [Actor[akka://ClusterSystem/]]<+[akka://ClusterSystem/] from [Actor[akka.tcp://ClusterSystem@hostname1:2551/system/cluster/metrics#616453818]()] DEBUG | 2016-06-22 20:00:01 | [ClusterSystem-akka.actor.default-dispatcher-17] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-5, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:01.257UTC] - sending message RemoteMessage: [ActorSelectionMessage(MetricsGossipEnvelope(akka.tcp://ClusterSystem@hostname2:2552,MetricsGossip(Set(NodeMetrics(akka.tcp://ClusterSystem@hostname1:2551,1466625598253,Set(Metric(heap-memory-used,671845048,Some(EWMA(6.71845048E8,0.15910417746033534))), Metric(heap-memory-max,7460618240,None), Metric(processors,8,None), Metric(system-load-average,0.00732421875,None), Metric(heap-memory-committed,1200619520,Some(EWMA(1.20061952E9,0.15910417746033534))))), NodeMetrics(akka.tcp://ClusterSystem@hostname2:2552,1466625600897,Set(Metric(heap-memory-used,154272488,Some(EWMA(1.54272488E8,0.15910417746033534))), Metric(heap-memory-max,7460618240,None), Metric(processors,8,None), Metric(system-load-average,0.0068359375,None), Metric(heap-memory-committed,1372585984,Some(EWMA(1.372585984E9,0.15910417746033534))))))),true),Vector(system, cluster, metrics),false)] to [Actor[akka.tcp://ClusterSystem@hostname1:2551/]]<+[akka.tcp://ClusterSystem@hostname1:2551/] from [Actor[akka://ClusterSystem/system/cluster/metrics#1040761959]] DEBUG | 2016-06-22 20:00:01 | [ClusterSystem-akka.actor.default-dispatcher-27] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-9, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:43209/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:01.383UTC] - sending message RemoteMessage: [ActorSelectionMessage(akka.cluster.GossipEnvelope@353a0603,Vector(system, cluster, core, daemon),false)] to [Actor[akka.tcp://ClusterSystem@hostname1:2551/]]<+[akka.tcp://ClusterSystem@hostname1:2551/] from [Actor[akka://ClusterSystem/system/cluster/core/daemon#-1375582141]] DEBUG | 2016-06-22 20:00:01 | [ClusterSystem-akka.actor.default-dispatcher-27] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-8, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:43209/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname2%3A2552-1/endpointWriter, akkaTimestamp=20:00:01.383UTC] - sending message RemoteMessage: [ActorSelectionMessage(Status(Map(akka.tcp://ClusterSystem@hostname1:2551 -> 1466624412169)),Vector(system, distributedPubSubMediator),false)] to [Actor[akka.tcp://ClusterSystem@hostname2:2552/]]<+[akka.tcp://ClusterSystem@hostname2:2552/] from [Actor[akka://ClusterSystem/system/distributedPubSubMediator#78424083]] DEBUG | 2016-06-22 20:00:01 | [ClusterSystem-akka.actor.default-dispatcher-17] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-6, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname2%3A43209-3/endpointWriter, akkaTimestamp=20:00:01.384UTC] - received local message RemoteMessage: [ActorSelectionMessage(Status(Map(akka.tcp://ClusterSystem@hostname1:2551 -> 1466624412169)),Vector(system, distributedPubSubMediator),false)] to [Actor[akka://ClusterSystem/]]<+[akka://ClusterSystem/] from [Actor[akka.tcp://ClusterSystem@hostname2:43209/system/distributedPubSubMediator#78424083]()] DEBUG | 2016-06-22 20:00:01 | [ClusterSystem-akka.actor.default-dispatcher-2] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-5, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname2%3A43209-3/endpointWriter, akkaTimestamp=20:00:01.396UTC] - sending message RemoteMessage: [ActorSelectionMessage(Heartbeat(akka.tcp://ClusterSystem@hostname2:2552),Vector(system, cluster, heartbeatReceiver),false)] to [Actor[akka.tcp://ClusterSystem@hostname2:43209/]]<+[akka.tcp://ClusterSystem@hostname2:43209/] from [Actor[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender#-132561496]] DEBUG | 2016-06-22 20:00:01 | [ClusterSystem-akka.actor.default-dispatcher-2] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-6, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:01.396UTC] - sending message RemoteMessage: [ActorSelectionMessage(Heartbeat(akka.tcp://ClusterSystem@hostname2:2552),Vector(system, cluster, heartbeatReceiver),false)] to [Actor[akka.tcp://ClusterSystem@hostname1:2551/]]<+[akka.tcp://ClusterSystem@hostname1:2551/] from [Actor[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender#-132561496]] DEBUG | 2016-06-22 20:00:01 | [ClusterSystem-akka.actor.default-dispatcher-2] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-5, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A42282-4/endpointWriter, akkaTimestamp=20:00:01.396UTC] - sending message RemoteMessage: [ActorSelectionMessage(Heartbeat(akka.tcp://ClusterSystem@hostname2:2552),Vector(system, cluster, heartbeatReceiver),false)] to [Actor[akka.tcp://ClusterSystem@hostname1:42282/]]<+[akka.tcp://ClusterSystem@hostname1:42282/] from [Actor[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender#-132561496]] DEBUG | 2016-06-22 20:00:01 | [ClusterSystem-akka.actor.default-dispatcher-27] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-9, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:43209/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname2%3A2552-1/endpointWriter, akkaTimestamp=20:00:01.396UTC] - received local message RemoteMessage: [ActorSelectionMessage(Heartbeat(akka.tcp://ClusterSystem@hostname2:2552),Vector(system, cluster, heartbeatReceiver),false)] to [Actor[akka://ClusterSystem/]]<+[akka://ClusterSystem/] from [Actor[akka.tcp://ClusterSystem@hostname2:2552/system/cluster/core/daemon/heartbeatSender#-132561496]()] DEBUG | 2016-06-22 20:00:01 | [ClusterSystem-akka.actor.default-dispatcher-27] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-9, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:43209/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname2%3A2552-1/endpointWriter, akkaTimestamp=20:00:01.396UTC] - sending message RemoteMessage: [HeartbeatRsp(UniqueAddress(akka.tcp://ClusterSystem@hostname2:43209,-951268098))] to [Actor[akka.tcp://ClusterSystem@hostname2:2552/system/cluster/core/daemon/heartbeatSender#-132561496]]<+[akka.tcp://ClusterSystem@hostname2:2552/system/cluster/core/daemon/heartbeatSender] from [Actor[akka://ClusterSystem/system/cluster/heartbeatReceiver#-787172203]] DEBUG | 2016-06-22 20:00:01 | [ClusterSystem-akka.actor.default-dispatcher-2] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-5, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname2%3A43209-3/endpointWriter, akkaTimestamp=20:00:01.397UTC] - received local message RemoteMessage: [HeartbeatRsp(UniqueAddress(akka.tcp://ClusterSystem@hostname2:43209,-951268098))] to [Actor[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender#-132561496]]<+[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender] from [Actor[akka.tcp://ClusterSystem@hostname2:43209/system/cluster/heartbeatReceiver#-787172203]()] DEBUG | 2016-06-22 20:00:01 | [ClusterSystem-akka.actor.default-dispatcher-15] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-5, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:01.397UTC] - received local message RemoteMessage: [HeartbeatRsp(UniqueAddress(akka.tcp://ClusterSystem@hostname1:2551,598411574))] to [Actor[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender#-132561496]]<+[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender] from [Actor[akka.tcp://ClusterSystem@hostname1:2551/system/cluster/heartbeatReceiver#-2035426665]()] DEBUG | 2016-06-22 20:00:01 | [ClusterSystem-akka.actor.default-dispatcher-15] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-5, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A42282-4/endpointWriter, akkaTimestamp=20:00:01.397UTC] - received local message RemoteMessage: [HeartbeatRsp(UniqueAddress(akka.tcp://ClusterSystem@hostname1:42282,1853147177))] to [Actor[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender#-132561496]]<+[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender] from [Actor[akka.tcp://ClusterSystem@hostname1:42282/system/cluster/heartbeatReceiver#1799437957]()] DEBUG | 2016-06-22 20:00:01 | [ClusterSystem-akka.actor.default-dispatcher-27] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-9, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:43209/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:01.444UTC] - sending message RemoteMessage: [ActorSelectionMessage(Heartbeat(akka.tcp://ClusterSystem@hostname2:43209),Vector(system, cluster, heartbeatReceiver),false)] to [Actor[akka.tcp://ClusterSystem@hostname1:2551/]]<+[akka.tcp://ClusterSystem@hostname1:2551/] from [Actor[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender#-271688942]] DEBUG | 2016-06-22 20:00:01 | [ClusterSystem-akka.actor.default-dispatcher-27] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-8, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:43209/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname2%3A2552-1/endpointWriter, akkaTimestamp=20:00:01.444UTC] - sending message RemoteMessage: [ActorSelectionMessage(Heartbeat(akka.tcp://ClusterSystem@hostname2:43209),Vector(system, cluster, heartbeatReceiver),false)] to [Actor[akka.tcp://ClusterSystem@hostname2:2552/]]<+[akka.tcp://ClusterSystem@hostname2:2552/] from [Actor[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender#-271688942]] DEBUG | 2016-06-22 20:00:01 | [ClusterSystem-akka.actor.default-dispatcher-15] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-6, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname2%3A43209-3/endpointWriter, akkaTimestamp=20:00:01.444UTC] - received local message RemoteMessage: [ActorSelectionMessage(Heartbeat(akka.tcp://ClusterSystem@hostname2:43209),Vector(system, cluster, heartbeatReceiver),false)] to [Actor[akka://ClusterSystem/]]<+[akka://ClusterSystem/] from [Actor[akka.tcp://ClusterSystem@hostname2:43209/system/cluster/core/daemon/heartbeatSender#-271688942]()] DEBUG | 2016-06-22 20:00:01 | [ClusterSystem-akka.actor.default-dispatcher-15] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-6, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname2%3A43209-3/endpointWriter, akkaTimestamp=20:00:01.444UTC] - sending message RemoteMessage: [HeartbeatRsp(UniqueAddress(akka.tcp://ClusterSystem@hostname2:2552,235402593))] to [Actor[akka.tcp://ClusterSystem@hostname2:43209/system/cluster/core/daemon/heartbeatSender#-271688942]]<+[akka.tcp://ClusterSystem@hostname2:43209/system/cluster/core/daemon/heartbeatSender] from [Actor[akka://ClusterSystem/system/cluster/heartbeatReceiver#-670753192]] DEBUG | 2016-06-22 20:00:01 | [ClusterSystem-akka.actor.default-dispatcher-27] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-8, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:43209/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname2%3A2552-1/endpointWriter, akkaTimestamp=20:00:01.444UTC] - received local message RemoteMessage: [HeartbeatRsp(UniqueAddress(akka.tcp://ClusterSystem@hostname2:2552,235402593))] to [Actor[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender#-271688942]]<+[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender] from [Actor[akka.tcp://ClusterSystem@hostname2:2552/system/cluster/heartbeatReceiver#-670753192]()] DEBUG | 2016-06-22 20:00:01 | [ClusterSystem-akka.actor.default-dispatcher-27] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-8, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:43209/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:01.444UTC] - received local message RemoteMessage: [HeartbeatRsp(UniqueAddress(akka.tcp://ClusterSystem@hostname1:2551,598411574))] to [Actor[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender#-271688942]]<+[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender] from [Actor[akka.tcp://ClusterSystem@hostname1:2551/system/cluster/heartbeatReceiver#-2035426665]()] DEBUG | 2016-06-22 20:00:01 | [ClusterSystem-akka.actor.default-dispatcher-15] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-5, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A42282-4/endpointWriter, akkaTimestamp=20:00:01.503UTC] - received local message RemoteMessage: [ActorSelectionMessage(akka.cluster.GossipEnvelope@125e72f7,Vector(system, cluster, core, daemon),false)] to [Actor[akka://ClusterSystem/]]<+[akka://ClusterSystem/] from [Actor[akka.tcp://ClusterSystem@hostname1:42282/system/cluster/core/daemon#-1814189187]()] DEBUG | 2016-06-22 20:00:01 | [ClusterSystem-akka.actor.default-dispatcher-15] ? - [sourceThread=ClusterSystem-akka.actor.default-dispatcher-17, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/cluster/core/daemon, akkaTimestamp=20:00:01.504UTC] - Cluster Node [akka.tcp://ClusterSystem@hostname2:2552] - Receiving gossip from [UniqueAddress(akka.tcp://ClusterSystem@hostname1:42282,1853147177)] DEBUG | 2016-06-22 20:00:01 | [ClusterSystem-akka.actor.default-dispatcher-17] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-6, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A42282-4/endpointWriter, akkaTimestamp=20:00:01.513UTC] - received local message RemoteMessage: [ActorSelectionMessage(Status(Map(akka.tcp://ClusterSystem@hostname1:2551 -> 1466624412169)),Vector(system, distributedPubSubMediator),false)] to [Actor[akka://ClusterSystem/]]<+[akka://ClusterSystem/] from [Actor[akka.tcp://ClusterSystem@hostname1:42282/system/distributedPubSubMediator#-959991352]()] DEBUG | 2016-06-22 20:00:01 | [ClusterSystem-akka.actor.default-dispatcher-15] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-6, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:01.636UTC] - sending message RemoteMessage: [ActorSelectionMessage(GossipStatus(UniqueAddress(akka.tcp://ClusterSystem@hostname2:2552,235402593),VectorClock(0674e80a1aa6eaa2990d3dab1b9beedc -> 1, b7a6f622639ec0187ab601337796f207 -> 7, bec77a033021fc1ab6448cb0c1af53fd -> 1, e191b0013b1bc705772a35b0e9cdbca3 -> 1)),Vector(system, cluster, core, daemon),false)] to [Actor[akka.tcp://ClusterSystem@hostname1:2551/]]<+[akka.tcp://ClusterSystem@hostname1:2551/] from [Actor[akka://ClusterSystem/system/cluster/core/daemon#-150228074]] DEBUG | 2016-06-22 20:00:01 | [ClusterSystem-akka.actor.default-dispatcher-15] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-5, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname2%3A43209-3/endpointWriter, akkaTimestamp=20:00:01.696UTC] - sending message RemoteMessage: [ActorSelectionMessage(Status(Map(akka.tcp://ClusterSystem@hostname1:2551 -> 1466624412169)),Vector(system, distributedPubSubMediator),false)] to [Actor[akka.tcp://ClusterSystem@hostname2:43209/]]<+[akka.tcp://ClusterSystem@hostname2:43209/] from [Actor[akka://ClusterSystem/system/distributedPubSubMediator#638476632]] DEBUG | 2016-06-22 20:00:01 | [ClusterSystem-akka.actor.default-dispatcher-27] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-9, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:43209/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname2%3A2552-1/endpointWriter, akkaTimestamp=20:00:01.696UTC] - received local message RemoteMessage: [ActorSelectionMessage(Status(Map(akka.tcp://ClusterSystem@hostname1:2551 -> 1466624412169)),Vector(system, distributedPubSubMediator),false)] to [Actor[akka://ClusterSystem/]]<+[akka://ClusterSystem/] from [Actor[akka.tcp://ClusterSystem@hostname2:2552/system/distributedPubSubMediator#638476632]()] DEBUG | 2016-06-22 20:00:01 | [ClusterSystem-akka.actor.default-dispatcher-15] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-6, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2Fcmnet-WorkerSystem%40127.0.0.1%3A46637-7/endpointWriter, akkaTimestamp=20:00:01.795UTC] - received local message RemoteMessage: [Heartbeat] to [Actor[akka://ClusterSystem/system/receptionist#-1830908093]]<+[akka://ClusterSystem/system/receptionist] from [Actor[akka.tcp://[email protected]:46637/user/clusterClient#-808306789]()] DEBUG | 2016-06-22 20:00:01 | [ClusterSystem-akka.actor.default-dispatcher-15] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-5, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2Fcmnet-WorkerSystem%40127.0.0.1%3A46637-7/endpointWriter, akkaTimestamp=20:00:01.795UTC] - sending message RemoteMessage: [HeartbeatRsp] to [Actor[akka.tcp://[email protected]:46637/user/clusterClient#-808306789]]<+[akka.tcp://[email protected]:46637/user/clusterClient] from [Actor[akka://ClusterSystem/system/receptionist#-1830908093]] DEBUG | 2016-06-22 20:00:01 | [ClusterSystem-akka.actor.default-dispatcher-15] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-6, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2Fcmnet-WorkerSystem%40127.0.0.1%3A46637-7/endpointWriter, akkaTimestamp=20:00:01.795UTC] - received local message RemoteMessage: [SendToAll(/user/master/singleton,RegisterWorker{workerId='CMNET-7c4c2109-c5e2-42bc-bdbb-ab8a262d7cd0'workertype='point:cmnet'},false)] to [Actor[akka://ClusterSystem/system/receptionist#-1830908093]]<+[akka://ClusterSystem/system/receptionist] from [Actor[akka.tcp://[email protected]:46637/user/cmnetMember#437938352]()] DEBUG | 2016-06-22 20:00:01 | [ClusterSystem-akka.actor.default-dispatcher-15] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-6, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:01.795UTC] - sending message RemoteMessage: [RegisterWorker{workerId='CMNET-7c4c2109-c5e2-42bc-bdbb-ab8a262d7cd0'workertype='point:cmnet'}] to [Actor[akka.tcp://ClusterSystem@hostname1:2551/user/master/singleton#-1599543352]]<+[akka.tcp://ClusterSystem@hostname1:2551/user/master/singleton] from [Actor[akka://ClusterSystem/system/receptionist/akka.tcp%3A%2F%2Fcmnet-WorkerSystem%40127.0.0.1%3A46637%2Fuser%2FcmnetMember%23437938352#1194534746]] DEBUG | 2016-06-22 20:00:01 | [ClusterSystem-akka.actor.default-dispatcher-15] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-6, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:01.796UTC] - received local message RemoteMessage: [MasterActorProtocol$WorkIsReady@5e8ca0c7] to [Actor[akka://ClusterSystem/system/receptionist/akka.tcp%3A%2F%2Fcmnet-WorkerSystem%40127.0.0.1%3A46637%2Fuser%2FcmnetMember%23437938352#1194534746]]<+[akka://ClusterSystem/system/receptionist/akka.tcp%3A%2F%2Fcmnet-WorkerSystem%40127.0.0.1%3A46637%2Fuser%2FcmnetMember%23437938352] from [Actor[akka.tcp://ClusterSystem@hostname1:2551/user/master/singleton#-1599543352]()] DEBUG | 2016-06-22 20:00:01 | [ClusterSystem-akka.actor.default-dispatcher-15] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-6, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2Fcmnet-WorkerSystem%40127.0.0.1%3A46637-7/endpointWriter, akkaTimestamp=20:00:01.796UTC] - sending message RemoteMessage: [MasterActorProtocol$WorkIsReady@5e8ca0c7] to [Actor[akka.tcp://[email protected]:46637/user/cmnetMember#437938352]]<+[akka.tcp://[email protected]:46637/user/cmnetMember] from [Actor[akka://ClusterSystem/deadLetters]] DEBUG | 2016-06-22 20:00:01 | [ClusterSystem-akka.actor.default-dispatcher-15] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-6, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2Fcmnet-WorkerSystem%40127.0.0.1%3A46637-7/endpointWriter, akkaTimestamp=20:00:01.800UTC] - received local message RemoteMessage: [SendToAll(/user/master/singleton,WorkerRequestsWork{workerId='CMNET-7c4c2109-c5e2-42bc-bdbb-ab8a262d7cd0'},false)] to [Actor[akka://ClusterSystem/system/receptionist#-1830908093]]<+[akka://ClusterSystem/system/receptionist] from [Actor[akka.tcp://[email protected]:46637/user/cmnetMember#437938352]()] DEBUG | 2016-06-22 20:00:01 | [ClusterSystem-akka.actor.default-dispatcher-15] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-6, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:01.800UTC] - sending message RemoteMessage: [WorkerRequestsWork{workerId='CMNET-7c4c2109-c5e2-42bc-bdbb-ab8a262d7cd0'}] to [Actor[akka.tcp://ClusterSystem@hostname1:2551/user/master/singleton#-1599543352]]<+[akka.tcp://ClusterSystem@hostname1:2551/user/master/singleton] from [Actor[akka://ClusterSystem/system/receptionist/akka.tcp%3A%2F%2Fcmnet-WorkerSystem%40127.0.0.1%3A46637%2Fuser%2FcmnetMember%23437938352#1194534746]] DEBUG | 2016-06-22 20:00:01 | [ClusterSystem-akka.actor.default-dispatcher-15] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-6, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A42282-4/endpointWriter, akkaTimestamp=20:00:01.864UTC] - received local message RemoteMessage: [ActorSelectionMessage(Heartbeat(akka.tcp://ClusterSystem@hostname1:42282),Vector(system, cluster, heartbeatReceiver),false)] to [Actor[akka://ClusterSystem/]]<+[akka://ClusterSystem/] from [Actor[akka.tcp://ClusterSystem@hostname1:42282/system/cluster/core/daemon/heartbeatSender#1072477085]()] DEBUG | 2016-06-22 20:00:01 | [ClusterSystem-akka.actor.default-dispatcher-15] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-6, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A42282-4/endpointWriter, akkaTimestamp=20:00:01.864UTC] - sending message RemoteMessage: [HeartbeatRsp(UniqueAddress(akka.tcp://ClusterSystem@hostname2:2552,235402593))] to [Actor[akka.tcp://ClusterSystem@hostname1:42282/system/cluster/core/daemon/heartbeatSender#1072477085]]<+[akka.tcp://ClusterSystem@hostname1:42282/system/cluster/core/daemon/heartbeatSender] from [Actor[akka://ClusterSystem/system/cluster/heartbeatReceiver#-670753192]] DEBUG | 2016-06-22 20:00:02 | [ClusterSystem-akka.actor.default-dispatcher-15] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-6, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:02.018UTC] - received local message RemoteMessage: [ActorSelectionMessage(GossipStatus(UniqueAddress(akka.tcp://ClusterSystem@hostname1:2551,598411574),VectorClock(0674e80a1aa6eaa2990d3dab1b9beedc -> 1, b7a6f622639ec0187ab601337796f207 -> 7, bec77a033021fc1ab6448cb0c1af53fd -> 1, e191b0013b1bc705772a35b0e9cdbca3 -> 1)),Vector(system, cluster, core, daemon),false)] to [Actor[akka://ClusterSystem/]]<+[akka://ClusterSystem/] from [Actor[akka.tcp://ClusterSystem@hostname1:2551/system/cluster/core/daemon#1532363050]()] DEBUG | 2016-06-22 20:00:02 | [ClusterSystem-akka.actor.default-dispatcher-15] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-6, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:02.087UTC] - received local message RemoteMessage: [ActorSelectionMessage(Heartbeat(akka.tcp://ClusterSystem@hostname1:2551),Vector(system, cluster, heartbeatReceiver),false)] to [Actor[akka://ClusterSystem/]]<+[akka://ClusterSystem/] from [Actor[akka.tcp://ClusterSystem@hostname1:2551/system/cluster/core/daemon/heartbeatSender#-1740097823]()] DEBUG | 2016-06-22 20:00:02 | [ClusterSystem-akka.actor.default-dispatcher-27] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-8, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:43209/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:02.087UTC] - received local message RemoteMessage: [ActorSelectionMessage(Heartbeat(akka.tcp://ClusterSystem@hostname1:2551),Vector(system, cluster, heartbeatReceiver),false)] to [Actor[akka://ClusterSystem/]]<+[akka://ClusterSystem/] from [Actor[akka.tcp://ClusterSystem@hostname1:2551/system/cluster/core/daemon/heartbeatSender#-1740097823]()] DEBUG | 2016-06-22 20:00:02 | [ClusterSystem-akka.actor.default-dispatcher-15] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-6, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:02.087UTC] - sending message RemoteMessage: [HeartbeatRsp(UniqueAddress(akka.tcp://ClusterSystem@hostname2:2552,235402593))] to [Actor[akka.tcp://ClusterSystem@hostname1:2551/system/cluster/core/daemon/heartbeatSender#-1740097823]]<+[akka.tcp://ClusterSystem@hostname1:2551/system/cluster/core/daemon/heartbeatSender] from [Actor[akka://ClusterSystem/system/cluster/heartbeatReceiver#-670753192]] DEBUG | 2016-06-22 20:00:02 | [ClusterSystem-akka.actor.default-dispatcher-27] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-8, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:43209/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:02.087UTC] - sending message RemoteMessage: [HeartbeatRsp(UniqueAddress(akka.tcp://ClusterSystem@hostname2:43209,-951268098))] to [Actor[akka.tcp://ClusterSystem@hostname1:2551/system/cluster/core/daemon/heartbeatSender#-1740097823]]<+[akka.tcp://ClusterSystem@hostname1:2551/system/cluster/core/daemon/heartbeatSender] from [Actor[akka://ClusterSystem/system/cluster/heartbeatReceiver#-787172203]] DEBUG | 2016-06-22 20:00:02 | [ClusterSystem-akka.actor.default-dispatcher-28] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-9, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:43209/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:02.384UTC] - sending message RemoteMessage: [ActorSelectionMessage(akka.cluster.GossipEnvelope@32c762f2,Vector(system, cluster, core, daemon),false)] to [Actor[akka.tcp://ClusterSystem@hostname1:2551/]]<+[akka.tcp://ClusterSystem@hostname1:2551/] from [Actor[akka://ClusterSystem/system/cluster/core/daemon#-1375582141]] DEBUG | 2016-06-22 20:00:02 | [ClusterSystem-akka.actor.default-dispatcher-28] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-9, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:43209/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:02.384UTC] - sending message RemoteMessage: [ActorSelectionMessage(MetricsGossipEnvelope(akka.tcp://ClusterSystem@hostname2:43209,MetricsGossip(Set(NodeMetrics(akka.tcp://ClusterSystem@hostname2:2552,1466625597896,Set(Metric(heap-memory-used,146473928,Some(EWMA(1.46473928E8,0.15910417746033534))), Metric(heap-memory-max,7460618240,None), Metric(processors,8,None), Metric(system-load-average,0.0068359375,None), Metric(heap-memory-committed,1372585984,Some(EWMA(1.372585984E9,0.15910417746033534))))), NodeMetrics(akka.tcp://ClusterSystem@hostname1:2551,1466625598253,Set(Metric(heap-memory-used,671845048,Some(EWMA(6.71845048E8,0.15910417746033534))), Metric(heap-memory-max,7460618240,None), Metric(processors,8,None), Metric(system-load-average,0.00732421875,None), Metric(heap-memory-committed,1200619520,Some(EWMA(1.20061952E9,0.15910417746033534))))), NodeMetrics(akka.tcp://ClusterSystem@hostname2:43209,1466625602384,Set(Metric(heap-memory-used,154784384,Some(EWMA(1.54784384E8,0.15910417746033534))), Metric(heap-memory-max,7460618240,None), Metric(processors,8,None), Metric(system-load-average,0.00634765625,None), Metric(heap-memory-committed,1372585984,Some(EWMA(1.372585984E9,0.15910417746033534))))))),false),Vector(system, cluster, metrics),false)] to [Actor[akka.tcp://ClusterSystem@hostname1:2551/]]<+[akka.tcp://ClusterSystem@hostname1:2551/] from [Actor[akka://ClusterSystem/system/cluster/metrics#1391158606]] DEBUG | 2016-06-22 20:00:02 | [ClusterSystem-akka.actor.default-dispatcher-17] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-9, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:43209/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:02.385UTC] - received local message RemoteMessage: [ActorSelectionMessage(MetricsGossipEnvelope(akka.tcp://ClusterSystem@hostname1:2551,MetricsGossip(Set(NodeMetrics(akka.tcp://ClusterSystem@hostname2:2552,1466625600897,Set(Metric(heap-memory-used,154272488,Some(EWMA(1.54272488E8,0.15910417746033534))), Metric(heap-memory-max,7460618240,None), Metric(processors,8,None), Metric(system-load-average,0.0068359375,None), Metric(heap-memory-committed,1372585984,Some(EWMA(1.372585984E9,0.15910417746033534))))), NodeMetrics(akka.tcp://ClusterSystem@hostname1:2551,1466625601253,Set(Metric(heap-memory-used,680555144,Some(EWMA(6.80555144E8,0.15910417746033534))), Metric(heap-memory-max,7460618240,None), Metric(processors,8,None), Metric(system-load-average,0.0068359375,None), Metric(heap-memory-committed,1200619520,Some(EWMA(1.20061952E9,0.15910417746033534))))))),true),Vector(system, cluster, metrics),false)] to [Actor[akka://ClusterSystem/]]<+[akka://ClusterSystem/] from [Actor[akka.tcp://ClusterSystem@hostname1:2551/system/cluster/metrics#616453818]()] DEBUG | 2016-06-22 20:00:02 | [ClusterSystem-akka.actor.default-dispatcher-15] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-5, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname2%3A43209-3/endpointWriter, akkaTimestamp=20:00:02.396UTC] - sending message RemoteMessage: [ActorSelectionMessage(Heartbeat(akka.tcp://ClusterSystem@hostname2:2552),Vector(system, cluster, heartbeatReceiver),false)] to [Actor[akka.tcp://ClusterSystem@hostname2:43209/]]<+[akka.tcp://ClusterSystem@hostname2:43209/] from [Actor[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender#-132561496]] DEBUG | 2016-06-22 20:00:02 | [ClusterSystem-akka.actor.default-dispatcher-15] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-6, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:02.396UTC] - sending message RemoteMessage: [ActorSelectionMessage(Heartbeat(akka.tcp://ClusterSystem@hostname2:2552),Vector(system, cluster, heartbeatReceiver),false)] to [Actor[akka.tcp://ClusterSystem@hostname1:2551/]]<+[akka.tcp://ClusterSystem@hostname1:2551/] from [Actor[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender#-132561496]] DEBUG | 2016-06-22 20:00:02 | [ClusterSystem-akka.actor.default-dispatcher-15] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-5, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A42282-4/endpointWriter, akkaTimestamp=20:00:02.396UTC] - sending message RemoteMessage: [ActorSelectionMessage(Heartbeat(akka.tcp://ClusterSystem@hostname2:2552),Vector(system, cluster, heartbeatReceiver),false)] to [Actor[akka.tcp://ClusterSystem@hostname1:42282/]]<+[akka.tcp://ClusterSystem@hostname1:42282/] from [Actor[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender#-132561496]] DEBUG | 2016-06-22 20:00:02 | [ClusterSystem-akka.actor.default-dispatcher-17] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-8, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:43209/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname2%3A2552-1/endpointWriter, akkaTimestamp=20:00:02.396UTC] - received local message RemoteMessage: [ActorSelectionMessage(Heartbeat(akka.tcp://ClusterSystem@hostname2:2552),Vector(system, cluster, heartbeatReceiver),false)] to [Actor[akka://ClusterSystem/]]<+[akka://ClusterSystem/] from [Actor[akka.tcp://ClusterSystem@hostname2:2552/system/cluster/core/daemon/heartbeatSender#-132561496]()] DEBUG | 2016-06-22 20:00:02 | [ClusterSystem-akka.actor.default-dispatcher-17] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-9, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:43209/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname2%3A2552-1/endpointWriter, akkaTimestamp=20:00:02.396UTC] - sending message RemoteMessage: [HeartbeatRsp(UniqueAddress(akka.tcp://ClusterSystem@hostname2:43209,-951268098))] to [Actor[akka.tcp://ClusterSystem@hostname2:2552/system/cluster/core/daemon/heartbeatSender#-132561496]]<+[akka.tcp://ClusterSystem@hostname2:2552/system/cluster/core/daemon/heartbeatSender] from [Actor[akka://ClusterSystem/system/cluster/heartbeatReceiver#-787172203]] DEBUG | 2016-06-22 20:00:02 | [ClusterSystem-akka.actor.default-dispatcher-15] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-5, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname2%3A43209-3/endpointWriter, akkaTimestamp=20:00:02.396UTC] - received local message RemoteMessage: [HeartbeatRsp(UniqueAddress(akka.tcp://ClusterSystem@hostname2:43209,-951268098))] to [Actor[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender#-132561496]]<+[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender] from [Actor[akka.tcp://ClusterSystem@hostname2:43209/system/cluster/heartbeatReceiver#-787172203]()] DEBUG | 2016-06-22 20:00:02 | [ClusterSystem-akka.actor.default-dispatcher-15] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-5, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:02.397UTC] - received local message RemoteMessage: [HeartbeatRsp(UniqueAddress(akka.tcp://ClusterSystem@hostname1:2551,598411574))] to [Actor[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender#-132561496]]<+[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender] from [Actor[akka.tcp://ClusterSystem@hostname1:2551/system/cluster/heartbeatReceiver#-2035426665]()] DEBUG | 2016-06-22 20:00:02 | [ClusterSystem-akka.actor.default-dispatcher-15] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-6, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A42282-4/endpointWriter, akkaTimestamp=20:00:02.397UTC] - received local message RemoteMessage: [HeartbeatRsp(UniqueAddress(akka.tcp://ClusterSystem@hostname1:42282,1853147177))] to [Actor[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender#-132561496]]<+[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender] from [Actor[akka.tcp://ClusterSystem@hostname1:42282/system/cluster/heartbeatReceiver#1799437957]()] DEBUG | 2016-06-22 20:00:02 | [ClusterSystem-akka.actor.default-dispatcher-17] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-9, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:43209/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname2%3A2552-1/endpointWriter, akkaTimestamp=20:00:02.443UTC] - sending message RemoteMessage: [ActorSelectionMessage(Heartbeat(akka.tcp://ClusterSystem@hostname2:43209),Vector(system, cluster, heartbeatReceiver),false)] to [Actor[akka.tcp://ClusterSystem@hostname2:2552/]]<+[akka.tcp://ClusterSystem@hostname2:2552/] from [Actor[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender#-271688942]] DEBUG | 2016-06-22 20:00:02 | [ClusterSystem-akka.actor.default-dispatcher-17] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-8, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:43209/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:02.443UTC] - sending message RemoteMessage: [ActorSelectionMessage(Heartbeat(akka.tcp://ClusterSystem@hostname2:43209),Vector(system, cluster, heartbeatReceiver),false)] to [Actor[akka.tcp://ClusterSystem@hostname1:2551/]]<+[akka.tcp://ClusterSystem@hostname1:2551/] from [Actor[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender#-271688942]] DEBUG | 2016-06-22 20:00:02 | [ClusterSystem-akka.actor.default-dispatcher-15] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-6, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname2%3A43209-3/endpointWriter, akkaTimestamp=20:00:02.443UTC] - received local message RemoteMessage: [ActorSelectionMessage(Heartbeat(akka.tcp://ClusterSystem@hostname2:43209),Vector(system, cluster, heartbeatReceiver),false)] to [Actor[akka://ClusterSystem/]]<+[akka://ClusterSystem/] from [Actor[akka.tcp://ClusterSystem@hostname2:43209/system/cluster/core/daemon/heartbeatSender#-271688942]()] DEBUG | 2016-06-22 20:00:02 | [ClusterSystem-akka.actor.default-dispatcher-15] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-6, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname2%3A43209-3/endpointWriter, akkaTimestamp=20:00:02.443UTC] - sending message RemoteMessage: [HeartbeatRsp(UniqueAddress(akka.tcp://ClusterSystem@hostname2:2552,235402593))] to [Actor[akka.tcp://ClusterSystem@hostname2:43209/system/cluster/core/daemon/heartbeatSender#-271688942]]<+[akka.tcp://ClusterSystem@hostname2:43209/system/cluster/core/daemon/heartbeatSender] from [Actor[akka://ClusterSystem/system/cluster/heartbeatReceiver#-670753192]] DEBUG | 2016-06-22 20:00:02 | [ClusterSystem-akka.actor.default-dispatcher-17] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-9, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:43209/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname2%3A2552-1/endpointWriter, akkaTimestamp=20:00:02.444UTC] - received local message RemoteMessage: [HeartbeatRsp(UniqueAddress(akka.tcp://ClusterSystem@hostname2:2552,235402593))] to [Actor[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender#-271688942]]<+[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender] from [Actor[akka.tcp://ClusterSystem@hostname2:2552/system/cluster/heartbeatReceiver#-670753192]()] DEBUG | 2016-06-22 20:00:02 | [ClusterSystem-akka.actor.default-dispatcher-17] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-9, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:43209/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:02.444UTC] - received local message RemoteMessage: [HeartbeatRsp(UniqueAddress(akka.tcp://ClusterSystem@hostname1:2551,598411574))] to [Actor[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender#-271688942]]<+[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender] from [Actor[akka.tcp://ClusterSystem@hostname1:2551/system/cluster/heartbeatReceiver#-2035426665]()] DEBUG | 2016-06-22 20:00:02 | [ClusterSystem-akka.actor.default-dispatcher-15] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-6, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A42282-4/endpointWriter, akkaTimestamp=20:00:02.504UTC] - received local message RemoteMessage: [ActorSelectionMessage(MetricsGossipEnvelope(akka.tcp://ClusterSystem@hostname1:42282,MetricsGossip(Set(NodeMetrics(akka.tcp://ClusterSystem@hostname1:42282,1466625599499,Set(Metric(heap-memory-used,677182360,Some(EWMA(6.7718236E8,0.15910417746033534))), Metric(heap-memory-max,7460618240,None), Metric(processors,8,None), Metric(system-load-average,0.0068359375,None), Metric(heap-memory-committed,1200619520,Some(EWMA(1.20061952E9,0.15910417746033534))))), NodeMetrics(akka.tcp://ClusterSystem@hostname2:2552,1466625597896,Set(Metric(heap-memory-used,146473928,Some(EWMA(1.46473928E8,0.15910417746033534))), Metric(heap-memory-max,7460618240,None), Metric(processors,8,None), Metric(system-load-average,0.0068359375,None), Metric(heap-memory-committed,1372585984,Some(EWMA(1.372585984E9,0.15910417746033534))))), NodeMetrics(akka.tcp://ClusterSystem@hostname1:2551,1466625598253,Set(Metric(heap-memory-used,671845048,Some(EWMA(6.71845048E8,0.15910417746033534))), Metric(heap-memory-max,7460618240,None), Metric(processors,8,None), Metric(system-load-average,0.00732421875,None), Metric(heap-memory-committed,1200619520,Some(EWMA(1.20061952E9,0.15910417746033534))))))),false),Vector(system, cluster, metrics),false)] to [Actor[akka://ClusterSystem/]]<+[akka://ClusterSystem/] from [Actor[akka.tcp://ClusterSystem@hostname1:42282/system/cluster/metrics#-1978863380]()] DEBUG | 2016-06-22 20:00:02 | [ClusterSystem-akka.actor.default-dispatcher-15] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-6, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A42282-4/endpointWriter, akkaTimestamp=20:00:02.504UTC] - sending message RemoteMessage: [ActorSelectionMessage(MetricsGossipEnvelope(akka.tcp://ClusterSystem@hostname2:2552,MetricsGossip(Set(NodeMetrics(akka.tcp://ClusterSystem@hostname2:2552,1466625600897,Set(Metric(heap-memory-used,154272488,Some(EWMA(1.54272488E8,0.15910417746033534))), Metric(heap-memory-max,7460618240,None), Metric(processors,8,None), Metric(system-load-average,0.0068359375,None), Metric(heap-memory-committed,1372585984,Some(EWMA(1.372585984E9,0.15910417746033534))))), NodeMetrics(akka.tcp://ClusterSystem@hostname1:2551,1466625598253,Set(Metric(heap-memory-used,671845048,Some(EWMA(6.71845048E8,0.15910417746033534))), Metric(heap-memory-max,7460618240,None), Metric(processors,8,None), Metric(system-load-average,0.00732421875,None), Metric(heap-memory-committed,1200619520,Some(EWMA(1.20061952E9,0.15910417746033534))))))),true),Vector(system, cluster, metrics),false)] to [Actor[akka.tcp://ClusterSystem@hostname1:42282/]]<+[akka.tcp://ClusterSystem@hostname1:42282/] from [Actor[akka://ClusterSystem/system/cluster/metrics#1040761959]] DEBUG | 2016-06-22 20:00:02 | [ClusterSystem-akka.actor.default-dispatcher-15] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-5, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A42282-4/endpointWriter, akkaTimestamp=20:00:02.514UTC] - received local message RemoteMessage: [ActorSelectionMessage(akka.cluster.GossipEnvelope@29a2d48a,Vector(system, cluster, core, daemon),false)] to [Actor[akka://ClusterSystem/]]<+[akka://ClusterSystem/] from [Actor[akka.tcp://ClusterSystem@hostname1:42282/system/cluster/core/daemon#-1814189187]()] DEBUG | 2016-06-22 20:00:02 | [ClusterSystem-akka.actor.default-dispatcher-15] ? - [sourceThread=ClusterSystem-akka.actor.default-dispatcher-3, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/cluster/core/daemon, akkaTimestamp=20:00:02.514UTC] - Cluster Node [akka.tcp://ClusterSystem@hostname2:2552] - Receiving gossip from [UniqueAddress(akka.tcp://ClusterSystem@hostname1:42282,1853147177)] DEBUG | 2016-06-22 20:00:02 | [ClusterSystem-akka.actor.default-dispatcher-3] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-5, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:02.637UTC] - sending message RemoteMessage: [ActorSelectionMessage(GossipStatus(UniqueAddress(akka.tcp://ClusterSystem@hostname2:2552,235402593),VectorClock(0674e80a1aa6eaa2990d3dab1b9beedc -> 1, b7a6f622639ec0187ab601337796f207 -> 7, bec77a033021fc1ab6448cb0c1af53fd -> 1, e191b0013b1bc705772a35b0e9cdbca3 -> 1)),Vector(system, cluster, core, daemon),false)] to [Actor[akka.tcp://ClusterSystem@hostname1:2551/]]<+[akka.tcp://ClusterSystem@hostname1:2551/] from [Actor[akka://ClusterSystem/system/cluster/core/daemon#-150228074]] DEBUG | 2016-06-22 20:00:02 | [ClusterSystem-akka.actor.default-dispatcher-3] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-6, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname2%3A43209-3/endpointWriter, akkaTimestamp=20:00:02.696UTC] - sending message RemoteMessage: [ActorSelectionMessage(Status(Map(akka.tcp://ClusterSystem@hostname1:2551 -> 1466624412169)),Vector(system, distributedPubSubMediator),false)] to [Actor[akka.tcp://ClusterSystem@hostname2:43209/]]<+[akka.tcp://ClusterSystem@hostname2:43209/] from [Actor[akka://ClusterSystem/system/distributedPubSubMediator#638476632]] DEBUG | 2016-06-22 20:00:02 | [ClusterSystem-akka.actor.default-dispatcher-17] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-8, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:43209/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname2%3A2552-1/endpointWriter, akkaTimestamp=20:00:02.696UTC] - received local message RemoteMessage: [ActorSelectionMessage(Status(Map(akka.tcp://ClusterSystem@hostname1:2551 -> 1466624412169)),Vector(system, distributedPubSubMediator),false)] to [Actor[akka://ClusterSystem/]]<+[akka://ClusterSystem/] from [Actor[akka.tcp://ClusterSystem@hostname2:2552/system/distributedPubSubMediator#638476632]()] DEBUG | 2016-06-22 20:00:02 | [ClusterSystem-akka.actor.default-dispatcher-3] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-6, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2Fsnmp-WorkerSystem%40127.0.0.1%3A44183-13/endpointWriter, akkaTimestamp=20:00:02.812UTC] - received local message RemoteMessage: [SendToAll(/user/master/singleton,RegisterWorker{workerId='SNMP-f783d084-87e1-491a-8246-3d9d64d8a01d'workertype='point:snmp'},false)] to [Actor[akka://ClusterSystem/system/receptionist#-1830908093]]<+[akka://ClusterSystem/system/receptionist] from [Actor[akka.tcp://[email protected]:44183/user/snmpMember#-343146825]()] DEBUG | 2016-06-22 20:00:02 | [ClusterSystem-akka.actor.default-dispatcher-4] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-6, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:02.812UTC] - sending message RemoteMessage: [RegisterWorker{workerId='SNMP-f783d084-87e1-491a-8246-3d9d64d8a01d'workertype='point:snmp'}] to [Actor[akka.tcp://ClusterSystem@hostname1:2551/user/master/singleton#-1599543352]]<+[akka.tcp://ClusterSystem@hostname1:2551/user/master/singleton] from [Actor[akka://ClusterSystem/system/receptionist/akka.tcp%3A%2F%2Fsnmp-WorkerSystem%40127.0.0.1%3A44183%2Fuser%2FsnmpMember%23-343146825#1974211035]] DEBUG | 2016-06-22 20:00:02 | [ClusterSystem-akka.actor.default-dispatcher-16] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-5, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2Fsnmp-WorkerSystem%40127.0.0.1%3A44183-13/endpointWriter, akkaTimestamp=20:00:02.820UTC] - received local message RemoteMessage: [Heartbeat] to [Actor[akka://ClusterSystem/system/receptionist#-1830908093]]<+[akka://ClusterSystem/system/receptionist] from [Actor[akka.tcp://[email protected]:44183/user/clusterClient#1180082629]()] DEBUG | 2016-06-22 20:00:02 | [ClusterSystem-akka.actor.default-dispatcher-16] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-5, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2Fsnmp-WorkerSystem%40127.0.0.1%3A44183-13/endpointWriter, akkaTimestamp=20:00:02.820UTC] - sending message RemoteMessage: [HeartbeatRsp] to [Actor[akka.tcp://[email protected]:44183/user/clusterClient#1180082629]]<+[akka.tcp://[email protected]:44183/user/clusterClient] from [Actor[akka://ClusterSystem/system/receptionist#-1830908093]] DEBUG | 2016-06-22 20:00:02 | [ClusterSystem-akka.actor.default-dispatcher-16] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-5, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A42282-4/endpointWriter, akkaTimestamp=20:00:02.864UTC] - received local message RemoteMessage: [ActorSelectionMessage(Heartbeat(akka.tcp://ClusterSystem@hostname1:42282),Vector(system, cluster, heartbeatReceiver),false)] to [Actor[akka://ClusterSystem/]]<+[akka://ClusterSystem/] from [Actor[akka.tcp://ClusterSystem@hostname1:42282/system/cluster/core/daemon/heartbeatSender#1072477085]()] DEBUG | 2016-06-22 20:00:02 | [ClusterSystem-akka.actor.default-dispatcher-16] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-5, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A42282-4/endpointWriter, akkaTimestamp=20:00:02.864UTC] - sending message RemoteMessage: [HeartbeatRsp(UniqueAddress(akka.tcp://ClusterSystem@hostname2:2552,235402593))] to [Actor[akka.tcp://ClusterSystem@hostname1:42282/system/cluster/core/daemon/heartbeatSender#1072477085]]<+[akka.tcp://ClusterSystem@hostname1:42282/system/cluster/core/daemon/heartbeatSender] from [Actor[akka://ClusterSystem/system/cluster/heartbeatReceiver#-670753192]] DEBUG | 2016-06-22 20:00:03 | [ClusterSystem-akka.actor.default-dispatcher-16] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-5, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:03.017UTC] - received local message RemoteMessage: [ActorSelectionMessage(GossipStatus(UniqueAddress(akka.tcp://ClusterSystem@hostname1:2551,598411574),VectorClock(0674e80a1aa6eaa2990d3dab1b9beedc -> 1, b7a6f622639ec0187ab601337796f207 -> 7, bec77a033021fc1ab6448cb0c1af53fd -> 1, e191b0013b1bc705772a35b0e9cdbca3 -> 1)),Vector(system, cluster, core, daemon),false)] to [Actor[akka://ClusterSystem/]]<+[akka://ClusterSystem/] from [Actor[akka.tcp://ClusterSystem@hostname1:2551/system/cluster/core/daemon#1532363050]()] DEBUG | 2016-06-22 20:00:03 | [ClusterSystem-akka.actor.default-dispatcher-16] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-6, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:03.067UTC] - received local message RemoteMessage: [ActorSelectionMessage(Status(Map(akka.tcp://ClusterSystem@hostname1:2551 -> 1466624412169)),Vector(system, distributedPubSubMediator),false)] to [Actor[akka://ClusterSystem/]]<+[akka://ClusterSystem/] from [Actor[akka.tcp://ClusterSystem@hostname1:2551/system/distributedPubSubMediator#-379718827]()] DEBUG | 2016-06-22 20:00:03 | [ClusterSystem-akka.actor.default-dispatcher-16] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-6, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:03.087UTC] - received local message RemoteMessage: [ActorSelectionMessage(Heartbeat(akka.tcp://ClusterSystem@hostname1:2551),Vector(system, cluster, heartbeatReceiver),false)] to [Actor[akka://ClusterSystem/]]<+[akka://ClusterSystem/] from [Actor[akka.tcp://ClusterSystem@hostname1:2551/system/cluster/core/daemon/heartbeatSender#-1740097823]()] DEBUG | 2016-06-22 20:00:03 | [ClusterSystem-akka.actor.default-dispatcher-17] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-8, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:43209/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:03.087UTC] - received local message RemoteMessage: [ActorSelectionMessage(Heartbeat(akka.tcp://ClusterSystem@hostname1:2551),Vector(system, cluster, heartbeatReceiver),false)] to [Actor[akka://ClusterSystem/]]<+[akka://ClusterSystem/] from [Actor[akka.tcp://ClusterSystem@hostname1:2551/system/cluster/core/daemon/heartbeatSender#-1740097823]()] DEBUG | 2016-06-22 20:00:03 | [ClusterSystem-akka.actor.default-dispatcher-4] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-6, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:03.087UTC] - sending message RemoteMessage: [HeartbeatRsp(UniqueAddress(akka.tcp://ClusterSystem@hostname2:2552,235402593))] to [Actor[akka.tcp://ClusterSystem@hostname1:2551/system/cluster/core/daemon/heartbeatSender#-1740097823]]<+[akka.tcp://ClusterSystem@hostname1:2551/system/cluster/core/daemon/heartbeatSender] from [Actor[akka://ClusterSystem/system/cluster/heartbeatReceiver#-670753192]] DEBUG | 2016-06-22 20:00:03 | [ClusterSystem-akka.actor.default-dispatcher-29] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-8, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:43209/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:03.087UTC] - sending message RemoteMessage: [HeartbeatRsp(UniqueAddress(akka.tcp://ClusterSystem@hostname2:43209,-951268098))] to [Actor[akka.tcp://ClusterSystem@hostname1:2551/system/cluster/core/daemon/heartbeatSender#-1740097823]]<+[akka.tcp://ClusterSystem@hostname1:2551/system/cluster/core/daemon/heartbeatSender] from [Actor[akka://ClusterSystem/system/cluster/heartbeatReceiver#-787172203]] DEBUG | 2016-06-22 20:00:03 | [ClusterSystem-akka.actor.default-dispatcher-18] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-8, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:43209/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:03.383UTC] - sending message RemoteMessage: [ActorSelectionMessage(akka.cluster.GossipEnvelope@374dd45c,Vector(system, cluster, core, daemon),false)] to [Actor[akka.tcp://ClusterSystem@hostname1:2551/]]<+[akka.tcp://ClusterSystem@hostname1:2551/] from [Actor[akka://ClusterSystem/system/cluster/core/daemon#-1375582141]] DEBUG | 2016-06-22 20:00:03 | [ClusterSystem-akka.actor.default-dispatcher-18] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-9, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:43209/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname2%3A2552-1/endpointWriter, akkaTimestamp=20:00:03.383UTC] - sending message RemoteMessage: [ActorSelectionMessage(Status(Map(akka.tcp://ClusterSystem@hostname1:2551 -> 1466624412169)),Vector(system, distributedPubSubMediator),false)] to [Actor[akka.tcp://ClusterSystem@hostname2:2552/]]<+[akka.tcp://ClusterSystem@hostname2:2552/] from [Actor[akka://ClusterSystem/system/distributedPubSubMediator#78424083]] DEBUG | 2016-06-22 20:00:03 | [ClusterSystem-akka.actor.default-dispatcher-4] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-5, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname2%3A43209-3/endpointWriter, akkaTimestamp=20:00:03.384UTC] - received local message RemoteMessage: [ActorSelectionMessage(Status(Map(akka.tcp://ClusterSystem@hostname1:2551 -> 1466624412169)),Vector(system, distributedPubSubMediator),false)] to [Actor[akka://ClusterSystem/]]<+[akka://ClusterSystem/] from [Actor[akka.tcp://ClusterSystem@hostname2:43209/system/distributedPubSubMediator#78424083]()] DEBUG | 2016-06-22 20:00:03 | [ClusterSystem-akka.actor.default-dispatcher-3] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-6, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname2%3A43209-3/endpointWriter, akkaTimestamp=20:00:03.397UTC] - sending message RemoteMessage: [ActorSelectionMessage(Heartbeat(akka.tcp://ClusterSystem@hostname2:2552),Vector(system, cluster, heartbeatReceiver),false)] to [Actor[akka.tcp://ClusterSystem@hostname2:43209/]]<+[akka.tcp://ClusterSystem@hostname2:43209/] from [Actor[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender#-132561496]] DEBUG | 2016-06-22 20:00:03 | [ClusterSystem-akka.actor.default-dispatcher-3] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-5, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:03.397UTC] - sending message RemoteMessage: [ActorSelectionMessage(Heartbeat(akka.tcp://ClusterSystem@hostname2:2552),Vector(system, cluster, heartbeatReceiver),false)] to [Actor[akka.tcp://ClusterSystem@hostname1:2551/]]<+[akka.tcp://ClusterSystem@hostname1:2551/] from [Actor[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender#-132561496]] DEBUG | 2016-06-22 20:00:03 | [ClusterSystem-akka.actor.default-dispatcher-3] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-5, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A42282-4/endpointWriter, akkaTimestamp=20:00:03.397UTC] - sending message RemoteMessage: [ActorSelectionMessage(Heartbeat(akka.tcp://ClusterSystem@hostname2:2552),Vector(system, cluster, heartbeatReceiver),false)] to [Actor[akka.tcp://ClusterSystem@hostname1:42282/]]<+[akka.tcp://ClusterSystem@hostname1:42282/] from [Actor[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender#-132561496]] DEBUG | 2016-06-22 20:00:03 | [ClusterSystem-akka.actor.default-dispatcher-18] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-9, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:43209/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname2%3A2552-1/endpointWriter, akkaTimestamp=20:00:03.397UTC] - received local message RemoteMessage: [ActorSelectionMessage(Heartbeat(akka.tcp://ClusterSystem@hostname2:2552),Vector(system, cluster, heartbeatReceiver),false)] to [Actor[akka://ClusterSystem/]]<+[akka://ClusterSystem/] from [Actor[akka.tcp://ClusterSystem@hostname2:2552/system/cluster/core/daemon/heartbeatSender#-132561496]()] DEBUG | 2016-06-22 20:00:03 | [ClusterSystem-akka.actor.default-dispatcher-18] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-9, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:43209/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname2%3A2552-1/endpointWriter, akkaTimestamp=20:00:03.397UTC] - sending message RemoteMessage: [HeartbeatRsp(UniqueAddress(akka.tcp://ClusterSystem@hostname2:43209,-951268098))] to [Actor[akka.tcp://ClusterSystem@hostname2:2552/system/cluster/core/daemon/heartbeatSender#-132561496]]<+[akka.tcp://ClusterSystem@hostname2:2552/system/cluster/core/daemon/heartbeatSender] from [Actor[akka://ClusterSystem/system/cluster/heartbeatReceiver#-787172203]] DEBUG | 2016-06-22 20:00:03 | [ClusterSystem-akka.actor.default-dispatcher-3] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-5, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname2%3A43209-3/endpointWriter, akkaTimestamp=20:00:03.397UTC] - received local message RemoteMessage: [HeartbeatRsp(UniqueAddress(akka.tcp://ClusterSystem@hostname2:43209,-951268098))] to [Actor[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender#-132561496]]<+[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender] from [Actor[akka.tcp://ClusterSystem@hostname2:43209/system/cluster/heartbeatReceiver#-787172203]()] DEBUG | 2016-06-22 20:00:03 | [ClusterSystem-akka.actor.default-dispatcher-3] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-5, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:03.397UTC] - received local message RemoteMessage: [HeartbeatRsp(UniqueAddress(akka.tcp://ClusterSystem@hostname1:2551,598411574))] to [Actor[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender#-132561496]]<+[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender] from [Actor[akka.tcp://ClusterSystem@hostname1:2551/system/cluster/heartbeatReceiver#-2035426665]()] DEBUG | 2016-06-22 20:00:03 | [ClusterSystem-akka.actor.default-dispatcher-3] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-5, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A42282-4/endpointWriter, akkaTimestamp=20:00:03.398UTC] - received local message RemoteMessage: [HeartbeatRsp(UniqueAddress(akka.tcp://ClusterSystem@hostname1:42282,1853147177))] to [Actor[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender#-132561496]]<+[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender] from [Actor[akka.tcp://ClusterSystem@hostname1:42282/system/cluster/heartbeatReceiver#1799437957]()] DEBUG | 2016-06-22 20:00:03 | [ClusterSystem-akka.actor.default-dispatcher-18] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-9, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:43209/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:03.444UTC] - sending message RemoteMessage: [ActorSelectionMessage(Heartbeat(akka.tcp://ClusterSystem@hostname2:43209),Vector(system, cluster, heartbeatReceiver),false)] to [Actor[akka.tcp://ClusterSystem@hostname1:2551/]]<+[akka.tcp://ClusterSystem@hostname1:2551/] from [Actor[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender#-271688942]] DEBUG | 2016-06-22 20:00:03 | [ClusterSystem-akka.actor.default-dispatcher-18] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-9, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:43209/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname2%3A2552-1/endpointWriter, akkaTimestamp=20:00:03.444UTC] - sending message RemoteMessage: [ActorSelectionMessage(Heartbeat(akka.tcp://ClusterSystem@hostname2:43209),Vector(system, cluster, heartbeatReceiver),false)] to [Actor[akka.tcp://ClusterSystem@hostname2:2552/]]<+[akka.tcp://ClusterSystem@hostname2:2552/] from [Actor[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender#-271688942]] DEBUG | 2016-06-22 20:00:03 | [ClusterSystem-akka.actor.default-dispatcher-2] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-8, sourceActorSystem=ClusterSystem, akkaSource=akka://ClusterSystem/system/endpointManager, akkaTimestamp=20:00:03.444UTC] - now supervising Actor[akka://ClusterSystem/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A42282-207#1490820524] DEBUG | 2016-06-22 20:00:03 | [ClusterSystem-akka.actor.default-dispatcher-3] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-5, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname2%3A43209-3/endpointWriter, akkaTimestamp=20:00:03.444UTC] - received local message RemoteMessage: [ActorSelectionMessage(Heartbeat(akka.tcp://ClusterSystem@hostname2:43209),Vector(system, cluster, heartbeatReceiver),false)] to [Actor[akka://ClusterSystem/]]<+[akka://ClusterSystem/] from [Actor[akka.tcp://ClusterSystem@hostname2:43209/system/cluster/core/daemon/heartbeatSender#-271688942]()] DEBUG | 2016-06-22 20:00:03 | [ClusterSystem-akka.actor.default-dispatcher-4] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-5, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname2%3A43209-3/endpointWriter, akkaTimestamp=20:00:03.444UTC] - sending message RemoteMessage: [HeartbeatRsp(UniqueAddress(akka.tcp://ClusterSystem@hostname2:2552,235402593))] to [Actor[akka.tcp://ClusterSystem@hostname2:43209/system/cluster/core/daemon/heartbeatSender#-271688942]]<+[akka.tcp://ClusterSystem@hostname2:43209/system/cluster/core/daemon/heartbeatSender] from [Actor[akka://ClusterSystem/system/cluster/heartbeatReceiver#-670753192]] DEBUG | 2016-06-22 20:00:03 | [ClusterSystem-akka.actor.default-dispatcher-18] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-9, sourceActorSystem=ClusterSystem, akkaSource=akka://ClusterSystem/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A42282-207, akkaTimestamp=20:00:03.444UTC] - started (akka.remote.ReliableDeliverySupervisor@168d12e7) DEBUG | 2016-06-22 20:00:03 | [ClusterSystem-akka.actor.default-dispatcher-18] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-9, sourceActorSystem=ClusterSystem, akkaSource=akka://ClusterSystem/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A42282-207, akkaTimestamp=20:00:03.444UTC] - now watched by Actor[akka://ClusterSystem/system/endpointManager#891823629] DEBUG | 2016-06-22 20:00:03 | [ClusterSystem-akka.actor.default-dispatcher-18] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-9, sourceActorSystem=ClusterSystem, akkaSource=akka://ClusterSystem/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A42282-207, akkaTimestamp=20:00:03.444UTC] - now supervising Actor[akka://ClusterSystem/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A42282-207/endpointWriter#627674735] DEBUG | 2016-06-22 20:00:03 | [ClusterSystem-akka.actor.default-dispatcher-18] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-8, sourceActorSystem=ClusterSystem, akkaSource=akka://ClusterSystem/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A42282-207/endpointWriter, akkaTimestamp=20:00:03.444UTC] - started (akka.remote.EndpointWriter@61bedab2) DEBUG | 2016-06-22 20:00:03 | [ClusterSystem-akka.actor.default-dispatcher-18] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-8, sourceActorSystem=ClusterSystem, akkaSource=akka://ClusterSystem/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A42282-207/endpointWriter, akkaTimestamp=20:00:03.444UTC] - now watched by Actor[akka://ClusterSystem/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A42282-207#1490820524] DEBUG | 2016-06-22 20:00:03 | [ClusterSystem-akka.actor.default-dispatcher-18] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-8, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:43209/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname2%3A2552-1/endpointWriter, akkaTimestamp=20:00:03.444UTC] - received local message RemoteMessage: [HeartbeatRsp(UniqueAddress(akka.tcp://ClusterSystem@hostname2:2552,235402593))] to [Actor[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender#-271688942]]<+[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender] from [Actor[akka.tcp://ClusterSystem@hostname2:2552/system/cluster/heartbeatReceiver#-670753192]()] DEBUG | 2016-06-22 20:00:03 | [ClusterSystem-akka.actor.default-dispatcher-17] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-9, sourceActorSystem=ClusterSystem, akkaSource=akka://ClusterSystem/system/transports/akkaprotocolmanager.tcp1, akkaTimestamp=20:00:03.444UTC] - now supervising Actor[akka://ClusterSystem/system/transports/akkaprotocolmanager.tcp1/akkaProtocol-tcp%3A%2F%2FClusterSystem%40hostname1%3A42282-208#496938924] DEBUG | 2016-06-22 20:00:03 | [ClusterSystem-akka.actor.default-dispatcher-17] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-8, sourceActorSystem=ClusterSystem, akkaSource=akka://ClusterSystem/system/transports/akkaprotocolmanager.tcp1/akkaProtocol-tcp%3A%2F%2FClusterSystem%40hostname1%3A42282-208, akkaTimestamp=20:00:03.444UTC] - started (akka.remote.transport.ProtocolStateActor@687c23a9) DEBUG | 2016-06-22 20:00:03 | [ClusterSystem-akka.actor.default-dispatcher-17] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-9, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:43209/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:03.445UTC] - received local message RemoteMessage: [HeartbeatRsp(UniqueAddress(akka.tcp://ClusterSystem@hostname1:2551,598411574))] to [Actor[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender#-271688942]]<+[akka://ClusterSystem/system/cluster/core/daemon/heartbeatSender] from [Actor[akka.tcp://ClusterSystem@hostname1:2551/system/cluster/heartbeatReceiver#-2035426665]()] DEBUG | 2016-06-22 20:00:03 | [ClusterSystem-akka.actor.default-dispatcher-17] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-9, sourceActorSystem=ClusterSystem, akkaSource=akka://ClusterSystem/system/transports/akkaprotocolmanager.tcp1/akkaProtocol-tcp%3A%2F%2FClusterSystem%40hostname1%3A42282-208, akkaTimestamp=20:00:03.446UTC] - stopped DEBUG | 2016-06-22 20:00:03 | [ClusterSystem-akka.actor.default-dispatcher-17] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-8, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:43209/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A42282-207/endpointWriter, akkaTimestamp=20:00:03.446UTC] - AssociationError [akka.tcp://ClusterSystem@hostname2:43209] -> [akka.tcp://ClusterSystem@hostname1:42282]: Error [Association failed with [akka.tcp://ClusterSystem@hostname1:42282]] [ akka.remote.EndpointAssociationException: Association failed with [akka.tcp://ClusterSystem@hostname1:42282] Caused by: akka.remote.transport.netty.NettyTransportExceptionNoStack: Connection refused: hostname1/xxx.xxx.xxx.1:42282 ] WARN | 2016-06-22 20:00:03 | [ClusterSystem-akka.actor.default-dispatcher-17] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-8, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:43209/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A42282-207, akkaTimestamp=20:00:03.446UTC] - Association with remote system [akka.tcp://ClusterSystem@hostname1:42282] has failed, address is now gated for [5000] ms. Reason: [Association failed with [akka.tcp://ClusterSystem@hostname1:42282]] Caused by: [Connection refused: hostname1/xxx.xxx.xxx.1:42282] DEBUG | 2016-06-22 20:00:03 | [ClusterSystem-akka.actor.default-dispatcher-17] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-8, sourceActorSystem=ClusterSystem, akkaSource=akka://ClusterSystem/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A42282-207, akkaTimestamp=20:00:03.446UTC] - stopping DEBUG | 2016-06-22 20:00:03 | [ClusterSystem-akka.actor.default-dispatcher-17] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-9, sourceActorSystem=ClusterSystem, akkaSource=akka://ClusterSystem/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A42282-207/endpointWriter, akkaTimestamp=20:00:03.446UTC] - no longer watched by Actor[akka://ClusterSystem/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A42282-207#1490820524] DEBUG | 2016-06-22 20:00:03 | [ClusterSystem-akka.actor.default-dispatcher-17] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-9, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:43209/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A42282-207/endpointWriter, akkaTimestamp=20:00:03.446UTC] - Disassociated [akka.tcp://ClusterSystem@hostname2:43209] -> [akka.tcp://ClusterSystem@hostname1:42282] DEBUG | 2016-06-22 20:00:03 | [ClusterSystem-akka.actor.default-dispatcher-17] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-9, sourceActorSystem=ClusterSystem, akkaSource=akka://ClusterSystem/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A42282-207/endpointWriter, akkaTimestamp=20:00:03.446UTC] - stopped DEBUG | 2016-06-22 20:00:03 | [ClusterSystem-akka.actor.default-dispatcher-17] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-8, sourceActorSystem=ClusterSystem, akkaSource=akka://ClusterSystem/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A42282-207, akkaTimestamp=20:00:03.446UTC] - stopped DEBUG | 2016-06-22 20:00:03 | [ClusterSystem-akka.actor.default-dispatcher-17] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-9, sourceActorSystem=ClusterSystem, akkaSource=akka://ClusterSystem/system/endpointManager, akkaTimestamp=20:00:03.446UTC] - received AutoReceiveMessage Envelope(Terminated(Actor[akka://ClusterSystem/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A42282-207#1490820524]),Actor[akka://ClusterSystem/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A42282-207#1490820524]) DEBUG | 2016-06-22 20:00:03 | [ClusterSystem-akka.actor.default-dispatcher-16] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-6, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A42282-4/endpointWriter, akkaTimestamp=20:00:03.504UTC] - received local message RemoteMessage: [ActorSelectionMessage(akka.cluster.GossipEnvelope@215f3fdb,Vector(system, cluster, core, daemon),false)] to [Actor[akka://ClusterSystem/]]<+[akka://ClusterSystem/] from [Actor[akka.tcp://ClusterSystem@hostname1:42282/system/cluster/core/daemon#-1814189187]()] DEBUG | 2016-06-22 20:00:03 | [ClusterSystem-akka.actor.default-dispatcher-16] ? - [sourceThread=ClusterSystem-akka.actor.default-dispatcher-3, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/cluster/core/daemon, akkaTimestamp=20:00:03.504UTC] - Cluster Node [akka.tcp://ClusterSystem@hostname2:2552] - Receiving gossip from [UniqueAddress(akka.tcp://ClusterSystem@hostname1:42282,1853147177)] DEBUG | 2016-06-22 20:00:03 | [ClusterSystem-akka.actor.default-dispatcher-3] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-5, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:03.636UTC] - sending message RemoteMessage: [ActorSelectionMessage(GossipStatus(UniqueAddress(akka.tcp://ClusterSystem@hostname2:2552,235402593),VectorClock(0674e80a1aa6eaa2990d3dab1b9beedc -> 1, b7a6f622639ec0187ab601337796f207 -> 7, bec77a033021fc1ab6448cb0c1af53fd -> 1, e191b0013b1bc705772a35b0e9cdbca3 -> 1)),Vector(system, cluster, core, daemon),false)] to [Actor[akka.tcp://ClusterSystem@hostname1:2551/]]<+[akka.tcp://ClusterSystem@hostname1:2551/] from [Actor[akka://ClusterSystem/system/cluster/core/daemon#-150228074]] DEBUG | 2016-06-22 20:00:03 | [ClusterSystem-akka.actor.default-dispatcher-3] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-5, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:03.696UTC] - sending message RemoteMessage: [ActorSelectionMessage(Status(Map(akka.tcp://ClusterSystem@hostname1:2551 -> 1466624412169)),Vector(system, distributedPubSubMediator),false)] to [Actor[akka.tcp://ClusterSystem@hostname1:2551/]]<+[akka.tcp://ClusterSystem@hostname1:2551/] from [Actor[akka://ClusterSystem/system/distributedPubSubMediator#638476632]] DEBUG | 2016-06-22 20:00:03 | [ClusterSystem-akka.actor.default-dispatcher-3] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-6, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2Fcmnet-WorkerSystem%40127.0.0.1%3A46637-7/endpointWriter, akkaTimestamp=20:00:03.795UTC] - received local message RemoteMessage: [Heartbeat] to [Actor[akka://ClusterSystem/system/receptionist#-1830908093]]<+[akka://ClusterSystem/system/receptionist] from [Actor[akka.tcp://[email protected]:46637/user/clusterClient#-808306789]()] DEBUG | 2016-06-22 20:00:03 | [ClusterSystem-akka.actor.default-dispatcher-3] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-6, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2Fcmnet-WorkerSystem%40127.0.0.1%3A46637-7/endpointWriter, akkaTimestamp=20:00:03.795UTC] - sending message RemoteMessage: [HeartbeatRsp] to [Actor[akka.tcp://[email protected]:46637/user/clusterClient#-808306789]]<+[akka.tcp://[email protected]:46637/user/clusterClient] from [Actor[akka://ClusterSystem/system/receptionist#-1830908093]] DEBUG | 2016-06-22 20:00:03 | [ClusterSystem-akka.actor.default-dispatcher-3] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-6, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A42282-4/endpointWriter, akkaTimestamp=20:00:03.864UTC] - received local message RemoteMessage: [ActorSelectionMessage(Heartbeat(akka.tcp://ClusterSystem@hostname1:42282),Vector(system, cluster, heartbeatReceiver),false)] to [Actor[akka://ClusterSystem/]]<+[akka://ClusterSystem/] from [Actor[akka.tcp://ClusterSystem@hostname1:42282/system/cluster/core/daemon/heartbeatSender#1072477085]()] DEBUG | 2016-06-22 20:00:03 | [ClusterSystem-akka.actor.default-dispatcher-3] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-6, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A42282-4/endpointWriter, akkaTimestamp=20:00:03.864UTC] - sending message RemoteMessage: [HeartbeatRsp(UniqueAddress(akka.tcp://ClusterSystem@hostname2:2552,235402593))] to [Actor[akka.tcp://ClusterSystem@hostname1:42282/system/cluster/core/daemon/heartbeatSender#1072477085]]<+[akka.tcp://ClusterSystem@hostname1:42282/system/cluster/core/daemon/heartbeatSender] from [Actor[akka://ClusterSystem/system/cluster/heartbeatReceiver#-670753192]] DEBUG | 2016-06-22 20:00:03 | [ClusterSystem-akka.actor.default-dispatcher-2] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-5, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:03.896UTC] - sending message RemoteMessage: [ActorSelectionMessage(MetricsGossipEnvelope(akka.tcp://ClusterSystem@hostname2:2552,MetricsGossip(Set(NodeMetrics(akka.tcp://ClusterSystem@hostname2:2552,1466625600897,Set(Metric(heap-memory-used,154272488,Some(EWMA(1.54272488E8,0.15910417746033534))), Metric(heap-memory-max,7460618240,None), Metric(processors,8,None), Metric(system-load-average,0.0068359375,None), Metric(heap-memory-committed,1372585984,Some(EWMA(1.372585984E9,0.15910417746033534))))), NodeMetrics(akka.tcp://ClusterSystem@hostname1:2551,1466625598253,Set(Metric(heap-memory-used,671845048,Some(EWMA(6.71845048E8,0.15910417746033534))), Metric(heap-memory-max,7460618240,None), Metric(processors,8,None), Metric(system-load-average,0.00732421875,None), Metric(heap-memory-committed,1200619520,Some(EWMA(1.20061952E9,0.15910417746033534))))))),false),Vector(system, cluster, metrics),false)] to [Actor[akka.tcp://ClusterSystem@hostname1:2551/]]<+[akka.tcp://ClusterSystem@hostname1:2551/] from [Actor[akka://ClusterSystem/system/cluster/metrics#1040761959]] DEBUG | 2016-06-22 20:00:03 | [ClusterSystem-akka.actor.default-dispatcher-2] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-5, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:03.898UTC] - received local message RemoteMessage: [ActorSelectionMessage(MetricsGossipEnvelope(akka.tcp://ClusterSystem@hostname1:2551,MetricsGossip(Set(NodeMetrics(akka.tcp://ClusterSystem@hostname2:2552,1466625600897,Set(Metric(heap-memory-used,154272488,Some(EWMA(1.54272488E8,0.15910417746033534))), Metric(heap-memory-max,7460618240,None), Metric(processors,8,None), Metric(system-load-average,0.0068359375,None), Metric(heap-memory-committed,1372585984,Some(EWMA(1.372585984E9,0.15910417746033534))))), NodeMetrics(akka.tcp://ClusterSystem@hostname1:2551,1466625601253,Set(Metric(heap-memory-used,680555144,Some(EWMA(6.80555144E8,0.15910417746033534))), Metric(heap-memory-max,7460618240,None), Metric(processors,8,None), Metric(system-load-average,0.0068359375,None), Metric(heap-memory-committed,1200619520,Some(EWMA(1.20061952E9,0.15910417746033534))))))),true),Vector(system, cluster, metrics),false)] to [Actor[akka://ClusterSystem/]]<+[akka://ClusterSystem/] from [Actor[akka.tcp://ClusterSystem@hostname1:2551/system/cluster/metrics#616453818]()] DEBUG | 2016-06-22 20:00:04 | [ClusterSystem-akka.actor.default-dispatcher-2] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-6, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:04.017UTC] - received local message RemoteMessage: [ActorSelectionMessage(GossipStatus(UniqueAddress(akka.tcp://ClusterSystem@hostname1:2551,598411574),VectorClock(0674e80a1aa6eaa2990d3dab1b9beedc -> 1, b7a6f622639ec0187ab601337796f207 -> 7, bec77a033021fc1ab6448cb0c1af53fd -> 1, e191b0013b1bc705772a35b0e9cdbca3 -> 1)),Vector(system, cluster, core, daemon),false)] to [Actor[akka://ClusterSystem/]]<+[akka://ClusterSystem/] from [Actor[akka.tcp://ClusterSystem@hostname1:2551/system/cluster/core/daemon#1532363050]()] DEBUG | 2016-06-22 20:00:04 | [ClusterSystem-akka.actor.default-dispatcher-17] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-9, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:43209/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:04.067UTC] - received local message RemoteMessage: [ActorSelectionMessage(Status(Map(akka.tcp://ClusterSystem@hostname1:2551 -> 1466624412169)),Vector(system, distributedPubSubMediator),false)] to [Actor[akka://ClusterSystem/]]<+[akka://ClusterSystem/] from [Actor[akka.tcp://ClusterSystem@hostname1:2551/system/distributedPubSubMediator#-379718827]()] DEBUG | 2016-06-22 20:00:04 | [ClusterSystem-akka.actor.default-dispatcher-17] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-9, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:43209/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:04.087UTC] - received local message RemoteMessage: [ActorSelectionMessage(Heartbeat(akka.tcp://ClusterSystem@hostname1:2551),Vector(system, cluster, heartbeatReceiver),false)] to [Actor[akka://ClusterSystem/]]<+[akka://ClusterSystem/] from [Actor[akka.tcp://ClusterSystem@hostname1:2551/system/cluster/core/daemon/heartbeatSender#-1740097823]()] DEBUG | 2016-06-22 20:00:04 | [ClusterSystem-akka.actor.default-dispatcher-2] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-6, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:04.087UTC] - received local message RemoteMessage: [ActorSelectionMessage(Heartbeat(akka.tcp://ClusterSystem@hostname1:2551),Vector(system, cluster, heartbeatReceiver),false)] to [Actor[akka://ClusterSystem/]]<+[akka://ClusterSystem/] from [Actor[akka.tcp://ClusterSystem@hostname1:2551/system/cluster/core/daemon/heartbeatSender#-1740097823]()] DEBUG | 2016-06-22 20:00:04 | [ClusterSystem-akka.actor.default-dispatcher-2] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-6, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:04.087UTC] - sending message RemoteMessage: [HeartbeatRsp(UniqueAddress(akka.tcp://ClusterSystem@hostname2:2552,235402593))] to [Actor[akka.tcp://ClusterSystem@hostname1:2551/system/cluster/core/daemon/heartbeatSender#-1740097823]]<+[akka.tcp://ClusterSystem@hostname1:2551/system/cluster/core/daemon/heartbeatSender] from [Actor[akka://ClusterSystem/system/cluster/heartbeatReceiver#-670753192]] DEBUG | 2016-06-22 20:00:04 | [ClusterSystem-akka.actor.default-dispatcher-18] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-8, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:43209/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:04.087UTC] - sending message RemoteMessage: [HeartbeatRsp(UniqueAddress(akka.tcp://ClusterSystem@hostname2:43209,-951268098))] to [Actor[akka.tcp://ClusterSystem@hostname1:2551/system/cluster/core/daemon/heartbeatSender#-1740097823]]<+[akka.tcp://ClusterSystem@hostname1:2551/system/cluster/core/daemon/heartbeatSender] from [Actor[akka://ClusterSystem/system/cluster/heartbeatReceiver#-787172203]] DEBUG | 2016-06-22 20:00:04 | [ClusterSystem-akka.actor.default-dispatcher-2] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-6, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:04.257UTC] - received local message RemoteMessage: [ActorSelectionMessage(MetricsGossipEnvelope(akka.tcp://ClusterSystem@hostname1:2551,MetricsGossip(Set(NodeMetrics(akka.tcp://ClusterSystem@hostname2:2552,1466625600897,Set(Metric(heap-memory-used,154272488,Some(EWMA(1.54272488E8,0.15910417746033534))), Metric(heap-memory-max,7460618240,None), Metric(processors,8,None), Metric(system-load-average,0.0068359375,None), Metric(heap-memory-committed,1372585984,Some(EWMA(1.372585984E9,0.15910417746033534))))), NodeMetrics(akka.tcp://ClusterSystem@hostname1:2551,1466625604253,Set(Metric(heap-memory-used,686958472,Some(EWMA(6.86958472E8,0.15910417746033534))), Metric(heap-memory-max,7460618240,None), Metric(processors,8,None), Metric(system-load-average,0.16650390625,None), Metric(heap-memory-committed,1200619520,Some(EWMA(1.20061952E9,0.15910417746033534))))))),false),Vector(system, cluster, metrics),false)] to [Actor[akka://ClusterSystem/]]<+[akka://ClusterSystem/] from [Actor[akka.tcp://ClusterSystem@hostname1:2551/system/cluster/metrics#616453818]()] DEBUG | 2016-06-22 20:00:04 | [ClusterSystem-akka.actor.default-dispatcher-16] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-6, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:04.257UTC] - sending message RemoteMessage: [ActorSelectionMessage(MetricsGossipEnvelope(akka.tcp://ClusterSystem@hostname2:2552,MetricsGossip(Set(NodeMetrics(akka.tcp://ClusterSystem@hostname2:2552,1466625603896,Set(Metric(heap-memory-used,157777760,Some(EWMA(1.5777776E8,0.15910417746033534))), Metric(heap-memory-max,7460618240,None), Metric(processors,8,None), Metric(system-load-average,0.00634765625,None), Metric(heap-memory-committed,1372585984,Some(EWMA(1.372585984E9,0.15910417746033534))))), NodeMetrics(akka.tcp://ClusterSystem@hostname1:2551,1466625604253,Set(Metric(heap-memory-used,686958472,Some(EWMA(6.86958472E8,0.15910417746033534))), Metric(heap-memory-max,7460618240,None), Metric(processors,8,None), Metric(system-load-average,0.16650390625,None), Metric(heap-memory-committed,1200619520,Some(EWMA(1.20061952E9,0.15910417746033534))))))),true),Vector(system, cluster, metrics),false)] to [Actor[akka.tcp://ClusterSystem@hostname1:2551/]]<+[akka.tcp://ClusterSystem@hostname1:2551/] from [Actor[akka://ClusterSystem/system/cluster/metrics#1040761959]] DEBUG | 2016-06-22 20:00:04 | [ClusterSystem-akka.actor.default-dispatcher-21] ? - [sourceThread=ClusterSystem-akka.remote.default-remote-dispatcher-8, sourceActorSystem=ClusterSystem, akkaSource=akka.tcp://ClusterSystem@hostname2:43209/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40hostname1%3A2551-0/endpointWriter, akkaTimestamp=20:00:04.384UTC] - sending message RemoteMessage: [ActorSelectionMessage(akka.cluster.GossipEnvelope@6f48ca9a,Vector(system, cluster, core, daemon),false)] to [Actor[akka.tcp://ClusterSystem@hostname1:2551/]]<+[akka.tcp://ClusterSystem@hostname1:2551/] from [Actor[akka://ClusterSystem/system/cluster/core/daemon#-1375582141]]
