Can you share the hazelcast.xml?
On Tue, Aug 5, 2014 at 7:17 PM, Nirmal Fernando <[email protected]> wrote: > Thanks Azeez. Yes, that would be my next step. > > > On Tue, Aug 5, 2014 at 7:14 PM, Afkham Azeez <[email protected]> wrote: > >> Looks ok. Rather than creating multiple Hz instances within the same JVM, >> you should be creating a single Hz instance per JVM to replicate our >> clustering implementation more closely. >> >> >> On Tue, Aug 5, 2014 at 9:17 AM, Nirmal Fernando <[email protected]> wrote: >> >>> Hi Azeez, >>> >>> I ran the following test case and results showed that members whose >>> static initial members are shutdown, can still see new members. Let me know >>> if you see any issue in this test case. >>> >>> *TEST* >>> >>> public static void main(String[] args) { >>> System.setProperty("hazelcast.config", >>> "src/main/resources/hazelcast.xml"); >>> // Build Hazelcast cluster >>> System.out.println("Starting instance h1"); >>> HazelcastInstance h1 = Hazelcast.newHazelcastInstance(); >>> System.out.println("Starting instance h2"); >>> >>> HazelcastInstance h2 = Hazelcast.newHazelcastInstance(); >>> >>> IMap<String, String> map1 = h1.getMap("map"); >>> map1.put("nirmal", "fernando"); >>> >>> IMap<String, String> map2 = h2.getMap("map"); >>> System.out.println("from h2: " + map2.get("nirmal")); >>> map2.set("nirmal", "fdo"); >>> System.out.println("from h1: " + map1.get("nirmal")); >>> >>> System.out.println("Starting instance h3"); >>> HazelcastInstance h3 = Hazelcast.newHazelcastInstance(); >>> System.out.println("members of h1 : >>> "+h1.getCluster().getMembers().size()); >>> System.out.println("members of h2: >>> "+h2.getCluster().getMembers().size()); >>> System.out.println("members of h3: >>> "+h3.getCluster().getMembers().size()); >>> >>> h1.getLifecycleService().shutdown(); >>> System.out.println("Stopping instance h1"); >>> h2.getLifecycleService().shutdown(); >>> System.out.println("Stopping instance h2"); >>> >>> System.out.println("members of h3 after h1 is down: >>> "+h3.getCluster().getMembers().size()); >>> >>> HazelcastInstance h4 = Hazelcast.newHazelcastInstance(); >>> System.out.println("Starting instance h4"); >>> >>> System.out.println("members of h3 after h4 is up : >>> "+h3.getCluster().getMembers().size()); >>> System.out.println("members of h4 after h4 is up : >>> "+h4.getCluster().getMembers().size()); >>> >>> System.exit(0); >>> } >>> >>> *RESULT* >>> >>> Starting instance h1 >>> Starting instance h2 >>> Starting instance h3 >>> >>> members of h1 : 3 >>> members of h2: 3 >>> members of h3: 3 >>> >>> Stopping instance h1 >>> Stopping instance h2 >>> members of h3 after h1 is down: 1 >>> >>> Starting instance h4 >>> members of h3 after h4 is up : 2 >>> members of h4 after h4 is up : 2 >>> >>> >>> *DETAILED RESULT* >>> >>> Starting instance h1 >>> Aug 5, 2014 12:35:11 AM com.hazelcast.config.XmlConfigBuilder >>> INFO: Using configuration file at >>> /wso2/workspace/HazelcastTest/src/main/resources/hazelcast.xml >>> Aug 5, 2014 12:35:11 AM com.hazelcast.config.XmlConfigBuilder >>> INFO: Using configuration file at >>> /wso2/workspace/HazelcastTest/src/main/resources/hazelcast.xml >>> Aug 5, 2014 12:35:11 AM com.hazelcast.instance.DefaultAddressPicker >>> INFO: null [dev] [3.2.4] Interfaces is disabled, trying to pick one >>> address from TCP-IP config addresses: [127.0.0.1] >>> Aug 5, 2014 12:35:11 AM com.hazelcast.instance.DefaultAddressPicker >>> INFO: null [dev] [3.2.4] Picked Address[127.0.0.1]:5701, using socket >>> ServerSocket[addr=/0:0:0:0:0:0:0:0,localport=5701], bind any local is true >>> Aug 5, 2014 12:35:12 AM com.hazelcast.system >>> INFO: [127.0.0.1]:5701 [dev] [3.2.4] Hazelcast 3.2.4 (20140721) starting >>> at Address[127.0.0.1]:5701 >>> Aug 5, 2014 12:35:12 AM com.hazelcast.system >>> INFO: [127.0.0.1]:5701 [dev] [3.2.4] Copyright (C) 2008-2014 >>> Hazelcast.com >>> Aug 5, 2014 12:35:12 AM com.hazelcast.instance.Node >>> INFO: [127.0.0.1]:5701 [dev] [3.2.4] Creating TcpIpJoiner >>> Aug 5, 2014 12:35:12 AM com.hazelcast.core.LifecycleService >>> INFO: [127.0.0.1]:5701 [dev] [3.2.4] Address[127.0.0.1]:5701 is STARTING >>> Aug 5, 2014 12:35:12 AM com.hazelcast.cluster.TcpIpJoiner >>> INFO: [127.0.0.1]:5701 [dev] [3.2.4] Connecting to possible member: >>> Address[127.0.0.1]:5703 >>> Aug 5, 2014 12:35:12 AM com.hazelcast.cluster.TcpIpJoiner >>> INFO: [127.0.0.1]:5701 [dev] [3.2.4] Connecting to possible member: >>> Address[127.0.0.1]:5702 >>> Aug 5, 2014 12:35:12 AM com.hazelcast.nio.SocketConnector >>> INFO: [127.0.0.1]:5701 [dev] [3.2.4] Connecting to /127.0.0.1:5702, >>> timeout: 0, bind-any: true >>> Aug 5, 2014 12:35:12 AM com.hazelcast.nio.SocketConnector >>> INFO: [127.0.0.1]:5701 [dev] [3.2.4] Connecting to /127.0.0.1:5703, >>> timeout: 0, bind-any: true >>> Aug 5, 2014 12:35:12 AM com.hazelcast.nio.SocketConnector >>> INFO: [127.0.0.1]:5701 [dev] [3.2.4] Could not connect to: / >>> 127.0.0.1:5703. Reason: SocketException[Connection refused to address / >>> 127.0.0.1:5703] >>> Aug 5, 2014 12:35:12 AM com.hazelcast.nio.SocketConnector >>> INFO: [127.0.0.1]:5701 [dev] [3.2.4] Could not connect to: / >>> 127.0.0.1:5702. Reason: SocketException[Connection refused to address / >>> 127.0.0.1:5702] >>> Aug 5, 2014 12:35:13 AM com.hazelcast.cluster.TcpIpJoiner >>> INFO: [127.0.0.1]:5701 [dev] [3.2.4] >>> >>> >>> Members [1] { >>> Member [127.0.0.1]:5701 this >>> } >>> >>> Starting instance h2 >>> Aug 5, 2014 12:35:13 AM com.hazelcast.core.LifecycleService >>> INFO: [127.0.0.1]:5701 [dev] [3.2.4] Address[127.0.0.1]:5701 is STARTED >>> Aug 5, 2014 12:35:13 AM com.hazelcast.config.XmlConfigBuilder >>> INFO: Using configuration file at >>> /wso2/workspace/HazelcastTest/src/main/resources/hazelcast.xml >>> Aug 5, 2014 12:35:13 AM com.hazelcast.config.XmlConfigBuilder >>> INFO: Using configuration file at >>> /wso2/workspace/HazelcastTest/src/main/resources/hazelcast.xml >>> Aug 5, 2014 12:35:13 AM com.hazelcast.instance.DefaultAddressPicker >>> INFO: null [dev] [3.2.4] Interfaces is disabled, trying to pick one >>> address from TCP-IP config addresses: [127.0.0.1] >>> Aug 5, 2014 12:35:13 AM com.hazelcast.instance.DefaultAddressPicker >>> INFO: null [dev] [3.2.4] Picked Address[127.0.0.1]:5702, using socket >>> ServerSocket[addr=/0:0:0:0:0:0:0:0,localport=5702], bind any local is true >>> Aug 5, 2014 12:35:13 AM com.hazelcast.system >>> INFO: [127.0.0.1]:5702 [dev] [3.2.4] Hazelcast 3.2.4 (20140721) starting >>> at Address[127.0.0.1]:5702 >>> Aug 5, 2014 12:35:13 AM com.hazelcast.system >>> INFO: [127.0.0.1]:5702 [dev] [3.2.4] Copyright (C) 2008-2014 >>> Hazelcast.com >>> Aug 5, 2014 12:35:13 AM com.hazelcast.instance.Node >>> INFO: [127.0.0.1]:5702 [dev] [3.2.4] Creating TcpIpJoiner >>> Aug 5, 2014 12:35:13 AM com.hazelcast.core.LifecycleService >>> INFO: [127.0.0.1]:5702 [dev] [3.2.4] Address[127.0.0.1]:5702 is STARTING >>> Aug 5, 2014 12:35:13 AM com.hazelcast.cluster.TcpIpJoiner >>> INFO: [127.0.0.1]:5702 [dev] [3.2.4] Connecting to possible member: >>> Address[127.0.0.1]:5703 >>> Aug 5, 2014 12:35:13 AM com.hazelcast.cluster.TcpIpJoiner >>> INFO: [127.0.0.1]:5702 [dev] [3.2.4] Connecting to possible member: >>> Address[127.0.0.1]:5701 >>> Aug 5, 2014 12:35:13 AM com.hazelcast.nio.SocketConnector >>> INFO: [127.0.0.1]:5702 [dev] [3.2.4] Connecting to /127.0.0.1:5703, >>> timeout: 0, bind-any: true >>> Aug 5, 2014 12:35:13 AM com.hazelcast.nio.SocketConnector >>> INFO: [127.0.0.1]:5702 [dev] [3.2.4] Connecting to /127.0.0.1:5701, >>> timeout: 0, bind-any: true >>> Aug 5, 2014 12:35:13 AM com.hazelcast.nio.SocketConnector >>> INFO: [127.0.0.1]:5702 [dev] [3.2.4] Could not connect to: / >>> 127.0.0.1:5703. Reason: SocketException[Connection refused to address / >>> 127.0.0.1:5703] >>> Aug 5, 2014 12:35:13 AM com.hazelcast.nio.SocketAcceptor >>> INFO: [127.0.0.1]:5701 [dev] [3.2.4] Accepting socket connection from / >>> 127.0.0.1:50666 >>> Aug 5, 2014 12:35:13 AM com.hazelcast.nio.TcpIpConnectionManager >>> INFO: [127.0.0.1]:5701 [dev] [3.2.4] 5701 accepted socket connection >>> from /127.0.0.1:50666 >>> Aug 5, 2014 12:35:13 AM com.hazelcast.nio.TcpIpConnectionManager >>> INFO: [127.0.0.1]:5702 [dev] [3.2.4] 50666 accepted socket connection >>> from /127.0.0.1:5701 >>> Aug 5, 2014 12:35:19 AM com.hazelcast.cluster.ClusterService >>> INFO: [127.0.0.1]:5701 [dev] [3.2.4] >>> >>> Members [2] { >>> >>> Member [127.0.0.1]:5701 this >>> Member [127.0.0.1]:5702 >>> } >>> >>> Aug 5, 2014 12:35:19 AM com.hazelcast.cluster.ClusterService >>> INFO: [127.0.0.1]:5702 [dev] [3.2.4] >>> >>> Members [2] { >>> >>> Member [127.0.0.1]:5701 >>> Member [127.0.0.1]:5702 this >>> } >>> >>> Aug 5, 2014 12:35:21 AM com.hazelcast.core.LifecycleService >>> INFO: [127.0.0.1]:5702 [dev] [3.2.4] Address[127.0.0.1]:5702 is STARTED >>> Aug 5, 2014 12:35:21 AM com.hazelcast.partition.InternalPartitionService >>> INFO: [127.0.0.1]:5701 [dev] [3.2.4] Initializing cluster partition >>> table first arrangement... >>> from h2: fernando >>> from h1: fdo >>> Starting instance h3 >>> Aug 5, 2014 12:35:26 AM com.hazelcast.config.XmlConfigBuilder >>> INFO: Using configuration file at >>> /wso2/workspace/HazelcastTest/src/main/resources/hazelcast.xml >>> Aug 5, 2014 12:35:26 AM com.hazelcast.config.XmlConfigBuilder >>> INFO: Using configuration file at >>> /wso2/workspace/HazelcastTest/src/main/resources/hazelcast.xml >>> Aug 5, 2014 12:35:26 AM com.hazelcast.instance.DefaultAddressPicker >>> INFO: null [dev] [3.2.4] Interfaces is disabled, trying to pick one >>> address from TCP-IP config addresses: [127.0.0.1] >>> Aug 5, 2014 12:35:26 AM com.hazelcast.instance.DefaultAddressPicker >>> INFO: null [dev] [3.2.4] Picked Address[127.0.0.1]:5703, using socket >>> ServerSocket[addr=/0:0:0:0:0:0:0:0,localport=5703], bind any local is true >>> Aug 5, 2014 12:35:26 AM com.hazelcast.system >>> INFO: [127.0.0.1]:5703 [dev] [3.2.4] Hazelcast 3.2.4 (20140721) starting >>> at Address[127.0.0.1]:5703 >>> Aug 5, 2014 12:35:26 AM com.hazelcast.system >>> INFO: [127.0.0.1]:5703 [dev] [3.2.4] Copyright (C) 2008-2014 >>> Hazelcast.com >>> Aug 5, 2014 12:35:26 AM com.hazelcast.instance.Node >>> INFO: [127.0.0.1]:5703 [dev] [3.2.4] Creating TcpIpJoiner >>> Aug 5, 2014 12:35:26 AM com.hazelcast.core.LifecycleService >>> INFO: [127.0.0.1]:5703 [dev] [3.2.4] Address[127.0.0.1]:5703 is STARTING >>> Aug 5, 2014 12:35:26 AM com.hazelcast.cluster.TcpIpJoiner >>> INFO: [127.0.0.1]:5703 [dev] [3.2.4] Connecting to possible member: >>> Address[127.0.0.1]:5702 >>> Aug 5, 2014 12:35:26 AM com.hazelcast.cluster.TcpIpJoiner >>> INFO: [127.0.0.1]:5703 [dev] [3.2.4] Connecting to possible member: >>> Address[127.0.0.1]:5701 >>> Aug 5, 2014 12:35:26 AM com.hazelcast.nio.SocketConnector >>> INFO: [127.0.0.1]:5703 [dev] [3.2.4] Connecting to /127.0.0.1:5702, >>> timeout: 0, bind-any: true >>> Aug 5, 2014 12:35:26 AM com.hazelcast.nio.SocketConnector >>> INFO: [127.0.0.1]:5703 [dev] [3.2.4] Connecting to /127.0.0.1:5701, >>> timeout: 0, bind-any: true >>> Aug 5, 2014 12:35:26 AM com.hazelcast.nio.SocketAcceptor >>> INFO: [127.0.0.1]:5702 [dev] [3.2.4] Accepting socket connection from / >>> 127.0.0.1:57737 >>> Aug 5, 2014 12:35:26 AM com.hazelcast.nio.SocketAcceptor >>> INFO: [127.0.0.1]:5701 [dev] [3.2.4] Accepting socket connection from / >>> 127.0.0.1:46497 >>> Aug 5, 2014 12:35:26 AM com.hazelcast.nio.TcpIpConnectionManager >>> INFO: [127.0.0.1]:5703 [dev] [3.2.4] 57737 accepted socket connection >>> from /127.0.0.1:5702 >>> Aug 5, 2014 12:35:26 AM com.hazelcast.nio.TcpIpConnectionManager >>> INFO: [127.0.0.1]:5701 [dev] [3.2.4] 5701 accepted socket connection >>> from /127.0.0.1:46497 >>> Aug 5, 2014 12:35:26 AM com.hazelcast.nio.TcpIpConnectionManager >>> INFO: [127.0.0.1]:5703 [dev] [3.2.4] 46497 accepted socket connection >>> from /127.0.0.1:5701 >>> Aug 5, 2014 12:35:26 AM com.hazelcast.nio.TcpIpConnectionManager >>> INFO: [127.0.0.1]:5702 [dev] [3.2.4] 5702 accepted socket connection >>> from /127.0.0.1:57737 >>> Aug 5, 2014 12:35:32 AM com.hazelcast.cluster.ClusterService >>> INFO: [127.0.0.1]:5702 [dev] [3.2.4] >>> >>> Members [3] { >>> Member [127.0.0.1]:5701 >>> Member [127.0.0.1]:5702 this >>> Member [127.0.0.1]:5703 >>> } >>> >>> Aug 5, 2014 12:35:32 AM com.hazelcast.cluster.ClusterService >>> INFO: [127.0.0.1]:5703 [dev] [3.2.4] >>> >>> Members [3] { >>> Member [127.0.0.1]:5701 >>> Member [127.0.0.1]:5702 >>> Member [127.0.0.1]:5703 this >>> } >>> >>> Aug 5, 2014 12:35:32 AM com.hazelcast.cluster.ClusterService >>> INFO: [127.0.0.1]:5701 [dev] [3.2.4] >>> >>> Members [3] { >>> Member [127.0.0.1]:5701 this >>> Member [127.0.0.1]:5702 >>> Member [127.0.0.1]:5703 >>> } >>> >>> Aug 5, 2014 12:35:32 AM com.hazelcast.partition.InternalPartitionService >>> INFO: [127.0.0.1]:5701 [dev] [3.2.4] Re-partitioning cluster data... >>> Migration queue size: 90 >>> Aug 5, 2014 12:35:34 AM com.hazelcast.partition.InternalPartitionService >>> INFO: [127.0.0.1]:5701 [dev] [3.2.4] All migration tasks has been >>> completed, queues are empty. >>> Aug 5, 2014 12:35:34 AM com.hazelcast.core.LifecycleService >>> INFO: [127.0.0.1]:5703 [dev] [3.2.4] Address[127.0.0.1]:5703 is STARTED >>> members of h1 : 3 >>> members of h2: 3 >>> members of h3: 3 >>> Aug 5, 2014 12:35:34 AM com.hazelcast.core.LifecycleService >>> INFO: [127.0.0.1]:5701 [dev] [3.2.4] Address[127.0.0.1]:5701 is >>> SHUTTING_DOWN >>> Aug 5, 2014 12:35:34 AM com.hazelcast.cluster.ClusterService >>> INFO: [127.0.0.1]:5703 [dev] [3.2.4] Master Address[127.0.0.1]:5701 left >>> the cluster. Assigning new master Member [127.0.0.1]:5702 >>> Aug 5, 2014 12:35:34 AM com.hazelcast.cluster.ClusterService >>> INFO: [127.0.0.1]:5702 [dev] [3.2.4] Master Address[127.0.0.1]:5701 left >>> the cluster. Assigning new master Member [127.0.0.1]:5702 this >>> Aug 5, 2014 12:35:34 AM com.hazelcast.nio.TcpIpConnection >>> INFO: [127.0.0.1]:5702 [dev] [3.2.4] Connection >>> [Address[127.0.0.1]:5701] lost. Reason: Socket explicitly closed >>> Aug 5, 2014 12:35:34 AM com.hazelcast.nio.TcpIpConnection >>> INFO: [127.0.0.1]:5703 [dev] [3.2.4] Connection >>> [Address[127.0.0.1]:5701] lost. Reason: Socket explicitly closed >>> Aug 5, 2014 12:35:34 AM com.hazelcast.nio.TcpIpConnection >>> INFO: [127.0.0.1]:5701 [dev] [3.2.4] Connection >>> [Address[127.0.0.1]:5703] lost. Reason: java.io.EOFException[Remote socket >>> closed!] >>> Aug 5, 2014 12:35:34 AM com.hazelcast.nio.TcpIpConnection >>> INFO: [127.0.0.1]:5701 [dev] [3.2.4] Connection >>> [Address[127.0.0.1]:5702] lost. Reason: java.io.EOFException[Remote socket >>> closed!] >>> Aug 5, 2014 12:35:34 AM com.hazelcast.cluster.ClusterService >>> INFO: [127.0.0.1]:5702 [dev] [3.2.4] Removing Member [127.0.0.1]:5701 >>> Aug 5, 2014 12:35:34 AM com.hazelcast.cluster.ClusterService >>> INFO: [127.0.0.1]:5703 [dev] [3.2.4] Removing Member [127.0.0.1]:5701 >>> Aug 5, 2014 12:35:34 AM com.hazelcast.cluster.ClusterService >>> INFO: [127.0.0.1]:5703 [dev] [3.2.4] >>> >>> Members [2] { >>> >>> Member [127.0.0.1]:5702 >>> Member [127.0.0.1]:5703 this >>> } >>> >>> Aug 5, 2014 12:35:34 AM com.hazelcast.cluster.ClusterService >>> INFO: [127.0.0.1]:5702 [dev] [3.2.4] >>> >>> Members [2] { >>> >>> Member [127.0.0.1]:5702 this >>> Member [127.0.0.1]:5703 >>> } >>> >>> Aug 5, 2014 12:35:34 AM com.hazelcast.initializer >>> INFO: [127.0.0.1]:5701 [dev] [3.2.4] Destroying node initializer. >>> Aug 5, 2014 12:35:34 AM com.hazelcast.instance.Node >>> INFO: [127.0.0.1]:5701 [dev] [3.2.4] Hazelcast Shutdown is completed in >>> 45 ms. >>> Aug 5, 2014 12:35:34 AM com.hazelcast.core.LifecycleService >>> INFO: [127.0.0.1]:5701 [dev] [3.2.4] Address[127.0.0.1]:5701 is SHUTDOWN >>> Stopping instance h1 >>> Aug 5, 2014 12:35:34 AM com.hazelcast.core.LifecycleService >>> INFO: [127.0.0.1]:5702 [dev] [3.2.4] Address[127.0.0.1]:5702 is >>> SHUTTING_DOWN >>> Aug 5, 2014 12:35:34 AM com.hazelcast.partition.InternalPartitionService >>> INFO: [127.0.0.1]:5702 [dev] [3.2.4] Waiting for cluster migration >>> tasks: 1 >>> Aug 5, 2014 12:35:35 AM com.hazelcast.partition.InternalPartitionService >>> INFO: [127.0.0.1]:5702 [dev] [3.2.4] Waiting for cluster migration >>> tasks: 1 >>> Aug 5, 2014 12:35:36 AM com.hazelcast.partition.InternalPartitionService >>> INFO: [127.0.0.1]:5702 [dev] [3.2.4] Partition balance is ok, no need to >>> re-partition cluster data... >>> Aug 5, 2014 12:35:36 AM com.hazelcast.cluster.ClusterService >>> INFO: [127.0.0.1]:5703 [dev] [3.2.4] Master Address[127.0.0.1]:5702 left >>> the cluster. Assigning new master Member [127.0.0.1]:5703 this >>> Aug 5, 2014 12:35:36 AM com.hazelcast.nio.TcpIpConnection >>> INFO: [127.0.0.1]:5703 [dev] [3.2.4] Connection >>> [Address[127.0.0.1]:5702] lost. Reason: Socket explicitly closed >>> Aug 5, 2014 12:35:36 AM com.hazelcast.nio.TcpIpConnection >>> INFO: [127.0.0.1]:5702 [dev] [3.2.4] Connection >>> [Address[127.0.0.1]:5703] lost. Reason: java.io.EOFException[Remote socket >>> closed!] >>> Aug 5, 2014 12:35:36 AM com.hazelcast.cluster.ClusterService >>> INFO: [127.0.0.1]:5703 [dev] [3.2.4] Removing Member [127.0.0.1]:5702 >>> Aug 5, 2014 12:35:36 AM com.hazelcast.cluster.ClusterService >>> INFO: [127.0.0.1]:5703 [dev] [3.2.4] >>> >>> Members [1] { >>> Member [127.0.0.1]:5703 this >>> } >>> >>> Aug 5, 2014 12:35:36 AM com.hazelcast.initializer >>> INFO: [127.0.0.1]:5702 [dev] [3.2.4] Destroying node initializer. >>> Aug 5, 2014 12:35:36 AM com.hazelcast.instance.Node >>> INFO: [127.0.0.1]:5702 [dev] [3.2.4] Hazelcast Shutdown is completed in >>> 2035 ms. >>> Aug 5, 2014 12:35:36 AM com.hazelcast.core.LifecycleService >>> INFO: [127.0.0.1]:5702 [dev] [3.2.4] Address[127.0.0.1]:5702 is SHUTDOWN >>> Stopping instance h2Aug 5, 2014 12:35:36 AM >>> com.hazelcast.config.XmlConfigBuilder >>> INFO: Using configuration file at >>> /wso2/workspace/HazelcastTest/src/main/resources/hazelcast.xml >>> Aug 5, 2014 12:35:36 AM com.hazelcast.config.XmlConfigBuilder >>> INFO: Using configuration file at >>> /wso2/workspace/HazelcastTest/src/main/resources/hazelcast.xml >>> >>> members of h3 after h1 is down: 1 >>> Aug 5, 2014 12:35:36 AM com.hazelcast.instance.DefaultAddressPicker >>> INFO: null [dev] [3.2.4] Interfaces is disabled, trying to pick one >>> address from TCP-IP config addresses: [127.0.0.1] >>> Aug 5, 2014 12:35:36 AM com.hazelcast.instance.DefaultAddressPicker >>> INFO: null [dev] [3.2.4] Picked Address[127.0.0.1]:5701, using socket >>> ServerSocket[addr=/0:0:0:0:0:0:0:0,localport=5701], bind any local is true >>> Aug 5, 2014 12:35:36 AM com.hazelcast.system >>> INFO: [127.0.0.1]:5701 [dev] [3.2.4] Hazelcast 3.2.4 (20140721) starting >>> at Address[127.0.0.1]:5701 >>> Aug 5, 2014 12:35:36 AM com.hazelcast.system >>> INFO: [127.0.0.1]:5701 [dev] [3.2.4] Copyright (C) 2008-2014 >>> Hazelcast.com >>> Aug 5, 2014 12:35:36 AM com.hazelcast.instance.Node >>> INFO: [127.0.0.1]:5701 [dev] [3.2.4] Creating TcpIpJoiner >>> Aug 5, 2014 12:35:36 AM com.hazelcast.core.LifecycleService >>> INFO: [127.0.0.1]:5701 [dev] [3.2.4] Address[127.0.0.1]:5701 is STARTING >>> Aug 5, 2014 12:35:36 AM com.hazelcast.cluster.TcpIpJoiner >>> INFO: [127.0.0.1]:5701 [dev] [3.2.4] Connecting to possible member: >>> Address[127.0.0.1]:5703 >>> Aug 5, 2014 12:35:36 AM com.hazelcast.cluster.TcpIpJoiner >>> INFO: [127.0.0.1]:5701 [dev] [3.2.4] Connecting to possible member: >>> Address[127.0.0.1]:5702 >>> Aug 5, 2014 12:35:36 AM com.hazelcast.nio.SocketConnector >>> INFO: [127.0.0.1]:5701 [dev] [3.2.4] Connecting to /127.0.0.1:5703, >>> timeout: 0, bind-any: true >>> Aug 5, 2014 12:35:36 AM com.hazelcast.nio.SocketConnector >>> INFO: [127.0.0.1]:5701 [dev] [3.2.4] Connecting to /127.0.0.1:5702, >>> timeout: 0, bind-any: true >>> Aug 5, 2014 12:35:36 AM com.hazelcast.nio.SocketAcceptor >>> INFO: [127.0.0.1]:5703 [dev] [3.2.4] Accepting socket connection from / >>> 127.0.0.1:53551 >>> Aug 5, 2014 12:35:36 AM com.hazelcast.nio.TcpIpConnectionManager >>> INFO: [127.0.0.1]:5701 [dev] [3.2.4] 53551 accepted socket connection >>> from /127.0.0.1:5703 >>> Aug 5, 2014 12:35:36 AM com.hazelcast.nio.TcpIpConnectionManager >>> INFO: [127.0.0.1]:5703 [dev] [3.2.4] 5703 accepted socket connection >>> from /127.0.0.1:53551 >>> Aug 5, 2014 12:35:36 AM com.hazelcast.nio.SocketConnector >>> INFO: [127.0.0.1]:5701 [dev] [3.2.4] Could not connect to: / >>> 127.0.0.1:5702. Reason: SocketException[Connection refused to address / >>> 127.0.0.1:5702] >>> Aug 5, 2014 12:35:38 AM com.hazelcast.partition.InternalPartitionService >>> INFO: [127.0.0.1]:5703 [dev] [3.2.4] Partition balance is ok, no need to >>> re-partition cluster data... >>> Aug 5, 2014 12:35:42 AM com.hazelcast.cluster.ClusterService >>> INFO: [127.0.0.1]:5703 [dev] [3.2.4] >>> >>> Members [2] { >>> Member [127.0.0.1]:5703 this >>> Member [127.0.0.1]:5701 >>> } >>> >>> Aug 5, 2014 12:35:42 AM com.hazelcast.cluster.ClusterService >>> INFO: [127.0.0.1]:5701 [dev] [3.2.4] >>> >>> Members [2] { >>> Member [127.0.0.1]:5703 >>> Member [127.0.0.1]:5701 this >>> } >>> >>> Aug 5, 2014 12:35:42 AM com.hazelcast.partition.InternalPartitionService >>> INFO: [127.0.0.1]:5703 [dev] [3.2.4] Re-partitioning cluster data... >>> Migration queue size: 135 >>> Aug 5, 2014 12:35:44 AM com.hazelcast.partition.InternalPartitionService >>> INFO: [127.0.0.1]:5703 [dev] [3.2.4] All migration tasks has been >>> completed, queues are empty. >>> Aug 5, 2014 12:35:44 AM com.hazelcast.core.LifecycleService >>> INFO: [127.0.0.1]:5701 [dev] [3.2.4] Address[127.0.0.1]:5701 is STARTED >>> Starting instance h4 >>> members of h3 after h4 is up : 2 >>> members of h4 after h4 is up : 2 >>> >>> >>> On Sat, Aug 2, 2014 at 2:41 PM, Afkham Azeez <[email protected]> wrote: >>> >>>> Dynamic discovery happens after the static nodes are added, and the >>>> static nodes being available is crucial for the cluster to work. >>>> >>>> >>>> On Sat, Aug 2, 2014 at 11:04 AM, Nirmal Fernando <[email protected]> >>>> wrote: >>>> >>>>> Hi Azeez, >>>>> >>>>> Thanks for the replies. Sorry, if I misunderstood the Hazelcast >>>>> TcpIpJoiner, but I did following test on Hazelcast tcp/ip [1] and it seems >>>>> each node discovers every other dynamically [2]. My Hazelcast.xml file is >>>>> [3]. Does my test case incorrect? Appreciate your thoughts. >>>>> >>>>> [1] >>>>> >>>>> public static void main(String[] args) { >>>>> System.setProperty("hazelcast.config", >>>>> "src/main/resources/hazelcast.xml"); >>>>> // Build Hazelcast cluster >>>>> System.out.println("Starting instance 1"); >>>>> HazelcastInstance h1 = Hazelcast.newHazelcastInstance(); >>>>> System.out.println("Starting instance 2"); >>>>> HazelcastInstance h2 = Hazelcast.newHazelcastInstance(); >>>>> >>>>> IMap<String, String> map1 = h1.getMap("map"); >>>>> map1.put("nirmal", "fernando"); >>>>> >>>>> IMap<String, String> map2 = h2.getMap("map"); >>>>> System.out.println("from h2: " + map2.get("nirmal")); >>>>> map2.set("nirmal", "fdo"); >>>>> System.out.println("from h1: " + map1.get("nirm")); >>>>> >>>>> HazelcastInstance h3 = Hazelcast.newHazelcastInstance(); >>>>> System.out.println(h1.getCluster().getMembers().size()); >>>>> System.out.println(h2.getCluster().getMembers().size()); >>>>> System.out.println(h3.getCluster().getMembers().size()); >>>>> System.exit(0); >>>>> } >>>>> >>>>> [2] >>>>> >>>>> ..... >>>>> >>>>> Members [3] { >>>>> Member [127.0.0.1]:5701 this >>>>> Member [127.0.0.1]:5702 >>>>> Member [127.0.0.1]:5703 >>>>> } >>>>> >>>>> Aug 2, 2014 10:59:17 AM com.hazelcast.cluster.ClusterService >>>>> INFO: [127.0.0.1]:5703 [dev] >>>>> >>>>> Members [3] { >>>>> Member [127.0.0.1]:5701 >>>>> Member [127.0.0.1]:5702 >>>>> Member [127.0.0.1]:5703 this >>>>> } >>>>> >>>>> Aug 2, 2014 10:59:17 AM com.hazelcast.cluster.ClusterService >>>>> INFO: [127.0.0.1]:5702 [dev] >>>>> >>>>> Members [3] { >>>>> Member [127.0.0.1]:5701 >>>>> Member [127.0.0.1]:5702 this >>>>> Member [127.0.0.1]:5703 >>>>> } >>>>> >>>>> Aug 2, 2014 10:59:17 AM com.hazelcast.partition.PartitionService >>>>> INFO: [127.0.0.1]:5701 [dev] Re-partitioning cluster data... Migration >>>>> queue size: 90 >>>>> Aug 2, 2014 10:59:19 AM com.hazelcast.core.LifecycleService >>>>> INFO: [127.0.0.1]:5703 [dev] Address[127.0.0.1]:5703 is STARTED >>>>> 3 >>>>> 3 >>>>> 3 >>>>> >>>>> [3] >>>>> >>>>> <hazelcast xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; >>>>> xsi:schemaLocation="http://www.hazelcast.com/schema/config >>>>> http://www.hazelcast.com/schema/config/hazelcast-config-3.2.xsd"; >>>>> xmlns="http://www.hazelcast.com/schema/config";> >>>>> <network> >>>>> <join> >>>>> <tcp-ip enabled="true"> >>>>> <member>127.0.0.1</member> >>>>> </tcp-ip> >>>>> </join> >>>>> </network> >>>>> >>>>> </hazelcast> >>>>> >>>>> >>>>> >>>>> >>>>> On Sat, Aug 2, 2014 at 10:25 AM, Afkham Azeez <[email protected]> wrote: >>>>> >>>>>> Anyway, there is a catch here. The membership scheme uses TCP/IP >>>>>> underneath, and adds the members it sees as static members. But when all >>>>>> the members each node added fails, that node has to be restarted. So, >>>>>> when >>>>>> the cluster if formed, node 1 will not have any WK members; node 2 will >>>>>> have 1 WK member, node 3 will have 2 and so on. So if node 1 fails, node >>>>>> 2 >>>>>> also fails, if nodes 1 & 2 fail, node 3 also fails. >>>>>> >>>>>> >>>>>> On Sat, Aug 2, 2014 at 10:10 AM, Nirmal Fernando <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Hi All, >>>>>>> >>>>>>> In a Cloud environment, it's not ideal to mark one or more WKA >>>>>>> members since it brings lot of challenges such as; >>>>>>> >>>>>>> 1. Keeping WKA members up and running always >>>>>>> 2. If they got destroyed spin up replacement WKA members and respawn >>>>>>> the whole cluster. >>>>>>> etc. >>>>>>> >>>>>>> These could possibly lead to lot of instability to the cluster and >>>>>>> in turn affect the sole purpose of clustering. >>>>>>> >>>>>>> According to [1] and also to the Hazelcast book, Hazelcast has a >>>>>>> solution to this problem for AWS EC2. Here I quote [1]. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> *Hazelcast either uses Multicast or TCP/IP for discovery, but EC2 >>>>>>> does not support multicast. To configure discovery using TCP/IP, you >>>>>>> need >>>>>>> the IP addresses upfront and this is not always possible. To solve this >>>>>>> problem, Hazelcast supports EC2 auto discovery which is a layer on top >>>>>>> ofTCP/IP discovery. EC2 auto discovery uses AWS API to get the IP >>>>>>> addresses >>>>>>> of possible Hazelcast nodes and feeds those IP addresses to TCP/IP >>>>>>> discovery. This way the discovery process becomes dynamic and it >>>>>>> eliminates >>>>>>> a need for knowing the IP addresses upfront. To limit theIP addresses >>>>>>> only >>>>>>> to Hazelcast related nodes, EC2 discovery supports filtering based on >>>>>>> security group and/or tags.* >>>>>>> >>>>>>> Current idea is to use tags to specify the cluster domain and need >>>>>>> to research more and come up with a design. Further, we could leverage >>>>>>> Hazelcast's partition groups to support HA across Availability zones. >>>>>>> >>>>>>> Also, if this is only for EC2, that would not be much useful. But >>>>>>> Hazelcast seems to have an extension point to support other Clouds via >>>>>>> JClouds. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> *In case you are using a different cloud provider than Amazon EC2, >>>>>>> you can still make use of Hazelcast. What you can do it to use the >>>>>>> programmatic api toconfigure a tcp-ip cluster and the well known members >>>>>>> need to be retrieved fromyour cloud provider (e.g. using jclouds).* >>>>>>> >>>>>>> In addition to this, at [1] Hazelcast describes some best practices >>>>>>> to use in AWS EC2 Cloud. And I think we could leverage these in many of >>>>>>> the >>>>>>> real world deployments. >>>>>>> >>>>>>> I hope it's feasible to port this support to Carbon Clustering and >>>>>>> I'd like to work on it. Let me know your thoughts. >>>>>>> >>>>>>> [1] http://hazelcast.com/resources/amazon-ec2-deployment-guide/ >>>>>>> >>>>>>> -- >>>>>>> >>>>>>> Thanks & regards, >>>>>>> Nirmal >>>>>>> >>>>>>> Senior Software Engineer- Platform Technologies Team, WSO2 Inc. >>>>>>> Mobile: +94715779733 >>>>>>> Blog: http://nirmalfdo.blogspot.com/ >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> *Afkham Azeez* >>>>>> Director of Architecture; WSO2, Inc.; http://wso2.com >>>>>> Member; Apache Software Foundation; http://www.apache.org/ >>>>>> * <http://www.apache.org/>* >>>>>> *email: **[email protected]* <[email protected]> >>>>>> * cell: +94 77 3320919 <%2B94%2077%203320919> blog: * >>>>>> *http://blog.afkham.org* <http://blog.afkham.org> >>>>>> *twitter: **http://twitter.com/afkham_azeez* >>>>>> <http://twitter.com/afkham_azeez> >>>>>> * linked-in: **http://lk.linkedin.com/in/afkhamazeez >>>>>> <http://lk.linkedin.com/in/afkhamazeez>* >>>>>> >>>>>> *Lean . Enterprise . Middleware* >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> Thanks & regards, >>>>> Nirmal >>>>> >>>>> Senior Software Engineer- Platform Technologies Team, WSO2 Inc. >>>>> Mobile: +94715779733 >>>>> Blog: http://nirmalfdo.blogspot.com/ >>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> *Afkham Azeez* >>>> Director of Architecture; WSO2, Inc.; http://wso2.com >>>> Member; Apache Software Foundation; http://www.apache.org/ >>>> * <http://www.apache.org/>* >>>> *email: **[email protected]* <[email protected]> >>>> * cell: +94 77 3320919 <%2B94%2077%203320919> blog: * >>>> *http://blog.afkham.org* <http://blog.afkham.org> >>>> *twitter: **http://twitter.com/afkham_azeez* >>>> <http://twitter.com/afkham_azeez> >>>> * linked-in: **http://lk.linkedin.com/in/afkhamazeez >>>> <http://lk.linkedin.com/in/afkhamazeez>* >>>> >>>> *Lean . Enterprise . Middleware* >>>> >>> >>> >>> >>> -- >>> >>> Thanks & regards, >>> Nirmal >>> >>> Senior Software Engineer- Platform Technologies Team, WSO2 Inc. >>> Mobile: +94715779733 >>> Blog: http://nirmalfdo.blogspot.com/ >>> >>> >>> >> >> >> -- >> *Afkham Azeez* >> Director of Architecture; WSO2, Inc.; http://wso2.com >> Member; Apache Software Foundation; http://www.apache.org/ >> * <http://www.apache.org/>* >> *email: **[email protected]* <[email protected]> >> * cell: +94 77 3320919 <%2B94%2077%203320919> blog: * >> *http://blog.afkham.org* <http://blog.afkham.org> >> *twitter: **http://twitter.com/afkham_azeez* >> <http://twitter.com/afkham_azeez> >> * linked-in: **http://lk.linkedin.com/in/afkhamazeez >> <http://lk.linkedin.com/in/afkhamazeez>* >> >> *Lean . Enterprise . Middleware* >> > > > > -- > > Thanks & regards, > Nirmal > > Senior Software Engineer- Platform Technologies Team, WSO2 Inc. > Mobile: +94715779733 > Blog: http://nirmalfdo.blogspot.com/ > > > -- *Afkham Azeez* Director of Architecture; WSO2, Inc.; http://wso2.com Member; Apache Software Foundation; http://www.apache.org/ * <http://www.apache.org/>* *email: **[email protected]* <[email protected]> * cell: +94 77 3320919 blog: **http://blog.afkham.org* <http://blog.afkham.org> *twitter: **http://twitter.com/afkham_azeez* <http://twitter.com/afkham_azeez> * linked-in: **http://lk.linkedin.com/in/afkhamazeez <http://lk.linkedin.com/in/afkhamazeez>* *Lean . Enterprise . Middleware*
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
