No. It is because the cluster was not properly set. I could access other nodes from these nodes. Even though the cluster was setup it seems nodes does not recognize other nodes as members.
On Tue, Oct 27, 2015 at 2:02 PM, Imesh Gunaratne <[email protected]> wrote: > On Tue, Oct 27, 2015 at 12:55 PM, Maninda Edirisooriya <[email protected]> > wrote: > >> >> Here I have started each node one after the other. And once a member was >> stopped also any message was not printed. >> > > Then it might be a connection issue between nodes. > > On Tue, Oct 27, 2015 at 12:55 PM, Maninda Edirisooriya <[email protected]> > wrote: > >> We have not tested AWS clustering with old kernel. We moved to AWS >> clustering with new kernel. I thought it was due to new kernel version as I >> have heard AWS was working properly with products released with older >> kernel. >> Here I have started each node one after the other. And once a member was >> stopped also any message was not printed. >> >> >> >> On Mon, Oct 26, 2015 at 11:26 PM, Imesh Gunaratne <[email protected]> wrote: >> >>> Member joined message may not come with AWS membership scheme if all the >>> members were started at the same time. However the member left message >>> should appear if a member is stopped afterwards. >>> >>> >>> On Monday, October 26, 2015, Maninda Edirisooriya <[email protected]> >>> wrote: >>> >>>> I have setup the AWS clustering according to [1] but it seems the >>>> "AWSBasedMembershipScheme >>>> Member joined" message is not displayed once a new member is joined >>>> and no message when it left. This seems to be an issue with new kernel >>>> version. >>>> >>>> <clustering >>>> class="org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent" >>>> enable="true"> >>>> >>>> >>>> <parameter name="membershipScheme">aws</parameter> >>>> >>>> >>>> <parameter name="localMemberPort">5701</parameter> >>>> >>>> >>>> <parameter name="accessKey">xxxxxxx</parameter> >>>> <parameter name="secretKey">zzzzzzz</parameter> >>>> <parameter name="securityGroup">sg-16f4b571</parameter> >>>> <parameter name="region">us-east-1</parameter> >>>> <parameter name="tagKey">a_tag_key</parameter> >>>> <parameter name="tagValue">a_tag_value</parameter> >>>> >>>> >>>> [1] >>>> https://docs.wso2.com/display/CLUSTER420/Setting+up+a+Cluster+in+AWS+Mode >>>> >>>> >>>> Thanks. >>>> >>>> >>>> *Maninda Edirisooriya* >>>> Senior Software Engineer >>>> >>>> *WSO2, Inc.*lean.enterprise.middleware. >>>> >>>> *Blog* : http://maninda.blogspot.com/ >>>> *E-mail* : [email protected] >>>> *Skype* : @manindae >>>> *Twitter* : @maninda >>>> >>>> On Sun, Oct 25, 2015 at 6:59 PM, Afkham Azeez <[email protected]> wrote: >>>> >>>>> In EC2 with AWS mode, you should use ports 5701 to 5800 >>>>> >>>>> On Sun, Oct 25, 2015 at 5:52 PM, Niranda Perera <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi Thusitha, >>>>>> >>>>>> 4000 is the local host port specified in the axis2.xml >>>>>> >>>>>> quote axis2.xml >>>>>> " <!-- The host name or IP address of this member --> >>>>>> >>>>>> <parameter name="localMemberHost">10.100.5.109</parameter> >>>>>> >>>>>> <!-- >>>>>> The bind adress of this member. The difference between >>>>>> localMemberHost & localMemberBindAddress >>>>>> is that localMemberHost is the one that is advertised by >>>>>> this member, while localMemberBindAddress >>>>>> is the address to which this member is bound to. >>>>>> --> >>>>>> <!-- >>>>>> <parameter >>>>>> name="localMemberBindAddress">10.100.5.109</parameter> >>>>>> --> >>>>>> >>>>>> <!-- >>>>>> The TCP port used by this member. This is the port through >>>>>> which other nodes will >>>>>> contact this member >>>>>> --> >>>>>> <parameter name="localMemberPort">4000</parameter>" >>>>>> >>>>>> in DAS we are using wka as the membership scheme. >>>>>> >>>>>> cheers >>>>>> >>>>>> On Sun, Oct 25, 2015 at 4:47 PM, Thusitha Thilina Dayaratne < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Hi Anjana, >>>>>>> >>>>>>> I noticed following error line on the error stack that you posted. >>>>>>> >>>>>>> com.hazelcast.spi.exception.WrongTargetException: WrongTarget! >>>>>>>> this:Address[172.18.1.227]:4000 >>>>>>> >>>>>>> >>>>>>> Here it tries to connect using the port 4000. But AFAIK we usually >>>>>>> use ports between 5701 to 5800. Did you set the port to 4000 in any >>>>>>> instance? >>>>>>> >>>>>>> Thanks >>>>>>> Thusitha >>>>>>> >>>>>>> On Sat, Oct 24, 2015 at 5:27 PM, Anjana Fernando <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> So I'd a chat with Azeez, and the first thing he mentioned was, we >>>>>>>> should be using the AWS mode in EC2 it seems, the docs can be found for >>>>>>>> this here [1]. And his comments are, these kind of issues can be seen >>>>>>>> sometimes there, but if it recovers eventually it would be okay, but >>>>>>>> if it >>>>>>>> continues to give same kind of errors always, that would be a problem. >>>>>>>> >>>>>>>> [1] >>>>>>>> https://docs.wso2.com/display/CLUSTER420/Setting+up+a+Cluster+in+AWS+Mode >>>>>>>> >>>>>>>> Cheers, >>>>>>>> Anjana. >>>>>>>> >>>>>>>> On Sat, Oct 24, 2015 at 5:11 PM, Anjana Fernando <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi everyone, >>>>>>>>> >>>>>>>>> When testing DAS in EC2, and also with the latest Kernel 4.4.2, QA >>>>>>>>> has reported some Hazlecast issues, specifically [1] and [2]. Some of >>>>>>>>> the >>>>>>>>> errors as follows:- >>>>>>>>> >>>>>>>>> TID: [-1234] [] [2015-10-24 08:40:05,130] ERROR >>>>>>>>> {org.wso2.carbon.registry.core.jdbc.handlers.builtin.MountHandler} - >>>>>>>>> Unable to test existence of resource >>>>>>>>> {org.wso2.carbon.registry.core.jdbc.handlers.builtin.MountHandler} >>>>>>>>> com.hazelcast.spi.exception.WrongTargetException: WrongTarget! >>>>>>>>> this:Address[172.18.1.227]:4000, target:Address[172.18.1.228]:4000, >>>>>>>>> partitionId: 5, replicaIndex: 0, operation: >>>>>>>>> com.hazelcast.map.impl.operation.GetOperation, service: >>>>>>>>> hz:impl:mapService >>>>>>>>> at >>>>>>>>> com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.ensureNoPartitionProblems(OperationRunnerImpl.java:246) >>>>>>>>> at >>>>>>>>> com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:127) >>>>>>>>> at >>>>>>>>> com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:315) >>>>>>>>> at >>>>>>>>> com.hazelcast.spi.impl.operationexecutor.classic.OperationThread.processPacket(OperationThread.java:142) >>>>>>>>> at >>>>>>>>> com.hazelcast.spi.impl.operationexecutor.classic.OperationThread.process(OperationThread.java:115) >>>>>>>>> at >>>>>>>>> com.hazelcast.spi.impl.operationexecutor.classic.OperationThread.doRun(OperationThread.java:101) >>>>>>>>> at >>>>>>>>> com.hazelcast.spi.impl.operationexecutor.classic.OperationThread.run(OperationThread.java:76) >>>>>>>>> at ------ End remote and begin local stack-trace >>>>>>>>> ------.(Unknown Source) >>>>>>>>> at >>>>>>>>> com.hazelcast.spi.impl.operationservice.impl.InvocationFuture.resolveApplicationResponse(InvocationFuture.java:384) >>>>>>>>> at >>>>>>>>> com.hazelcast.spi.impl.operationservice.impl.InvocationFuture.resolveApplicationResponseOrThrowException(InvocationFuture.java:334) >>>>>>>>> at >>>>>>>>> com.hazelcast.spi.impl.operationservice.impl.InvocationFuture.get(InvocationFuture.java:225) >>>>>>>>> at >>>>>>>>> com.hazelcast.spi.impl.operationservice.impl.InvocationFuture.get(InvocationFuture.java:204) >>>>>>>>> at >>>>>>>>> com.hazelcast.map.impl.proxy.MapProxySupport.invokeOperation(MapProxySupport.java:456) >>>>>>>>> at >>>>>>>>> com.hazelcast.map.impl.proxy.MapProxySupport.getInternal(MapProxySupport.java:275) >>>>>>>>> at >>>>>>>>> com.hazelcast.map.impl.proxy.MapProxyImpl.get(MapProxyImpl.java:82) >>>>>>>>> at >>>>>>>>> org.wso2.carbon.core.clustering.hazelcast.HazelcastDistributedMapProvider$DistMap.get(HazelcastDistributedMapProvider.java:141) >>>>>>>>> at >>>>>>>>> org.wso2.carbon.caching.impl.CacheImpl.get(CacheImpl.java:203) >>>>>>>>> >>>>>>>>> >>>>>>>>> ==================================================================== >>>>>>>>> >>>>>>>>> >>>>>>>>> TID: [-1234] [] [2015-10-24 08:44:18,800] ERROR >>>>>>>>> {org.wso2.carbon.registry.core.jdbc.handlers.builtin.MountHandler} - >>>>>>>>> Unable to test existence of resource >>>>>>>>> {org.wso2.carbon.registry.core.jdbc.handlers.builtin.MountHandler} >>>>>>>>> com.hazelcast.spi.exception.CallerNotMemberException: Not Member! >>>>>>>>> caller:Address[172.18.1.228]:4000, partitionId: 30, operation: >>>>>>>>> com.hazelcast.map.impl.operation.GetOperation, service: >>>>>>>>> hz:impl:mapService >>>>>>>>> at >>>>>>>>> com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.ensureValidMember(OperationRunnerImpl.java:336) >>>>>>>>> at >>>>>>>>> com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:308) >>>>>>>>> at >>>>>>>>> com.hazelcast.spi.impl.operationexecutor.classic.OperationThread.processPacket(OperationThread.java:142) >>>>>>>>> at >>>>>>>>> com.hazelcast.spi.impl.operationexecutor.classic.OperationThread.process(OperationThread.java:115) >>>>>>>>> >>>>>>>>> >>>>>>>>> ==================================================================== >>>>>>>>> >>>>>>>>> ERROR {com.hazelcast.cluster.impl.operations.JoinCheckOperation} >>>>>>>>> - [172.18.1.229]:4000 [wso2.qa.das.domain] [3.5.2] Cannot send >>>>>>>>> response: >>>>>>>>> JoinRequest{packetVersion=4, buildNumber=20150826, >>>>>>>>> address=Address[172.18.1.229]:4000, >>>>>>>>> uuid='6848559f-141d-47cd-9e75-53bb2de10d52', credentials=null, >>>>>>>>> memberCount=4, tryCount=0} to Address[172.18.1.228]:4000 >>>>>>>>> {com.hazelcast.cluster.impl.operations.JoinCheckOperation} >>>>>>>>> com.hazelcast.core.HazelcastException: Cannot send response: >>>>>>>>> JoinRequest{packetVersion=4, buildNumber=20150826, >>>>>>>>> address=Address[172.18.1.229]:4000, >>>>>>>>> uuid='6848559f-141d-47cd-9e75-53bb2de10d52', credentials=null, >>>>>>>>> memberCount=4, tryCount=0} to Address[172.18.1.228]:4000 >>>>>>>>> at >>>>>>>>> com.hazelcast.spi.impl.ResponseHandlerFactory$RemoteInvocationResponseHandler.sendResponse(ResponseHandlerFactory.java:131) >>>>>>>>> at >>>>>>>>> com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.handleResponse(OperationRunnerImpl.java:204) >>>>>>>>> >>>>>>>>> >>>>>>>>> ==================================================================== >>>>>>>>> >>>>>>>>> Have any one of you guys seen issues similar to the above in EC2? >>>>>>>>> .. and what are the possibilities this could arise, the most obvious >>>>>>>>> thing >>>>>>>>> would be networking issues that can arise in EC2, which maybe related >>>>>>>>> to a >>>>>>>>> higher latency that's there or something. Any of you guys know how to >>>>>>>>> maybe >>>>>>>>> tune HZ in this situations to have a higher timeout values or >>>>>>>>> anything like >>>>>>>>> that, or else, doing this in EC2 is altogether not recommended? .. >>>>>>>>> >>>>>>>>> [1] https://wso2.org/jira/browse/DAS-301 >>>>>>>>> [2] https://wso2.org/jira/browse/DAS-302 >>>>>>>>> >>>>>>>>> Cheers, >>>>>>>>> Anjana. >>>>>>>>> -- >>>>>>>>> *Anjana Fernando* >>>>>>>>> Senior Technical Lead >>>>>>>>> WSO2 Inc. | http://wso2.com >>>>>>>>> lean . enterprise . middleware >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> *Anjana Fernando* >>>>>>>> Senior Technical Lead >>>>>>>> WSO2 Inc. | http://wso2.com >>>>>>>> lean . enterprise . middleware >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Dev mailing list >>>>>>>> [email protected] >>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Thusitha Dayaratne >>>>>>> Software Engineer >>>>>>> WSO2 Inc. - lean . enterprise . middleware | wso2.com >>>>>>> >>>>>>> Mobile +94712756809 >>>>>>> Blog alokayasoya.blogspot.com >>>>>>> About http://about.me/thusithathilina >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Dev mailing list >>>>>>> [email protected] >>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> *Niranda Perera* >>>>>> Software Engineer, WSO2 Inc. >>>>>> Mobile: +94-71-554-8430 >>>>>> Twitter: @n1r44 <https://twitter.com/N1R44> >>>>>> https://pythagoreanscript.wordpress.com/ >>>>>> >>>>>> _______________________________________________ >>>>>> Dev mailing list >>>>>> [email protected] >>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> *Afkham Azeez* >>>>> Director of Architecture; WSO2, Inc.; http://wso2.com >>>>> Member; Apache Software Foundation; http://www.apache.org/ >>>>> * <http://www.apache.org/>* >>>>> *email: **[email protected]* >>>>> * cell: +94 77 3320919 <%2B94%2077%203320919>blog: * >>>>> *http://blog.afkham.org* <http://blog.afkham.org> >>>>> *twitter: **http://twitter.com/afkham_azeez* >>>>> <http://twitter.com/afkham_azeez> >>>>> *linked-in: **http://lk.linkedin.com/in/afkhamazeez >>>>> <http://lk.linkedin.com/in/afkhamazeez>* >>>>> >>>>> *Lean . Enterprise . Middleware* >>>>> >>>>> _______________________________________________ >>>>> Dev mailing list >>>>> [email protected] >>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>> >>>>> >>>> >>> >>> -- >>> *Imesh Gunaratne* >>> Senior Technical Lead >>> WSO2 Inc: http://wso2.com >>> T: +94 11 214 5345 M: +94 77 374 2057 >>> W: http://imesh.gunaratne.org >>> Lean . Enterprise . Middleware >>> >>> >>> >> > > > -- > *Imesh Gunaratne* > Senior Technical Lead > WSO2 Inc: http://wso2.com > T: +94 11 214 5345 M: +94 77 374 2057 > W: http://imesh.gunaratne.org > Lean . Enterprise . Middleware > >
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
