Looping in Azeez and Dinusha
On Wed, May 13, 2015 at 2:33 PM, Nuwan Silva <[email protected]> wrote: > Hi All, > > I came across the following issue where we see a depsync error when a > single node in the cluster goes down and the other nodes not knowing it > till its restarted. > observation was that when a node (node 1) in the cluster goes down the > other nodes (node 2 and node 3) do not know that node (node 1) is down. So > it tries to synchronize its artifacts. > > further we observed that when the node which was down (node 1) restarts. > the other nodes find out that node1 was down. > > TID: [0] [AM] [2015-05-13 09:00:08,861] INFO > {org.wso2.carbon.core.clustering.hazelcast.wka.WKABasedMembershipScheme} - > Member left [cc9950e3-af57-48b1-9f45-b511f0f30863]: /192.168.48.2:4001 > {org.wso2.carbon.core.clustering.hazelcast.wka.WKABasedMembershipScheme} > TID: [0] [AM] [2015-05-13 09:00:08,877] INFO > {org.wso2.carbon.core.clustering.hazelcast.wka.WKABasedMembershipScheme} - > WKA member Host:192.168.48.2, Remote Host:null, Port: 4001, HTTP:8281, > HTTPS:8244, Domain: wso2.am.storepub.domain, Sub-domain:worker, Active:true > left cluster. > {org.wso2.carbon.core.clustering.hazelcast.wka.WKABasedMembershipScheme} > TID: [0] [AM] [2015-05-13 09:00:08,890] INFO > {org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent} - > Elected this member [10bb1dec-4d25-4966-a39a-eb42e42d1706] as the > Coordinator for the cluster [wso2.am.storepub.domain] > {org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent} > TID: [0] [AM] [2015-05-13 09:00:14,849] INFO > {org.wso2.carbon.core.clustering.hazelcast.wka.WKABasedMembershipScheme} - > Member joined [bc438078-ee97-4643-aadb-cf7ba970dc7e]: /192.168.48.2:4001 > {org.wso2.carbon.core.clustering.hazelcast.wka.WKABasedMembershipScheme} > > > > *Depsync Error:* > > TID: [0] [AM] [2015-05-13 08:52:03,333] ERROR > {org.wso2.carbon.core.deployment.CarbonDeploymentSchedulerTask} - > Deployment synchronization commit for tenant -1234 failed > {org.wso2.carbon.core.deployment.CarbonDeploymentSchedulerTask} > com.hazelcast.core.HazelcastException: > com.hazelcast.spi.exception.RetryableIOException: Packet not sent to -> > Address[192.168.48.5]:4000 > at com.hazelcast.util.ExceptionUtil.rethrow(ExceptionUtil.java:45) > at com.hazelcast.util.ExceptionUtil.rethrow(ExceptionUtil.java:40) > at > com.hazelcast.map.proxy.MapProxySupport.containsKeyInternal(MapProxySupport.java:371) > at > com.hazelcast.map.proxy.MapProxyImpl.containsKey(MapProxyImpl.java:215) > at > org.wso2.carbon.core.clustering.hazelcast.HazelcastDistributedMapProvider$DistMap.containsKey(HazelcastDistributedMapProvider.java:119) > at > org.wso2.carbon.caching.impl.CacheImpl.containsKey(CacheImpl.java:237) > at > org.wso2.carbon.registry.core.caching.CacheBackedRegistry.resourceExists(CacheBackedRegistry.java:291) > at > org.wso2.carbon.registry.core.session.UserRegistry.resourceExistsInternal(UserRegistry.java:777) > at > org.wso2.carbon.registry.core.session.UserRegistry.access$800(UserRegistry.java:60) > at > org.wso2.carbon.registry.core.session.UserRegistry$9.run(UserRegistry.java:760) > at > org.wso2.carbon.registry.core.session.UserRegistry$9.run(UserRegistry.java:757) > at java.security.AccessController.doPrivileged(Native Method) > at > org.wso2.carbon.registry.core.session.UserRegistry.resourceExists(UserRegistry.java:757) > at > org.wso2.carbon.deployment.synchronizer.internal.repository.CarbonRepositoryUtils.getDeploymentSyncConfigurationFromRegistry(CarbonRepositoryUtils.java:262) > at > org.wso2.carbon.deployment.synchronizer.internal.repository.CarbonRepositoryUtils.getActiveSynchronizerConfiguration(CarbonRepositoryUtils.java:108) > at > org.wso2.carbon.deployment.synchronizer.internal.DeploymentSynchronizerServiceImpl.commit(DeploymentSynchronizerServiceImpl.java:96) > at > org.wso2.carbon.core.deployment.CarbonDeploymentSchedulerTask.deploymentSyncCommit(CarbonDeploymentSchedulerTask.java:207) > at > org.wso2.carbon.core.deployment.CarbonDeploymentSchedulerTask.run(CarbonDeploymentSchedulerTask.java:128) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > Caused by: com.hazelcast.spi.exception.RetryableIOException: Packet not > sent to -> Address[192.168.48.5]:4000 > at > com.hazelcast.spi.impl.BasicInvocation.doInvoke(BasicInvocation.java:360) > at > com.hazelcast.spi.impl.BasicInvocation.access$900(BasicInvocation.java:62) > at > com.hazelcast.spi.impl.BasicInvocation$ReInvocationTask.run(BasicInvocation.java:549) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > at > com.hazelcast.util.executor.PoolExecutorThreadFactory$ManagedThread.run(PoolExecutorThreadFactory.java:59) > at ------ End remote and begin local stack-trace ------.(Unknown > Source) > at > com.hazelcast.spi.impl.BasicInvocation$InvocationFuture.resolveResponse(BasicInvocation.java:862) > at > com.hazelcast.spi.impl.BasicInvocation$InvocationFuture.resolveResponseOrThrowException(BasicInvocation.java:795) > at > com.hazelcast.spi.impl.BasicInvocation$InvocationFuture.get(BasicInvocation.java:698) > at > com.hazelcast.spi.impl.BasicInvocation$InvocationFuture.get(BasicInvocation.java:676) > at > com.hazelcast.map.proxy.MapProxySupport.containsKeyInternal(MapProxySupport.java:369) > ... 22 more > > > Regards, > NuwanS. > -- > > > *Nuwan Silva* > *Senior Software Engineer - QA* > Mobile: +9477 980 4543 > > WSO2 Inc. > lean . enterprise . middlewear. > http://www.wso2.com > > _______________________________________________ > Dev mailing list > [email protected] > http://wso2.org/cgi-bin/mailman/listinfo/dev > > -- *Asanthi Kulasinghe* WSO2 Inc; http://www.wso2.com/. Mobile: +94777355522
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
