Hi Azeez, This exception does not get logged in the coordinator node, only in all other nodes. So this error happens when they are blocked to get the distributed lock. Yeah we can implement your suggestion for the unlock scenario, but for the lock scenario we still don't have a solution other than ignoring the exception.
Thanks, Sameera. On Thu, Oct 16, 2014 at 9:19 AM, Afkham Azeez <[email protected]> wrote: > There is a method to check in CarbonKernel if the server is shutting down. > It sets the SHUTTING_DOWN status or something like that. So what we could > do is this; > > * try {* > * lock.unlock();* > * } catch (HazelcastInstanceNotActiveException e) {* > * if(!server.getStatus().equal(SHUTTING_DOWN){* > * log.error("Could not release Hazelcast > coordinator lock", e);* > * }* > * // Ignore this exception if the server is > shutting down <https://wso2.org/jira/browse/CARBON-14887>* > * }* > > On Thu, Oct 16, 2014 at 1:04 AM, Sameera Jayasoma <[email protected]> > wrote: > >> How about ignoring this exception for the moment. This is an harmless >> error log it seems. Until we find a proper way to fix this we can ignore >> this exception. This exception occurs during the server shutdown and it >> does not do any harm. >> >> ILock lock = >> primaryHazelcastInstance.getLock(HazelcastConstants.CLUSTER_COORDINATOR_LOCK); >> try { >> lock.lock(); >> isCoordinator = true; >> log.info("Elected this member [" + >> primaryHazelcastInstance.getCluster().getLocalMember().getUuid() + "] " + >> "as the Coordinator for the cluster [" + >> carbonLocalMember.getDomain() + "]"); >> >> // Notify all OSGi services which are waiting for >> this member to become the coordinator >> List<CoordinatedActivity> coordinatedActivities = >> CarbonCoreDataHolder.getInstance().getCoordinatedActivities(); >> for (CoordinatedActivity coordinatedActivity : >> coordinatedActivities) { >> coordinatedActivity.execute(); >> } >> *} catch (HazelcastInstanceNotActiveException ignored) {* >> * // Ignoring this exception as a workaround to >> https://wso2.org/jira/browse/CARBON-14887 >> <https://wso2.org/jira/browse/CARBON-14887>* >> * } finally {* >> * try {* >> * lock.unlock();* >> * } catch (HazelcastInstanceNotActiveException e) {* >> * // Ignoring this exception as a workaround to >> https://wso2.org/jira/browse/CARBON-14887 >> <https://wso2.org/jira/browse/CARBON-14887>* >> * }* >> * }* >> >> On Tue, Oct 14, 2014 at 11:07 PM, Aruna Karunarathna <[email protected]> >> wrote: >> >>> Hi Azeez, >>> >>> We are currently doing testing in carbon 4.3.0 with the latest hazelcast >>> 3.2.6 upgrade. And when shutting down a server using (ctrl +c ) following >>> error occurred. >>> This was reported earlier also here [1] in carbon 4.2.0. >>> >>> Exception in thread "Cluster [wso2.as.domain] coordinator elector >>> thread" com.hazelcast.core.HazelcastInstanceNotActiveException: Hazelcast >>> instance is not active! >>> at >>> com.hazelcast.spi.impl.BasicOperationService.shutdown(BasicOperationService.java:749) >>> at >>> com.hazelcast.spi.impl.NodeEngineImpl.shutdown(NodeEngineImpl.java:347) >>> at com.hazelcast.instance.Node.shutdown(Node.java:420) >>> at com.hazelcast.instance.Node$NodeShutdownHookThread.run(Node.java:507) >>> at ------ End remote and begin local stack-trace ------.(Unknown Source) >>> at >>> com.hazelcast.spi.impl.BasicInvocation$InvocationFuture.resolveResponse(BasicInvocation.java:869) >>> at >>> com.hazelcast.spi.impl.BasicInvocation$InvocationFuture.resolveResponseOrThrowException(BasicInvocation.java:802) >>> at >>> com.hazelcast.spi.impl.BasicInvocation$InvocationFuture.get(BasicInvocation.java:698) >>> at >>> com.hazelcast.spi.impl.BasicInvocation$InvocationFuture.get(BasicInvocation.java:676) >>> at >>> com.hazelcast.spi.impl.BasicInvocation$InvocationFuture.getSafely(BasicInvocation.java:689) >>> at >>> com.hazelcast.concurrent.lock.LockProxySupport.lock(LockProxySupport.java:80) >>> at >>> com.hazelcast.concurrent.lock.LockProxySupport.lock(LockProxySupport.java:74) >>> at com.hazelcast.concurrent.lock.LockProxy.lock(LockProxy.java:70) >>> at >>> org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent$2.run(HazelcastClusteringAgent.java:228) >>> >>> Any idea on how to overcome this error?. I looked at the >>> HazelcastClusteringAgent code and AFAIU coordinatorElectorThread is waiting >>> for acquire the lock and the >>> hazelcast instance is not valid at this point since the shutdown is >>> triggered. >>> >>> I tried using the *hazelcast.shutdownhook.enabled=false* property and >>> it ended up with the following error. >>> >>> Exception in thread "Cluster [wso2.as.domain] coordinator elector >>> thread" com.hazelcast.core.HazelcastInstanceNotActiveException: Hazelcast >>> instance is not active! >>> at >>> com.hazelcast.spi.impl.BasicOperationService.shutdown(BasicOperationService.java:749) >>> at >>> com.hazelcast.spi.impl.NodeEngineImpl.shutdown(NodeEngineImpl.java:347) >>> at com.hazelcast.instance.Node.shutdown(Node.java:420) >>> at >>> com.hazelcast.instance.LifecycleServiceImpl.shutdown(LifecycleServiceImpl.java:79) >>> at >>> com.hazelcast.instance.HazelcastInstanceFactory.shutdownAll(HazelcastInstanceFactory.java:213) >>> at com.hazelcast.core.Hazelcast.shutdownAll(Hazelcast.java:41) >>> at >>> org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent.stop(HazelcastClusteringAgent.java:347) >>> at >>> org.apache.axis2.engine.AxisConfiguration.cleanup(AxisConfiguration.java:1462) >>> at >>> org.apache.axis2.context.ConfigurationContext.terminate(ConfigurationContext.java:788) >>> at >>> org.apache.axis2.transport.http.AxisServlet.destroy(AxisServlet.java:528) >>> at >>> org.eclipse.equinox.http.servlet.internal.ServletRegistration.destroy(ServletRegistration.java:38) >>> at >>> org.eclipse.equinox.http.servlet.internal.ProxyServlet.unregister(ProxyServlet.java:153) >>> at >>> org.eclipse.equinox.http.servlet.internal.HttpServiceImpl.unregister(HttpServiceImpl.java:88) >>> at >>> org.wso2.carbon.core.init.CarbonServerManager.stop(CarbonServerManager.java:932) >>> at >>> org.wso2.carbon.core.internal.CarbonCoreServiceComponent.deactivate(CarbonCoreServiceComponent.java:102) >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>> at >>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >>> at >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>> at java.lang.reflect.Method.invoke(Method.java:606) >>> at >>> org.eclipse.equinox.internal.ds.model.ServiceComponent.deactivate(ServiceComponent.java:387) >>> at >>> org.eclipse.equinox.internal.ds.model.ServiceComponentProp.deactivate(ServiceComponentProp.java:161) >>> at >>> org.eclipse.equinox.internal.ds.model.ServiceComponentProp.dispose(ServiceComponentProp.java:389) >>> at >>> org.eclipse.equinox.internal.ds.model.ServiceComponentProp.dispose(ServiceComponentProp.java:102) >>> at >>> org.eclipse.equinox.internal.ds.InstanceProcess.disposeInstances(InstanceProcess.java:344) >>> at >>> org.eclipse.equinox.internal.ds.InstanceProcess.disposeInstances(InstanceProcess.java:306) >>> at >>> org.eclipse.equinox.internal.ds.Resolver.getEligible(Resolver.java:368) >>> at >>> org.eclipse.equinox.internal.ds.SCRManager.serviceChanged(SCRManager.java:222) >>> at >>> org.eclipse.osgi.internal.serviceregistry.FilteredServiceListener.serviceChanged(FilteredServiceListener.java:107) >>> at >>> org.eclipse.osgi.framework.internal.core.BundleContextImpl.dispatchEvent(BundleContextImpl.java:861) >>> at >>> org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:230) >>> at >>> org.eclipse.osgi.framework.eventmgr.ListenerQueue.dispatchEventSynchronous(ListenerQueue.java:148) >>> at >>> org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEventPrivileged(ServiceRegistry.java:819) >>> at >>> org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEvent(ServiceRegistry.java:771) >>> at >>> org.eclipse.osgi.internal.serviceregistry.ServiceRegistrationImpl.unregister(ServiceRegistrationImpl.java:225) >>> at >>> org.eclipse.equinox.http.servlet.internal.Activator.removeProxyServlet(Activator.java:87) >>> at >>> org.eclipse.equinox.http.servlet.internal.ProxyServlet.destroy(ProxyServlet.java:44) >>> at >>> org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.destroy(DelegationServlet.java:43) >>> at >>> org.apache.catalina.core.StandardWrapper.unload(StandardWrapper.java:1481) >>> at >>> org.apache.catalina.core.StandardWrapper.stopInternal(StandardWrapper.java:1842) >>> at org.apache.catalina.util.LifecycleBase.stop(LifecycleBase.java:232) >>> at >>> org.apache.catalina.core.StandardContext.stopInternal(StandardContext.java:5647) >>> at org.apache.catalina.util.LifecycleBase.stop(LifecycleBase.java:232) >>> at >>> org.apache.catalina.core.ContainerBase$StopChild.call(ContainerBase.java:1575) >>> at >>> org.apache.catalina.core.ContainerBase$StopChild.call(ContainerBase.java:1564) >>> at java.util.concurrent.FutureTask.run(FutureTask.java:262) >>> at >>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >>> at java.lang.Thread.run(Thread.java:745) >>> at ------ End remote and begin local stack-trace ------.(Unknown Source) >>> at >>> com.hazelcast.spi.impl.BasicInvocation$InvocationFuture.resolveResponse(BasicInvocation.java:869) >>> at >>> com.hazelcast.spi.impl.BasicInvocation$InvocationFuture.resolveResponseOrThrowException(BasicInvocation.java:802) >>> at >>> com.hazelcast.spi.impl.BasicInvocation$InvocationFuture.get(BasicInvocation.java:698) >>> at >>> com.hazelcast.spi.impl.BasicInvocation$InvocationFuture.get(BasicInvocation.java:676) >>> at >>> com.hazelcast.spi.impl.BasicInvocation$InvocationFuture.getSafely(BasicInvocation.java:689) >>> at >>> com.hazelcast.concurrent.lock.LockProxySupport.lock(LockProxySupport.java:80) >>> at >>> com.hazelcast.concurrent.lock.LockProxySupport.lock(LockProxySupport.java:74) >>> at com.hazelcast.concurrent.lock.LockProxy.lock(LockProxy.java:70) >>> at >>> org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent$2.run(HazelcastClusteringAgent.java:228) >>> >>> >>> [1]. https://wso2.org/jira/browse/CARBON-14887 >>> >>> >>> Regards, >>> Aruna >>> -- >>> >>> *Aruna Sujith Karunarathna* | Software Engineer >>> WSO2, Inc | lean. enterprise. middleware. >>> #20, Palm Grove, Colombo 03, Sri Lanka >>> Mobile: +94 71 9040362 | Work: +94 112145345 >>> Email: [email protected] | Web: www.wso2.com >>> >>> >> >> >> >> -- >> Sameera Jayasoma, >> Software Architect, >> >> WSO2, Inc. (http://wso2.com) >> email: [email protected] >> blog: http://sameera.adahas.org >> twitter: https://twitter.com/sameerajayasoma >> flickr: http://www.flickr.com/photos/sameera-jayasoma/collections >> Mobile: 0094776364456 >> >> Lean . Enterprise . Middleware >> >> > > > -- > *Afkham Azeez* > Director of Architecture; WSO2, Inc.; http://wso2.com > Member; Apache Software Foundation; http://www.apache.org/ > * <http://www.apache.org/>* > *email: **[email protected]* <[email protected]> > * cell: +94 77 3320919 <%2B94%2077%203320919>blog: * > *http://blog.afkham.org* <http://blog.afkham.org> > *twitter: **http://twitter.com/afkham_azeez* > <http://twitter.com/afkham_azeez> > *linked-in: **http://lk.linkedin.com/in/afkhamazeez > <http://lk.linkedin.com/in/afkhamazeez>* > > *Lean . Enterprise . Middleware* > -- Sameera Jayasoma, Software Architect, WSO2, Inc. (http://wso2.com) email: [email protected] blog: http://sameera.adahas.org twitter: https://twitter.com/sameerajayasoma flickr: http://www.flickr.com/photos/sameera-jayasoma/collections Mobile: 0094776364456 Lean . Enterprise . Middleware
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
