Can do the same thing, and catch the exception on lock as well, and ignore if the status is SHUTTING_DOWN
On Thu, Oct 16, 2014 at 11:06 AM, Sameera Jayasoma <[email protected]> wrote: > Hi Azeez, > > This exception does not get logged in the coordinator node, only in all > other nodes. So this error happens when they are blocked to get the > distributed lock. Yeah we can implement your suggestion for the unlock > scenario, but for the lock scenario we still don't have a solution other > than ignoring the exception. > > Thanks, > Sameera. > > On Thu, Oct 16, 2014 at 9:19 AM, Afkham Azeez <[email protected]> wrote: > >> There is a method to check in CarbonKernel if the server is shutting >> down. It sets the SHUTTING_DOWN status or something like that. So what we >> could do is this; >> >> * try {* >> * lock.unlock();* >> * } catch (HazelcastInstanceNotActiveException e) {* >> * if(!server.getStatus().equal(SHUTTING_DOWN){* >> * log.error("Could not release Hazelcast >> coordinator lock", e);* >> * }* >> * // Ignore this exception if the server is >> shutting down <https://wso2.org/jira/browse/CARBON-14887>* >> * }* >> >> On Thu, Oct 16, 2014 at 1:04 AM, Sameera Jayasoma <[email protected]> >> wrote: >> >>> How about ignoring this exception for the moment. This is an harmless >>> error log it seems. Until we find a proper way to fix this we can ignore >>> this exception. This exception occurs during the server shutdown and it >>> does not do any harm. >>> >>> ILock lock = >>> primaryHazelcastInstance.getLock(HazelcastConstants.CLUSTER_COORDINATOR_LOCK); >>> try { >>> lock.lock(); >>> isCoordinator = true; >>> log.info("Elected this member [" + >>> primaryHazelcastInstance.getCluster().getLocalMember().getUuid() + "] " + >>> "as the Coordinator for the cluster [" + >>> carbonLocalMember.getDomain() + "]"); >>> >>> // Notify all OSGi services which are waiting for >>> this member to become the coordinator >>> List<CoordinatedActivity> coordinatedActivities = >>> CarbonCoreDataHolder.getInstance().getCoordinatedActivities(); >>> for (CoordinatedActivity coordinatedActivity : >>> coordinatedActivities) { >>> coordinatedActivity.execute(); >>> } >>> *} catch (HazelcastInstanceNotActiveException ignored) >>> {* >>> * // Ignoring this exception as a workaround to >>> https://wso2.org/jira/browse/CARBON-14887 >>> <https://wso2.org/jira/browse/CARBON-14887>* >>> * } finally {* >>> * try {* >>> * lock.unlock();* >>> * } catch (HazelcastInstanceNotActiveException e) {* >>> * // Ignoring this exception as a workaround to >>> https://wso2.org/jira/browse/CARBON-14887 >>> <https://wso2.org/jira/browse/CARBON-14887>* >>> * }* >>> * }* >>> >>> On Tue, Oct 14, 2014 at 11:07 PM, Aruna Karunarathna <[email protected]> >>> wrote: >>> >>>> Hi Azeez, >>>> >>>> We are currently doing testing in carbon 4.3.0 with the latest >>>> hazelcast 3.2.6 upgrade. And when shutting down a server using (ctrl +c ) >>>> following error occurred. >>>> This was reported earlier also here [1] in carbon 4.2.0. >>>> >>>> Exception in thread "Cluster [wso2.as.domain] coordinator elector >>>> thread" com.hazelcast.core.HazelcastInstanceNotActiveException: Hazelcast >>>> instance is not active! >>>> at >>>> com.hazelcast.spi.impl.BasicOperationService.shutdown(BasicOperationService.java:749) >>>> at >>>> com.hazelcast.spi.impl.NodeEngineImpl.shutdown(NodeEngineImpl.java:347) >>>> at com.hazelcast.instance.Node.shutdown(Node.java:420) >>>> at com.hazelcast.instance.Node$NodeShutdownHookThread.run(Node.java:507) >>>> at ------ End remote and begin local stack-trace ------.(Unknown Source) >>>> at >>>> com.hazelcast.spi.impl.BasicInvocation$InvocationFuture.resolveResponse(BasicInvocation.java:869) >>>> at >>>> com.hazelcast.spi.impl.BasicInvocation$InvocationFuture.resolveResponseOrThrowException(BasicInvocation.java:802) >>>> at >>>> com.hazelcast.spi.impl.BasicInvocation$InvocationFuture.get(BasicInvocation.java:698) >>>> at >>>> com.hazelcast.spi.impl.BasicInvocation$InvocationFuture.get(BasicInvocation.java:676) >>>> at >>>> com.hazelcast.spi.impl.BasicInvocation$InvocationFuture.getSafely(BasicInvocation.java:689) >>>> at >>>> com.hazelcast.concurrent.lock.LockProxySupport.lock(LockProxySupport.java:80) >>>> at >>>> com.hazelcast.concurrent.lock.LockProxySupport.lock(LockProxySupport.java:74) >>>> at com.hazelcast.concurrent.lock.LockProxy.lock(LockProxy.java:70) >>>> at >>>> org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent$2.run(HazelcastClusteringAgent.java:228) >>>> >>>> Any idea on how to overcome this error?. I looked at the >>>> HazelcastClusteringAgent code and AFAIU coordinatorElectorThread is waiting >>>> for acquire the lock and the >>>> hazelcast instance is not valid at this point since the shutdown is >>>> triggered. >>>> >>>> I tried using the *hazelcast.shutdownhook.enabled=false* property and >>>> it ended up with the following error. >>>> >>>> Exception in thread "Cluster [wso2.as.domain] coordinator elector >>>> thread" com.hazelcast.core.HazelcastInstanceNotActiveException: Hazelcast >>>> instance is not active! >>>> at >>>> com.hazelcast.spi.impl.BasicOperationService.shutdown(BasicOperationService.java:749) >>>> at >>>> com.hazelcast.spi.impl.NodeEngineImpl.shutdown(NodeEngineImpl.java:347) >>>> at com.hazelcast.instance.Node.shutdown(Node.java:420) >>>> at >>>> com.hazelcast.instance.LifecycleServiceImpl.shutdown(LifecycleServiceImpl.java:79) >>>> at >>>> com.hazelcast.instance.HazelcastInstanceFactory.shutdownAll(HazelcastInstanceFactory.java:213) >>>> at com.hazelcast.core.Hazelcast.shutdownAll(Hazelcast.java:41) >>>> at >>>> org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent.stop(HazelcastClusteringAgent.java:347) >>>> at >>>> org.apache.axis2.engine.AxisConfiguration.cleanup(AxisConfiguration.java:1462) >>>> at >>>> org.apache.axis2.context.ConfigurationContext.terminate(ConfigurationContext.java:788) >>>> at >>>> org.apache.axis2.transport.http.AxisServlet.destroy(AxisServlet.java:528) >>>> at >>>> org.eclipse.equinox.http.servlet.internal.ServletRegistration.destroy(ServletRegistration.java:38) >>>> at >>>> org.eclipse.equinox.http.servlet.internal.ProxyServlet.unregister(ProxyServlet.java:153) >>>> at >>>> org.eclipse.equinox.http.servlet.internal.HttpServiceImpl.unregister(HttpServiceImpl.java:88) >>>> at >>>> org.wso2.carbon.core.init.CarbonServerManager.stop(CarbonServerManager.java:932) >>>> at >>>> org.wso2.carbon.core.internal.CarbonCoreServiceComponent.deactivate(CarbonCoreServiceComponent.java:102) >>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>>> at >>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >>>> at >>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>>> at java.lang.reflect.Method.invoke(Method.java:606) >>>> at >>>> org.eclipse.equinox.internal.ds.model.ServiceComponent.deactivate(ServiceComponent.java:387) >>>> at >>>> org.eclipse.equinox.internal.ds.model.ServiceComponentProp.deactivate(ServiceComponentProp.java:161) >>>> at >>>> org.eclipse.equinox.internal.ds.model.ServiceComponentProp.dispose(ServiceComponentProp.java:389) >>>> at >>>> org.eclipse.equinox.internal.ds.model.ServiceComponentProp.dispose(ServiceComponentProp.java:102) >>>> at >>>> org.eclipse.equinox.internal.ds.InstanceProcess.disposeInstances(InstanceProcess.java:344) >>>> at >>>> org.eclipse.equinox.internal.ds.InstanceProcess.disposeInstances(InstanceProcess.java:306) >>>> at >>>> org.eclipse.equinox.internal.ds.Resolver.getEligible(Resolver.java:368) >>>> at >>>> org.eclipse.equinox.internal.ds.SCRManager.serviceChanged(SCRManager.java:222) >>>> at >>>> org.eclipse.osgi.internal.serviceregistry.FilteredServiceListener.serviceChanged(FilteredServiceListener.java:107) >>>> at >>>> org.eclipse.osgi.framework.internal.core.BundleContextImpl.dispatchEvent(BundleContextImpl.java:861) >>>> at >>>> org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:230) >>>> at >>>> org.eclipse.osgi.framework.eventmgr.ListenerQueue.dispatchEventSynchronous(ListenerQueue.java:148) >>>> at >>>> org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEventPrivileged(ServiceRegistry.java:819) >>>> at >>>> org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEvent(ServiceRegistry.java:771) >>>> at >>>> org.eclipse.osgi.internal.serviceregistry.ServiceRegistrationImpl.unregister(ServiceRegistrationImpl.java:225) >>>> at >>>> org.eclipse.equinox.http.servlet.internal.Activator.removeProxyServlet(Activator.java:87) >>>> at >>>> org.eclipse.equinox.http.servlet.internal.ProxyServlet.destroy(ProxyServlet.java:44) >>>> at >>>> org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.destroy(DelegationServlet.java:43) >>>> at >>>> org.apache.catalina.core.StandardWrapper.unload(StandardWrapper.java:1481) >>>> at >>>> org.apache.catalina.core.StandardWrapper.stopInternal(StandardWrapper.java:1842) >>>> at org.apache.catalina.util.LifecycleBase.stop(LifecycleBase.java:232) >>>> at >>>> org.apache.catalina.core.StandardContext.stopInternal(StandardContext.java:5647) >>>> at org.apache.catalina.util.LifecycleBase.stop(LifecycleBase.java:232) >>>> at >>>> org.apache.catalina.core.ContainerBase$StopChild.call(ContainerBase.java:1575) >>>> at >>>> org.apache.catalina.core.ContainerBase$StopChild.call(ContainerBase.java:1564) >>>> at java.util.concurrent.FutureTask.run(FutureTask.java:262) >>>> at >>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >>>> at >>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >>>> at java.lang.Thread.run(Thread.java:745) >>>> at ------ End remote and begin local stack-trace ------.(Unknown Source) >>>> at >>>> com.hazelcast.spi.impl.BasicInvocation$InvocationFuture.resolveResponse(BasicInvocation.java:869) >>>> at >>>> com.hazelcast.spi.impl.BasicInvocation$InvocationFuture.resolveResponseOrThrowException(BasicInvocation.java:802) >>>> at >>>> com.hazelcast.spi.impl.BasicInvocation$InvocationFuture.get(BasicInvocation.java:698) >>>> at >>>> com.hazelcast.spi.impl.BasicInvocation$InvocationFuture.get(BasicInvocation.java:676) >>>> at >>>> com.hazelcast.spi.impl.BasicInvocation$InvocationFuture.getSafely(BasicInvocation.java:689) >>>> at >>>> com.hazelcast.concurrent.lock.LockProxySupport.lock(LockProxySupport.java:80) >>>> at >>>> com.hazelcast.concurrent.lock.LockProxySupport.lock(LockProxySupport.java:74) >>>> at com.hazelcast.concurrent.lock.LockProxy.lock(LockProxy.java:70) >>>> at >>>> org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent$2.run(HazelcastClusteringAgent.java:228) >>>> >>>> >>>> [1]. https://wso2.org/jira/browse/CARBON-14887 >>>> >>>> >>>> Regards, >>>> Aruna >>>> -- >>>> >>>> *Aruna Sujith Karunarathna* | Software Engineer >>>> WSO2, Inc | lean. enterprise. middleware. >>>> #20, Palm Grove, Colombo 03, Sri Lanka >>>> Mobile: +94 71 9040362 | Work: +94 112145345 >>>> Email: [email protected] | Web: www.wso2.com >>>> >>>> >>> >>> >>> >>> -- >>> Sameera Jayasoma, >>> Software Architect, >>> >>> WSO2, Inc. (http://wso2.com) >>> email: [email protected] >>> blog: http://sameera.adahas.org >>> twitter: https://twitter.com/sameerajayasoma >>> flickr: http://www.flickr.com/photos/sameera-jayasoma/collections >>> Mobile: 0094776364456 >>> >>> Lean . Enterprise . Middleware >>> >>> >> >> >> -- >> *Afkham Azeez* >> Director of Architecture; WSO2, Inc.; http://wso2.com >> Member; Apache Software Foundation; http://www.apache.org/ >> * <http://www.apache.org/>* >> *email: **[email protected]* <[email protected]> >> * cell: +94 77 3320919 <%2B94%2077%203320919>blog: * >> *http://blog.afkham.org* <http://blog.afkham.org> >> *twitter: **http://twitter.com/afkham_azeez* >> <http://twitter.com/afkham_azeez> >> *linked-in: **http://lk.linkedin.com/in/afkhamazeez >> <http://lk.linkedin.com/in/afkhamazeez>* >> >> *Lean . Enterprise . Middleware* >> > > > > -- > Sameera Jayasoma, > Software Architect, > > WSO2, Inc. (http://wso2.com) > email: [email protected] > blog: http://sameera.adahas.org > twitter: https://twitter.com/sameerajayasoma > flickr: http://www.flickr.com/photos/sameera-jayasoma/collections > Mobile: 0094776364456 > > Lean . Enterprise . Middleware > > -- *Afkham Azeez* Director of Architecture; WSO2, Inc.; http://wso2.com Member; Apache Software Foundation; http://www.apache.org/ * <http://www.apache.org/>* *email: **[email protected]* <[email protected]> * cell: +94 77 3320919blog: **http://blog.afkham.org* <http://blog.afkham.org> *twitter: **http://twitter.com/afkham_azeez* <http://twitter.com/afkham_azeez> *linked-in: **http://lk.linkedin.com/in/afkhamazeez <http://lk.linkedin.com/in/afkhamazeez>* *Lean . Enterprise . Middleware*
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
