+1
try {
lock.unlock();
} catch (HazelcastInstanceNotActiveException e) {
if(!ServerStatus.STATUS_SHUTTING_DOWN.equals(ServerStatus.getCurrentStatus())){
log.error("Could not release Hazelcast
coordinator lock", e);
}
// Ignoring this exception if the server is
shutting down.
}
On Thu, Oct 16, 2014 at 11:08 AM, Afkham Azeez <[email protected]> wrote:
> Can do the same thing, and catch the exception on lock as well, and ignore
> if the status is SHUTTING_DOWN
>
>
> On Thu, Oct 16, 2014 at 11:06 AM, Sameera Jayasoma <[email protected]>
> wrote:
>
>> Hi Azeez,
>>
>> This exception does not get logged in the coordinator node, only in all
>> other nodes. So this error happens when they are blocked to get the
>> distributed lock. Yeah we can implement your suggestion for the unlock
>> scenario, but for the lock scenario we still don't have a solution other
>> than ignoring the exception.
>>
>> Thanks,
>> Sameera.
>>
>> On Thu, Oct 16, 2014 at 9:19 AM, Afkham Azeez <[email protected]> wrote:
>>
>>> There is a method to check in CarbonKernel if the server is shutting
>>> down. It sets the SHUTTING_DOWN status or something like that. So what we
>>> could do is this;
>>>
>>> * try {*
>>> * lock.unlock();*
>>> * } catch (HazelcastInstanceNotActiveException e) {*
>>> * if(!server.getStatus().equal(SHUTTING_DOWN){*
>>> * log.error("Could not release Hazelcast
>>> coordinator lock", e);*
>>> * }*
>>> * // Ignore this exception if the server is
>>> shutting down <https://wso2.org/jira/browse/CARBON-14887>*
>>> * }*
>>>
>>> On Thu, Oct 16, 2014 at 1:04 AM, Sameera Jayasoma <[email protected]>
>>> wrote:
>>>
>>>> How about ignoring this exception for the moment. This is an harmless
>>>> error log it seems. Until we find a proper way to fix this we can ignore
>>>> this exception. This exception occurs during the server shutdown and it
>>>> does not do any harm.
>>>>
>>>> ILock lock =
>>>> primaryHazelcastInstance.getLock(HazelcastConstants.CLUSTER_COORDINATOR_LOCK);
>>>> try {
>>>> lock.lock();
>>>> isCoordinator = true;
>>>> log.info("Elected this member [" +
>>>> primaryHazelcastInstance.getCluster().getLocalMember().getUuid() + "] " +
>>>> "as the Coordinator for the cluster [" +
>>>> carbonLocalMember.getDomain() + "]");
>>>>
>>>> // Notify all OSGi services which are waiting for
>>>> this member to become the coordinator
>>>> List<CoordinatedActivity> coordinatedActivities =
>>>> CarbonCoreDataHolder.getInstance().getCoordinatedActivities();
>>>> for (CoordinatedActivity coordinatedActivity :
>>>> coordinatedActivities) {
>>>> coordinatedActivity.execute();
>>>> }
>>>> *} catch (HazelcastInstanceNotActiveException ignored)
>>>> {*
>>>> * // Ignoring this exception as a workaround to
>>>> https://wso2.org/jira/browse/CARBON-14887
>>>> <https://wso2.org/jira/browse/CARBON-14887>*
>>>> * } finally {*
>>>> * try {*
>>>> * lock.unlock();*
>>>> * } catch (HazelcastInstanceNotActiveException e) {*
>>>> * // Ignoring this exception as a workaround to
>>>> https://wso2.org/jira/browse/CARBON-14887
>>>> <https://wso2.org/jira/browse/CARBON-14887>*
>>>> * }*
>>>> * }*
>>>>
>>>> On Tue, Oct 14, 2014 at 11:07 PM, Aruna Karunarathna <[email protected]>
>>>> wrote:
>>>>
>>>>> Hi Azeez,
>>>>>
>>>>> We are currently doing testing in carbon 4.3.0 with the latest
>>>>> hazelcast 3.2.6 upgrade. And when shutting down a server using (ctrl +c )
>>>>> following error occurred.
>>>>> This was reported earlier also here [1] in carbon 4.2.0.
>>>>>
>>>>> Exception in thread "Cluster [wso2.as.domain] coordinator elector
>>>>> thread" com.hazelcast.core.HazelcastInstanceNotActiveException: Hazelcast
>>>>> instance is not active!
>>>>> at
>>>>> com.hazelcast.spi.impl.BasicOperationService.shutdown(BasicOperationService.java:749)
>>>>> at
>>>>> com.hazelcast.spi.impl.NodeEngineImpl.shutdown(NodeEngineImpl.java:347)
>>>>> at com.hazelcast.instance.Node.shutdown(Node.java:420)
>>>>> at
>>>>> com.hazelcast.instance.Node$NodeShutdownHookThread.run(Node.java:507)
>>>>> at ------ End remote and begin local stack-trace ------.(Unknown
>>>>> Source)
>>>>> at
>>>>> com.hazelcast.spi.impl.BasicInvocation$InvocationFuture.resolveResponse(BasicInvocation.java:869)
>>>>> at
>>>>> com.hazelcast.spi.impl.BasicInvocation$InvocationFuture.resolveResponseOrThrowException(BasicInvocation.java:802)
>>>>> at
>>>>> com.hazelcast.spi.impl.BasicInvocation$InvocationFuture.get(BasicInvocation.java:698)
>>>>> at
>>>>> com.hazelcast.spi.impl.BasicInvocation$InvocationFuture.get(BasicInvocation.java:676)
>>>>> at
>>>>> com.hazelcast.spi.impl.BasicInvocation$InvocationFuture.getSafely(BasicInvocation.java:689)
>>>>> at
>>>>> com.hazelcast.concurrent.lock.LockProxySupport.lock(LockProxySupport.java:80)
>>>>> at
>>>>> com.hazelcast.concurrent.lock.LockProxySupport.lock(LockProxySupport.java:74)
>>>>> at com.hazelcast.concurrent.lock.LockProxy.lock(LockProxy.java:70)
>>>>> at
>>>>> org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent$2.run(HazelcastClusteringAgent.java:228)
>>>>>
>>>>> Any idea on how to overcome this error?. I looked at the
>>>>> HazelcastClusteringAgent code and AFAIU coordinatorElectorThread is
>>>>> waiting
>>>>> for acquire the lock and the
>>>>> hazelcast instance is not valid at this point since the shutdown is
>>>>> triggered.
>>>>>
>>>>> I tried using the *hazelcast.shutdownhook.enabled=false* property
>>>>> and it ended up with the following error.
>>>>>
>>>>> Exception in thread "Cluster [wso2.as.domain] coordinator elector
>>>>> thread" com.hazelcast.core.HazelcastInstanceNotActiveException: Hazelcast
>>>>> instance is not active!
>>>>> at
>>>>> com.hazelcast.spi.impl.BasicOperationService.shutdown(BasicOperationService.java:749)
>>>>> at
>>>>> com.hazelcast.spi.impl.NodeEngineImpl.shutdown(NodeEngineImpl.java:347)
>>>>> at com.hazelcast.instance.Node.shutdown(Node.java:420)
>>>>> at
>>>>> com.hazelcast.instance.LifecycleServiceImpl.shutdown(LifecycleServiceImpl.java:79)
>>>>> at
>>>>> com.hazelcast.instance.HazelcastInstanceFactory.shutdownAll(HazelcastInstanceFactory.java:213)
>>>>> at com.hazelcast.core.Hazelcast.shutdownAll(Hazelcast.java:41)
>>>>> at
>>>>> org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent.stop(HazelcastClusteringAgent.java:347)
>>>>> at
>>>>> org.apache.axis2.engine.AxisConfiguration.cleanup(AxisConfiguration.java:1462)
>>>>> at
>>>>> org.apache.axis2.context.ConfigurationContext.terminate(ConfigurationContext.java:788)
>>>>> at
>>>>> org.apache.axis2.transport.http.AxisServlet.destroy(AxisServlet.java:528)
>>>>> at
>>>>> org.eclipse.equinox.http.servlet.internal.ServletRegistration.destroy(ServletRegistration.java:38)
>>>>> at
>>>>> org.eclipse.equinox.http.servlet.internal.ProxyServlet.unregister(ProxyServlet.java:153)
>>>>> at
>>>>> org.eclipse.equinox.http.servlet.internal.HttpServiceImpl.unregister(HttpServiceImpl.java:88)
>>>>> at
>>>>> org.wso2.carbon.core.init.CarbonServerManager.stop(CarbonServerManager.java:932)
>>>>> at
>>>>> org.wso2.carbon.core.internal.CarbonCoreServiceComponent.deactivate(CarbonCoreServiceComponent.java:102)
>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>> at
>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>>>> at
>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>>> at java.lang.reflect.Method.invoke(Method.java:606)
>>>>> at
>>>>> org.eclipse.equinox.internal.ds.model.ServiceComponent.deactivate(ServiceComponent.java:387)
>>>>> at
>>>>> org.eclipse.equinox.internal.ds.model.ServiceComponentProp.deactivate(ServiceComponentProp.java:161)
>>>>> at
>>>>> org.eclipse.equinox.internal.ds.model.ServiceComponentProp.dispose(ServiceComponentProp.java:389)
>>>>> at
>>>>> org.eclipse.equinox.internal.ds.model.ServiceComponentProp.dispose(ServiceComponentProp.java:102)
>>>>> at
>>>>> org.eclipse.equinox.internal.ds.InstanceProcess.disposeInstances(InstanceProcess.java:344)
>>>>> at
>>>>> org.eclipse.equinox.internal.ds.InstanceProcess.disposeInstances(InstanceProcess.java:306)
>>>>> at
>>>>> org.eclipse.equinox.internal.ds.Resolver.getEligible(Resolver.java:368)
>>>>> at
>>>>> org.eclipse.equinox.internal.ds.SCRManager.serviceChanged(SCRManager.java:222)
>>>>> at
>>>>> org.eclipse.osgi.internal.serviceregistry.FilteredServiceListener.serviceChanged(FilteredServiceListener.java:107)
>>>>> at
>>>>> org.eclipse.osgi.framework.internal.core.BundleContextImpl.dispatchEvent(BundleContextImpl.java:861)
>>>>> at
>>>>> org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:230)
>>>>> at
>>>>> org.eclipse.osgi.framework.eventmgr.ListenerQueue.dispatchEventSynchronous(ListenerQueue.java:148)
>>>>> at
>>>>> org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEventPrivileged(ServiceRegistry.java:819)
>>>>> at
>>>>> org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEvent(ServiceRegistry.java:771)
>>>>> at
>>>>> org.eclipse.osgi.internal.serviceregistry.ServiceRegistrationImpl.unregister(ServiceRegistrationImpl.java:225)
>>>>> at
>>>>> org.eclipse.equinox.http.servlet.internal.Activator.removeProxyServlet(Activator.java:87)
>>>>> at
>>>>> org.eclipse.equinox.http.servlet.internal.ProxyServlet.destroy(ProxyServlet.java:44)
>>>>> at
>>>>> org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.destroy(DelegationServlet.java:43)
>>>>> at
>>>>> org.apache.catalina.core.StandardWrapper.unload(StandardWrapper.java:1481)
>>>>> at
>>>>> org.apache.catalina.core.StandardWrapper.stopInternal(StandardWrapper.java:1842)
>>>>> at org.apache.catalina.util.LifecycleBase.stop(LifecycleBase.java:232)
>>>>> at
>>>>> org.apache.catalina.core.StandardContext.stopInternal(StandardContext.java:5647)
>>>>> at org.apache.catalina.util.LifecycleBase.stop(LifecycleBase.java:232)
>>>>> at
>>>>> org.apache.catalina.core.ContainerBase$StopChild.call(ContainerBase.java:1575)
>>>>> at
>>>>> org.apache.catalina.core.ContainerBase$StopChild.call(ContainerBase.java:1564)
>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>>>>> at
>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>>>> at
>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>>>> at java.lang.Thread.run(Thread.java:745)
>>>>> at ------ End remote and begin local stack-trace ------.(Unknown
>>>>> Source)
>>>>> at
>>>>> com.hazelcast.spi.impl.BasicInvocation$InvocationFuture.resolveResponse(BasicInvocation.java:869)
>>>>> at
>>>>> com.hazelcast.spi.impl.BasicInvocation$InvocationFuture.resolveResponseOrThrowException(BasicInvocation.java:802)
>>>>> at
>>>>> com.hazelcast.spi.impl.BasicInvocation$InvocationFuture.get(BasicInvocation.java:698)
>>>>> at
>>>>> com.hazelcast.spi.impl.BasicInvocation$InvocationFuture.get(BasicInvocation.java:676)
>>>>> at
>>>>> com.hazelcast.spi.impl.BasicInvocation$InvocationFuture.getSafely(BasicInvocation.java:689)
>>>>> at
>>>>> com.hazelcast.concurrent.lock.LockProxySupport.lock(LockProxySupport.java:80)
>>>>> at
>>>>> com.hazelcast.concurrent.lock.LockProxySupport.lock(LockProxySupport.java:74)
>>>>> at com.hazelcast.concurrent.lock.LockProxy.lock(LockProxy.java:70)
>>>>> at
>>>>> org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent$2.run(HazelcastClusteringAgent.java:228)
>>>>>
>>>>>
>>>>> [1]. https://wso2.org/jira/browse/CARBON-14887
>>>>>
>>>>>
>>>>> Regards,
>>>>> Aruna
>>>>> --
>>>>>
>>>>> *Aruna Sujith Karunarathna* | Software Engineer
>>>>> WSO2, Inc | lean. enterprise. middleware.
>>>>> #20, Palm Grove, Colombo 03, Sri Lanka
>>>>> Mobile: +94 71 9040362 | Work: +94 112145345
>>>>> Email: [email protected] | Web: www.wso2.com
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Sameera Jayasoma,
>>>> Software Architect,
>>>>
>>>> WSO2, Inc. (http://wso2.com)
>>>> email: [email protected]
>>>> blog: http://sameera.adahas.org
>>>> twitter: https://twitter.com/sameerajayasoma
>>>> flickr: http://www.flickr.com/photos/sameera-jayasoma/collections
>>>> Mobile: 0094776364456
>>>>
>>>> Lean . Enterprise . Middleware
>>>>
>>>>
>>>
>>>
>>> --
>>> *Afkham Azeez*
>>> Director of Architecture; WSO2, Inc.; http://wso2.com
>>> Member; Apache Software Foundation; http://www.apache.org/
>>> * <http://www.apache.org/>*
>>> *email: **[email protected]* <[email protected]>
>>> * cell: +94 77 3320919 <%2B94%2077%203320919>blog: *
>>> *http://blog.afkham.org* <http://blog.afkham.org>
>>> *twitter: **http://twitter.com/afkham_azeez*
>>> <http://twitter.com/afkham_azeez>
>>> *linked-in: **http://lk.linkedin.com/in/afkhamazeez
>>> <http://lk.linkedin.com/in/afkhamazeez>*
>>>
>>> *Lean . Enterprise . Middleware*
>>>
>>
>>
>>
>> --
>> Sameera Jayasoma,
>> Software Architect,
>>
>> WSO2, Inc. (http://wso2.com)
>> email: [email protected]
>> blog: http://sameera.adahas.org
>> twitter: https://twitter.com/sameerajayasoma
>> flickr: http://www.flickr.com/photos/sameera-jayasoma/collections
>> Mobile: 0094776364456
>>
>> Lean . Enterprise . Middleware
>>
>>
>
>
> --
> *Afkham Azeez*
> Director of Architecture; WSO2, Inc.; http://wso2.com
> Member; Apache Software Foundation; http://www.apache.org/
> * <http://www.apache.org/>*
> *email: **[email protected]* <[email protected]>
> * cell: +94 77 3320919 <%2B94%2077%203320919>blog: *
> *http://blog.afkham.org* <http://blog.afkham.org>
> *twitter: **http://twitter.com/afkham_azeez*
> <http://twitter.com/afkham_azeez>
> *linked-in: **http://lk.linkedin.com/in/afkhamazeez
> <http://lk.linkedin.com/in/afkhamazeez>*
>
> *Lean . Enterprise . Middleware*
>
--
Sameera Jayasoma,
Software Architect,
WSO2, Inc. (http://wso2.com)
email: [email protected]
blog: http://sameera.adahas.org
twitter: https://twitter.com/sameerajayasoma
flickr: http://www.flickr.com/photos/sameera-jayasoma/collections
Mobile: 0094776364456
Lean . Enterprise . Middleware
_______________________________________________
Dev mailing list
[email protected]
http://wso2.org/cgi-bin/mailman/listinfo/dev
