Hi ,
I think we have found the root cause for this problem. In Cassandra Source
Code , it registers a shutdown hook in the Runtime as
Runtime.getRuntime().addShutdownHook(drainOnShutdown);
in the Source [1]. Though we get the control with OSGI level, Due to this
shut down hook registration, It triggers this and it make the cassandra
server shut down before all of the others.
In this kind of situation , what would be the best possible solution ?
Thanks
Shammi
[1]
https://svn.wso2.org/repos/wso2/carbon/platform/branches/4.1.0/dependencies/cassandra/1.1.3-wso2v3/src/java/org/apache/cassandra/service/StorageService.java
On Fri, Jun 28, 2013 at 4:46 PM, Pradeep Fernando <[email protected]> wrote:
> There is a concept call shutdownHandlers as well. There we allow component
> to register their own handler in to the system, and that handler gets
> called before we calling OSGiruntime.stop.
>
> refer, CarbonCoreServiceComponent#shutdown.
>
> will that serve your requirement. Basically you have a handle during
> Ctrl+C before OSGi runtime gets shut down. Othewise we have to look in to
> service ordering during shutdown.
>
>
> On Fri, Jun 28, 2013 at 4:37 PM, Pradeep Fernando <[email protected]>wrote:
>
>> and we are calling,
>>
>> EclipseStarter.shutdown();
>>
>>
>>
>> On Fri, Jun 28, 2013 at 4:26 PM, Pradeep Fernando <[email protected]>wrote:
>>
>>> Hi
>>>
>>> we are calling the shutdownGracefully durign Ctrl+C. -
>>> CarbonServerManager#addShutdownHook()
>>>
>>>
>>>
>>>
>>>
>>> On Fri, Jun 28, 2013 at 4:00 PM, Shammi Jayasinghe <[email protected]>wrote:
>>>
>>>> Basically What we want is ;
>>>>
>>>> We need CassandraServerService up and running until we trigger the
>>>> shutdown with Deactivate method and it should not be shut down once we
>>>> press CTRL+C.
>>>>
>>>> Is there any way that we can achieve this.
>>>>
>>>> Thanks
>>>> Shammi
>>>>
>>>>
>>>> On Fri, Jun 28, 2013 at 3:45 PM, Ishara Premadasa <[email protected]>wrote:
>>>>
>>>>> We experiencing the following issue when shutting down the MB server.
>>>>> While performing read/write operations on cassandra and if the server
>>>>> shuts
>>>>> down in middle (by clicking Ctl+C), we close the CassandraServerService
>>>>> first, before closing the AndesService. This is done in order to stop any
>>>>> operations currently running with Cassandra before shutting down the
>>>>> broker. However ever if this is handled in OSGI level, at the server
>>>>> shutdown the following logs are monitored,
>>>>>
>>>>> [2013-06-28 14:47:41,745] INFO
>>>>> {org.wso2.carbon.core.init.CarbonServerManager} - Shutdown hook
>>>>> triggered....
>>>>> [2013-06-28 14:47:41,745] INFO
>>>>> {org.wso2.carbon.core.init.CarbonServerManager} - Gracefully shutting down
>>>>> WSO2 Message Broker...
>>>>> [2013-06-28 14:47:41,747] INFO {org.wso2.carbon.core.ServerManagement}
>>>>> - Starting to switch to maintenance mode...
>>>>> *[2013-06-28 14:47:41,747] INFO {org.apache.cassandra.gms.Gossiper} -
>>>>> Announcing shutdown
>>>>> [2013-06-28 14:47:42,748] INFO
>>>>> {org.apache.cassandra.net.MessagingService} - Waiting for messaging
>>>>> service
>>>>> to quiesce
>>>>> [2013-06-28 14:47:42,749] INFO
>>>>> {org.apache.cassandra.net.MessagingService} - MessagingService shutting
>>>>> down server thread. *
>>>>> [2013-06-28 14:47:44,027] INFO {org.wso2.carbon.core.ServerManagement}
>>>>> - Stopped all transport listeners
>>>>> [2013-06-28 14:47:44,028] INFO {org.wso2.carbon.core.ServerManagement}
>>>>> - Waiting for request service completion...
>>>>> [2013-06-28 14:47:44,030] INFO {org.wso2.carbon.core.ServerManagement}
>>>>> - All requests have been served.
>>>>> [2013-06-28 14:47:44,030] INFO {org.wso2.carbon.core.ServerManagement}
>>>>> - Waiting for deployment completion...
>>>>> [2013-06-28 14:47:44,034] INFO {org.wso2.carbon.core.ServerManagement}
>>>>> - All deployment tasks have been completed.
>>>>> [2013-06-28 14:47:44,034] INFO {org.wso2.carbon.core.ServerManagement}
>>>>> - Waiting for server task completion...
>>>>> [2013-06-28 14:47:44,035] INFO {org.wso2.carbon.core.ServerManagement}
>>>>> - All server tasks have been completed.
>>>>> [2013-06-28 14:47:44,035] INFO
>>>>> {org.wso2.carbon.core.init.CarbonServerManager} - Shutting down WSO2
>>>>> Message Broker...
>>>>> [2013-06-28 14:47:44,036] INFO
>>>>> {org.apache.axis2.transport.tcp.TCPTransportSender} - TCP Sender Shutdown
>>>>> [2013-06-28 14:47:44,036] INFO
>>>>> *{org.wso2.carbon.core.init.CarbonServerManager}
>>>>> - Shutting down OSGi framework...*
>>>>> [2013-06-28 14:47:44,198] INFO
>>>>> {org.wso2.andes.server.registry.ApplicationRegistry} - Shutting down
>>>>> ApplicationRegistry(
>>>>> org.wso2.andes.server.registry.ConfigurationFileApplicationRegistry@74607cd0)
>>>>>
>>>>> [2013-06-28 14:47:44,198] INFO
>>>>> {org.wso2.andes.server.registry.ApplicationRegistry} - Shutting down
>>>>> ApplicationRegistry:
>>>>> org.wso2.andes.server.registry.ConfigurationFileApplicationRegistry@74607cd0
>>>>> [2013-06-28 14:47:44,200] INFO
>>>>> {org.wso2.andes.server.cluster.GlobalQueueManager} - Stopping all locally
>>>>> existing global queue workers
>>>>> [2013-06-28 14:47:44,200] INFO
>>>>> {org.wso2.andes.server.cluster.GlobalQueueManager} - Removing Global Queue
>>>>> Worker for Global Queue : GlobalQueue_0
>>>>> [2013-06-28 14:47:44,200] INFO
>>>>> {org.wso2.andes.server.cluster.GlobalQueueManager} - Removing Global Queue
>>>>> Worker for Global Queue : GlobalQueue_6
>>>>> [2013-06-28 14:47:44,200] INFO
>>>>> {org.wso2.andes.server.cluster.GlobalQueueManager} - Removing Global Queue
>>>>> Worker for Global Queue : GlobalQueue_8
>>>>> [2013-06-28 14:47:44,201] INFO
>>>>> {org.wso2.andes.server.cluster.GlobalQueueManager} - Removing Global Queue
>>>>> Worker for Global Queue : GlobalQueue_7
>>>>> [2013-06-28 14:47:44,201] INFO
>>>>> {org.wso2.andes.server.cluster.GlobalQueueManager} - Removing Global Queue
>>>>> Worker for Global Queue : GlobalQueue_5
>>>>> [2013-06-28 14:47:44,201] INFO
>>>>> {org.wso2.andes.server.cluster.GlobalQueueManager} - Removing Global Queue
>>>>> Worker for Global Queue : GlobalQueue_4
>>>>> [2013-06-28 14:47:44,201] INFO
>>>>> {org.wso2.andes.server.cluster.GlobalQueueManager} - Removing Global Queue
>>>>> Worker for Global Queue : GlobalQueue_1
>>>>> [2013-06-28 14:47:44,201] INFO
>>>>> {org.wso2.andes.server.cluster.GlobalQueueManager} - Removing Global Queue
>>>>> Worker for Global Queue : GlobalQueue_3
>>>>> [2013-06-28 14:47:44,201] INFO
>>>>> {org.wso2.andes.server.cluster.GlobalQueueManager} - Removing Global Queue
>>>>> Worker for Global Queue : GlobalQueue_2
>>>>> [2013-06-28 14:47:44,201] INFO
>>>>> {org.wso2.andes.server.cluster.GlobalQueueManager} - Removing Global Queue
>>>>> Worker for Global Queue : GlobalQueue_9
>>>>> [2013-06-28 14:47:44,201] INFO
>>>>> {org.wso2.andes.server.store.CassandraMessageStore} - Stopping all current
>>>>> queue message publishers
>>>>> [2013-06-28 14:47:44,201] INFO
>>>>> {org.wso2.andes.server.store.CassandraMessageStore} - Stopping all current
>>>>> topic message publishers
>>>>> [2013-06-28 14:47:44,201] INFO
>>>>> {org.wso2.andes.server.store.CassandraMessageStore} - Stopping all global
>>>>> queue workers locally
>>>>> [2013-06-28 14:47:44,202] INFO
>>>>> {org.wso2.andes.server.security.auth.manager.PrincipalDatabaseAuthenticationManager}
>>>>> - Unregistering UserManagementMBean
>>>>> [2013-06-28 14:47:44,205] INFO
>>>>> {org.wso2.carbon.andes.internal.QpidServiceComponent} - Deactivating
>>>>> Carbonized Cassandra Server...
>>>>> [2013-06-28 14:47:44,310] INFO
>>>>> {org.wso2.carbon.core.init.CarbonServerManager} - Stopping
>>>>> CarbonServerManager...
>>>>> [2013-06-28 14:47:44,342] INFO
>>>>> {org.wso2.carbon.tomcat.ext.internal.CarbonTomcatServiceComponent} -
>>>>> Stopping the carbon web-app registered under : /
>>>>> [2013-06-28 14:47:45,320] INFO
>>>>> {org.wso2.carbon.core.init.CarbonServerManager} - Shutdown complete
>>>>> [2013-06-28 14:47:45,320] INFO
>>>>> {org.wso2.carbon.core.init.CarbonServerManager} - Halting JVM
>>>>>
>>>>>
>>>>> From the line , {org.apache.cassandra.net.MessagingService} -
>>>>> MessagingService shutting down server thread, it seems that there are
>>>>> other
>>>>> cassandra services running and getting closed before we shut down
>>>>> cassandra
>>>>> component in the OSGI. We are getting cassandra connection errors due to
>>>>> this [1].
>>>>>
>>>>> How can we handle this, as it is needed to halt this thread only after
>>>>> stopping the OSGI framework?
>>>>>
>>>>> [1] https://wso2.org/jira/browse/SS-114
>>>>>
>>>>> --
>>>>> Ishara Premasada
>>>>> Software Engineer,
>>>>> WSO2 Inc. http://wso2.com/
>>>>> *Blog : http://isharapremadasa.blogspot.com/
>>>>> Twitter : https://twitter.com/ishadil
>>>>> Mobile : +94 714445832*
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Dev mailing list
>>>>> [email protected]
>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Best Regards,*
>>>>
>>>> Shammi Jayasinghe*
>>>> Associate Tech Lead
>>>> WSO2, Inc.; http://wso2.com,
>>>> mobile: +94 71 4493085
>>>>
>>>>
>>>
>>>
>>> --
>>> *Pradeep Fernando*
>>> Associate Technical Lead;WSO2 Inc.; http://wso2.com
>>>
>>> blog: http://pradeepfernando.blogspot.com
>>> m: +94776603662
>>>
>>
>>
>>
>> --
>> *Pradeep Fernando*
>> Associate Technical Lead;WSO2 Inc.; http://wso2.com
>>
>> blog: http://pradeepfernando.blogspot.com
>> m: +94776603662
>>
>
>
>
> --
> *Pradeep Fernando*
> Associate Technical Lead;WSO2 Inc.; http://wso2.com
>
> blog: http://pradeepfernando.blogspot.com
> m: +94776603662
>
--
Best Regards,*
Shammi Jayasinghe*
Associate Tech Lead
WSO2, Inc.; http://wso2.com,
mobile: +94 71 4493085
_______________________________________________
Dev mailing list
[email protected]
http://wso2.org/cgi-bin/mailman/listinfo/dev
