This is an ActiveMQ question but it is using Zookeeper. Posted in ActiveMQ
forum but didn't get any response
We have 3 brokers connected to Zookeeper using replicated leveldb. Some times
we get the following INFO message on Master
Client session timed out, have not heard from server in 2xxxms for sessionid
xxxxx
we do have zkSessionTimeout="3s"
and the broker recycles and starts as slave, most of the times it becomes slave
successfully. but some times while restarting gives the following warning and
broker dies
2017-02-01 17:24:33,062 | WARN | Exception while executing listener (ignored)
| org.apache.activemq.leveldb.replicated.groups.ZKClient | ZooKeeper state
change dispatcher thread
java.lang.NullPointerException
at
org.linkedin.zookeeper.tracker.ZooKeeperTreeTracker.raiseEvents(ZooKeeperTreeTracker.java:402)[org.linkedin.zookeeper-impl-1.4.0.jar:]
at
org.linkedin.zookeeper.tracker.ZooKeeperTreeTracker.track(ZooKeeperTreeTracker.java:240)[org.linkedin.zookeeper-impl-1.4.0.jar:]
at
org.linkedin.zookeeper.tracker.ZooKeeperTreeTracker.track(ZooKeeperTreeTracker.java:228)[org.linkedin.zookeeper-impl-1.4.0.jar:]
at
org.apache.activemq.leveldb.replicated.groups.ZooKeeperGroup.onConnected(ZooKeeperGroup.scala:124)[activemq-leveldb-store-5.13.3.jar:5.13.3]
at
org.apache.activemq.leveldb.replicated.groups.ZKClient.callListeners(ZKClient.java:385)[activemq-leveldb-store-5.13.3.jar:5.13.3]
at
org.apache.activemq.leveldb.replicated.groups.ZKClient$StateChangeDispatcher.run(ZKClient.java:354)[activemq-leveldb-store-5.13.3.jar:5.13.3]
2017-02-01 17:24:33,065 | ERROR | Failed to start Apache ActiveMQ
([broker-aamqwafiu1-amq, null], java.lang.NullPointerException) |
org.apache.activemq.broker.BrokerService | main
2017-02-01 17:24:33,066 | INFO | Apache ActiveMQ 5.13.3
(broker-aamqwafiu1-amq, null) is shutting down |
org.apache.activemq.broker.BrokerService | main
2017-02-01 17:24:33,067 | INFO | Connector openwire stopped |
org.apache.activemq.broker.TransportConnector | main
2017-02-01 17:24:33,068 | INFO | StateChangeDispatcher terminated. |
org.apache.activemq.leveldb.replicated.groups.ZKClient | ZooKeeper state change
dispatcher thread
2017-02-01 17:24:33,071 | INFO | Session: 0x458b45c4bc3001e closed |
org.apache.zookeeper.ZooKeeper | main
2017-02-01 17:24:33,072 | INFO | EventThread shut down |
org.apache.zookeeper.ClientCnxn | main-EventThread
2017-02-01 17:24:33,174 | INFO | Apache ActiveMQ 5.13.3
(broker-aamqwafiu1-amq, null) uptime 0.178 seconds |
org.apache.activemq.broker.BrokerService | main
2017-02-01 17:24:33,175 | INFO | Apache ActiveMQ 5.13.3
(broker-aamqwafiu1-amq, null) is shutdown |
org.apache.activemq.broker.BrokerService | main
2017-02-01 17:24:33,176 | INFO | Closing
org.apache.activemq.xbean.XBeanBrokerFactory$1@717ee24a: startup date [Wed Feb
01 17:24:32 EST 2017]; root of context hierarchy |
org.apache.activemq.xbean.XBeanBrokerFactory$1 | main
2017-02-01 17:24:33,177 | WARN | Exception thrown from LifecycleProcessor on
context close | org.apache.activemq.xbean.XBeanBrokerFactory$1 | main
java.lang.IllegalStateException: LifecycleProcessor not initialized - call
'refresh' before invoking lifecycle methods via the context:
org.apache.activemq.xbean.XBeanBrokerFactory$1@717ee24a: startup date [Wed Feb
01 17:24:32 EST 2017]; root of context hierarchy
at
org.springframework.context.support.AbstractApplicationContext.getLifecycleProcessor(AbstractApplicationContext.java:357)[spring-context-4.1.9.RELEASE.jar:4.1.9.RELEASE]
at
org.springframework.context.support.AbstractApplicationContext.doClose(AbstractApplicationContext.java:884)[spring-context-4.1.9.RELEASE.jar:4.1.9.RELEASE]
at
org.springframework.context.support.AbstractApplicationContext.close(AbstractApplicationContext.java:843)[spring-context-4.1.9.RELEASE.jar:4.1.9.RELEASE]
at
org.apache.activemq.hooks.SpringContextHook.run(SpringContextHook.java:30)[activemq-spring-5.13.3.jar:5.13.3]
at
org.apache.activemq.broker.BrokerService.stop(BrokerService.java:873)[activemq-broker-5.13.3.jar:5.13.3]
at
org.apache.activemq.xbean.XBeanBrokerService.stop(XBeanBrokerService.java:122)[activemq-spring-5.13.3.jar:5.13.3]
at
org.apache.activemq.broker.BrokerService.start(BrokerService.java:627)[activemq-broker-5.13.3.jar:5.13.3]
at
org.apache.activemq.xbean.XBeanBrokerService.afterPropertiesSet(XBeanBrokerService.java:73)[activemq-spring-5.13.3.jar:5.13.3]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)[:1.7.0_60]
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)[:1.7.0_60]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)[:1.7.0_60]
at java.lang.reflect.Method.invoke(Method.java:606)[:1.7.0_60]
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeCustomInitMethod(AbstractAutowireCapableBeanFactory.java:1700)[spring-beans-4.1.9.RELEASE.jar:4.1.9.RELEASE]
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1639)[spring-beans-4.1.9.RELEASE.jar:4.1.9.RELEASE]
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1568)[spring-beans-4.1.9.RELEASE.jar:4.1.9.RELEASE]
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:539)[spring-beans-4.1.9.RELEASE.jar:4.1.9.RELEASE]
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:476)[spring-beans-4.1.9.RELEASE.jar:4.1.9.RELEASE]
at
org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:303)[spring-beans-4.1.9.RELEASE.jar:4.1.9.RELEASE]
at
org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)[spring-beans-4.1.9.RELEASE.jar:4.1.9.RELEASE]
at
org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:299)[spring-beans-4.1.9.RELEASE.jar:4.1.9.RELEASE]
at
org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194)[spring-beans-4.1.9.RELEASE.jar:4.1.9.RELEASE]
at
org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:755)[spring-beans-4.1.9.RELEASE.jar:4.1.9.RELEASE]
We are using
Zookeeper 3.4.8
activemq 5.13.3
Any insight is appreciated
