[Dev] [DEV][MB][ERROR]In Clustering Setup [2 MB nodes + cassandra cql based database], cassandra timeout exception

2015-01-25 Thread Megala Uthayakumar 
Environment :
Cluster setup [2 MB nodes + cassanda cql basd database]
10 Publishers publishing 1K messages to the topic WSO2
1 subscriber for the topic.
Message is delivered to the subscriber. But I got the following error in MB
terminal.


[2015-01-26 11:24:32,750] ERROR
{org.wso2.andes.kernel.slot.SlotDeliveryWorker} -  Error running Message
Store Reader Error occurred while retrieving metadata list from queue
TOPIC_NODE_QUEUE_PREFIX|WSO2|NODE/10.100.4.40:4000between msg id 0 and
3340516370439
org.wso2.andes.kernel.AndesException: Error occurred while retrieving
metadata list from queue
TOPIC_NODE_QUEUE_PREFIX|WSO2|NODE/10.100.4.40:4000between
msg id 0 and 3340516370439
at
org.wso2.andes.store.cassandra.CQLBasedMessageStoreImpl.execute(CQLBasedMessageStoreImpl.java:629)
at
org.wso2.andes.store.cassandra.CQLBasedMessageStoreImpl.getMetaDataList(CQLBasedMessageStoreImpl.java:427)
at
org.wso2.andes.kernel.MessagingEngine.getMetaDataList(MessagingEngine.java:506)
at
org.wso2.andes.kernel.slot.SlotDeliveryWorker.run(SlotDeliveryWorker.java:145)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException:
Cassandra timeout during read query at consistency ONE (1 responses were
required but only 0 replica responded)
at
com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:69)
at
com.datastax.driver.core.DefaultResultSetFuture.extractCauseFromExecutionException(DefaultResultSetFuture.java:259)
at
com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:175)
at
com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:52)
at
org.wso2.andes.store.cassandra.CQLBasedMessageStoreImpl.execute(CQLBasedMessageStoreImpl.java:625)
... 6 more
Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException:
Cassandra timeout during read query at consistency ONE (1 responses were
required but only 0 replica responded)
at
com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:69)
at
com.datastax.driver.core.Responses$Error.asException(Responses.java:100)
at
com.datastax.driver.core.DefaultResultSetFuture.onSet(DefaultResultSetFuture.java:110)
at
com.datastax.driver.core.RequestHandler.setFinalResult(RequestHandler.java:249)
at
com.datastax.driver.core.RequestHandler.onSet(RequestHandler.java:433)
at
com.datastax.driver.core.Connection$Dispatcher.messageReceived(Connection.java:668)
at
org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at
org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at
org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
at
org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
at
org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:70)
at
org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at
org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
at
org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
at
org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:70)
at
org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at
org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
at
org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
at
org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
at
org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
at
org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
at
org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at
org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at
org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
at
org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
at
org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
at
org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108)
at
org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318)
at

Re: [Dev] [DEV][MB][ERROR]In Clustering Setup [2 MB nodes + cassandra cql based database], cassandra timeout exception

2015-01-25 Thread Madhawa Gunasekara 
Hi Megala,

This error has occurred due to Cassandra read consistency levels. It seems
your Cassandra cluster nodes didn't reply for your query. So Can you check
whether your Cassandra server is up or not ?
and also Did you configure Cassandra read consistency level in broker.xml ?

On Mon, Jan 26, 2015 at 11:36 AM, Megala Uthayakumar meg...@wso2.com
wrote:

 Environment :
 Cluster setup [2 MB nodes + cassanda cql basd database]
 10 Publishers publishing 1K messages to the topic WSO2
 1 subscriber for the topic.
 Message is delivered to the subscriber. But I got the following error in
 MB terminal.


 [2015-01-26 11:24:32,750] ERROR
 {org.wso2.andes.kernel.slot.SlotDeliveryWorker} -  Error running Message
 Store Reader Error occurred while retrieving metadata list from queue
 TOPIC_NODE_QUEUE_PREFIX|WSO2|NODE/10.100.4.40:4000between msg id 0 and
 3340516370439
 org.wso2.andes.kernel.AndesException: Error occurred while retrieving
 metadata list from queue 
 TOPIC_NODE_QUEUE_PREFIX|WSO2|NODE/10.100.4.40:4000between
 msg id 0 and 3340516370439
 at
 org.wso2.andes.store.cassandra.CQLBasedMessageStoreImpl.execute(CQLBasedMessageStoreImpl.java:629)
 at
 org.wso2.andes.store.cassandra.CQLBasedMessageStoreImpl.getMetaDataList(CQLBasedMessageStoreImpl.java:427)
 at
 org.wso2.andes.kernel.MessagingEngine.getMetaDataList(MessagingEngine.java:506)
 at
 org.wso2.andes.kernel.slot.SlotDeliveryWorker.run(SlotDeliveryWorker.java:145)
 at
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
 at
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 at java.lang.Thread.run(Thread.java:745)
 Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException:
 Cassandra timeout during read query at consistency ONE (1 responses were
 required but only 0 replica responded)
 at
 com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:69)
 at
 com.datastax.driver.core.DefaultResultSetFuture.extractCauseFromExecutionException(DefaultResultSetFuture.java:259)
 at
 com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:175)
 at
 com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:52)
 at
 org.wso2.andes.store.cassandra.CQLBasedMessageStoreImpl.execute(CQLBasedMessageStoreImpl.java:625)
 ... 6 more
 Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException:
 Cassandra timeout during read query at consistency ONE (1 responses were
 required but only 0 replica responded)
 at
 com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:69)
 at
 com.datastax.driver.core.Responses$Error.asException(Responses.java:100)
 at
 com.datastax.driver.core.DefaultResultSetFuture.onSet(DefaultResultSetFuture.java:110)
 at
 com.datastax.driver.core.RequestHandler.setFinalResult(RequestHandler.java:249)
 at
 com.datastax.driver.core.RequestHandler.onSet(RequestHandler.java:433)
 at
 com.datastax.driver.core.Connection$Dispatcher.messageReceived(Connection.java:668)
 at
 org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
 at
 org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
 at
 org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
 at
 org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
 at
 org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:70)
 at
 org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
 at
 org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
 at
 org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
 at
 org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:70)
 at
 org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
 at
 org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
 at
 org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
 at
 org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
 at
 org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
 at
 org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
 at
 org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
 at
 org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
 at