Hi Users,
One of our client deployment with single node we are experiencing Cassandra
read failures through java client for below query . I could see similar error
through cqlsh client as well. This was working fine few months started
happening couple of days ago. We might be having close to 10 million records in
this table. We tried to increase "read_request_timeout_in_ms" to 20 second, but
did not help. Please share solution or advice if anyone has experienced similar
issues . Thanks lot.
CQLSh client: select distinct [partition key column] from table;
Error: code=1300 [Replica(s) failed to execute read] message="Operation failed
- received 0 responses and 1 failures" info={'failures': 1,
'received_responses': 0, 'required_responses': 1, 'consistency': 'ONE'}
>From java client:
Select distinct [partitionKey column] from table
Complete stacktrace:
om.datastax.driver.core.exceptions.ReadFailureException: Cassandra failure
during read query at consistency LOCAL_ONE (1 responses were required but only
0 replica responded, 1 failed)
java.util.concurrent.ExecutionException:
com.datastax.driver.core.exceptions.ReadFailureException: Cassandra failure
during read query at consistency LOCAL_ONE (1 responses were required but only
0 replica responded, 1 failed)
at
com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:299)
at
com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:286)
at
com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:116)
at
rx.internal.operators.OnSubscribeToObservableFuture$ToObservableFuture.call(OnSubscribeToObservableFuture.java:74)
at
rx.internal.operators.OnSubscribeToObservableFuture$ToObservableFuture.call(OnSubscribeToObservableFuture.java:43)
at rx.Observable.unsafeSubscribe(Observable.java:8314)
at
rx.internal.operators.OperatorSubscribeOn$1.call(OperatorSubscribeOn.java:94)
at rx.internal.schedulers.ScheduledAction.run(ScheduledAction.java:55)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: com.datastax.driver.core.exceptions.ReadFailureException: Cassandra
failure during read query at consistency LOCAL_ONE (1 responses were required
but only 0 replica responded, 1 failed)
at
com.datastax.driver.core.exceptions.ReadFailureException.copy(ReadFailureException.java:95)
at
com.datastax.driver.core.Responses$Error.asException(Responses.java:128)
at
com.datastax.driver.core.DefaultResultSetFuture.onSet(DefaultResultSetFuture.java:179)
at
com.datastax.driver.core.RequestHandler.setFinalResult(RequestHandler.java:184)
at
com.datastax.driver.core.RequestHandler.access$2500(RequestHandler.java:43)
at
com.datastax.driver.core.RequestHandler$SpeculativeExecution.setFinalResult(RequestHandler.java:798)
at
com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:617)
at
com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1005)
at
com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:928)
at
io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:318)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:304)
at
io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:266)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:318)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:304)
at
io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:318)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:304)
at
io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:276)
at
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:354)
at
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:244)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:318)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:304)
at
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:846)
at
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131)
at
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511)
at
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
at
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
at
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
... 1 more
Caused by: com.datastax.driver.core.exceptions.ReadFailureException: Cassandra
failure during read query at consistency LOCAL_ONE (1 responses were required
but only 0 replica responded, 1 failed)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:76)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:37)
at
com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:266)
at
com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:246)
at
io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:89)
... 15 more
Thanks
Pushpa
This message and any attachments are solely for the use of intended recipients.
The information contained herein may include trade secrets, protected health or
personal information, privileged or otherwise confidential information.
Unauthorized review, forwarding, printing, copying, distributing, or using such
information is strictly prohibited and may be unlawful. If you are not an
intended recipient, you are hereby notified that you received this email in
error, and that any review, dissemination, distribution or copying of this
email and any attachment is strictly prohibited. If you have received this
email in error, please contact the sender and delete the message and any
attachment from your system. Thank you for your cooperation
