[
https://issues.apache.org/jira/browse/CASSANDRA-10041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15230017#comment-15230017
]
Rob Emery commented on CASSANDRA-10041:
---------------------------------------
Interesting; from our perspective the nodes that intermittently throw this
exception are not actually down. They are delivering other workloads without
any apparent problem. We just get a few of these per day and have been unable
to track down any cause (there does not appear to be any IO pressure or GC
pauses etc); therefore for our situation all information, no matter how small,
is vital for us to be able to diagnose the problem. Currently we have no way of
tracking the point during the mutation that the issue is actually occurring.
> "timeout during write query at consistency ONE" when updating counter at
> consistency QUORUM and 2 of 3 nodes alive
> ------------------------------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-10041
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10041
> Project: Cassandra
> Issue Type: Bug
> Environment: centos 6.6 server, java version "1.8.0_45", cassandra
> 2.1.8, 3 machines, keyspace with replication factor 3
> Reporter: Anton Lebedevich
> Fix For: 2.1.x
>
>
> Test scenario is: kill -9 one node, wait 60 seconds, start it back, wait till
> it becomes available, wait 120 seconds (during that time all 3 nodes are up),
> repeat with the next node. Application reads from one table and updates
> counters in another table with consistency QUORUM. When one node out of 3 is
> killed application logs this exception for several seconds:
> {noformat}
> Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException:
> Cassandra timeout during write query at consistency ONE (1 replica were
> required but only 0 acknowledged the write)
> at
> com.datastax.driver.core.Responses$Error$1.decode(Responses.java:57)
> ~[com.datastax.cassandra.cassandra-driver-core-2.1.6.jar:na]
> at
> com.datastax.driver.core.Responses$Error$1.decode(Responses.java:37)
> ~[com.datastax.cassandra.cassandra-driver-core-2.1.6.jar:na]
> at
> com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:204)
> ~[com.datastax.cassandra.cassandra-driver-core-2.1.6.jar:na]
> at
> com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:195)
> ~[com.datastax.cassandra.cassandra-driver-core-2.1.6.jar:na]
> at
> io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:89)
> [io.netty.netty-codec-4.0.27.Final.jar:4.0.27.Final]
> ... 13 common frames omitted
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
