Hi All,
We are facing problems of failure of Read-Repair stages with error Digest
Mismatch and count is 300+ per day per node.
At the same time, we are experiencing node is getting overloaded for a
quick couple of seconds due to long GC pauses (of around 7-8 seconds). We
are not running a repair
: Mutation dropped
What does rpc_timeout control? Only the reads/writes?
Yes.
like data stream,
streaming_socket_timeout_in_ms in the yaml
merkle tree request?
Either no time out or a number of days, cannot remember which right now.
What is the side effect if it's set to a really small number
under 3ms.
Any suggestion is very much appreciated.
Thanks.
-Wei
- Original Message -
From: aaron morton aa...@thelastpickle.com
To: Cassandra User user@cassandra.apache.org
Sent: Thursday, February 21, 2013 9:20:49 AM
Subject: Re: Mutation dropped
What does rpc_timeout
To: user@cassandra.apache.org
Sent: Tuesday, February 19, 2013 7:32 PM
Subject: Re: Mutation dropped
Does the rpc_timeout not control the client timeout ?
No it is how long a node will wait for a response from other nodes before
raising a TimedOutException if less than CL nodes have
is a bit high. Our 50th percentile was
great under 3ms.
Any suggestion is very much appreciated.
Thanks.
-Wei
- Original Message -
From: aaron morton aa...@thelastpickle.com
To: Cassandra User user@cassandra.apache.org
Sent: Thursday, February 21, 2013 9:20:49 AM
Subject: Re: Mutation dropped
, say 20ms?
Thanks.
-Wei
From: aaron morton aa...@thelastpickle.com
To: user@cassandra.apache.org
Sent: Tuesday, February 19, 2013 7:32 PM
Subject: Re: Mutation dropped
Does the rpc_timeout not control the client timeout ?No it is how long a node
will wait
UN 10.x.x.x 880.77 GB 256 100.0%
95d59054-be99-455f-90d1-f43981d3d778 rack1
Once we hit a very high TPS (around 50k/sec of inserts), the nodes start
falling behind and we see the mutation dropped messages. But there are no
failures on the client. Does that mean
...@thelastpickle.com]
Sent: 17 February 2013 11:26
To: user@cassandra.apache.org
Subject: Re: Mutation dropped
You are hitting the maximum throughput on the cluster.
The messages are dropped because the node fails to start processing them before
rpc_timeout.
However the request is still a success because
100.0%
95d59054-be99-455f-90d1-f43981d3d778 rack1
Once we hit a very high TPS (around 50k/sec of inserts), the nodes start
falling behind and we see the mutation dropped messages. But there are no
failures on the client. Does that mean other node is not able to persist
Hi - I am doing a load test using YCSB across 2 nodes in a cluster and seeing a
lot of mutation dropped messages. I understand that this is due to the replica
not being written to the
other node ? RF = 2, CL =1.
From the wiki -
For MUTATION messages this means that the mutation was not applied
:08
To: user@cassandra.apache.org
Subject: Mutation dropped
Hi - I am doing a load test using YCSB across 2 nodes in a cluster and seeing a
lot of mutation dropped messages. I understand that this is due to the replica
not being written to the
other node ? RF = 2, CL =1.
From the wiki
Subject: Re: Mutation Dropped Messages
I increased the size of the cluster also the concurrent_writes parameter. Still
there is a node which keeps on dropping the mutation messages.
Ensure all the nodes have the same spec, and the nodes have the same config. In
a virtual environment consider moving
@cassandra.apache.org
Subject: Re: Mutation Dropped Messages
I increased the size of the cluster also the concurrent_writes parameter.
Still there is a node which keeps on dropping the mutation messages.
Ensure all the nodes have the same spec, and the nodes have the same config.
In a virtual environment
Hi All,
While benchmarking Cassandra I found Mutation Dropped messages in the logs.
Now I know this is a good old question. It will be really great if someone can
provide a check list to recover when such a thing happens. I am looking for
answers of the following questions -
1. Which
“Mutation Dropped” messages in the logs.
Now I know this is a good old question. It will be really great if someone
can provide a check list to recover when such a thing happens. I am looking
for answers of the following questions -
1. Which parameters to tune in the config files
...@thelastpickle.com]
Sent: Monday, March 05, 2012 4:15 PM
To: user@cassandra.apache.org
Subject: Re: Mutation Dropped Messages
1. Which parameters to tune in the config files? – Especially looking
for heavy writes
The node is overloaded. It may be because there are no enough nodes
16 matches
Mail list logo