What is OneMinuteRate in Write Latency?

2017-11-02 Thread AI Rumman
Hi, I am trying to calculate the Read/second and Write/Second in my Cassandra 2.1 cluster. After searching and reading, I came to know about JMX bean "org.apache.cassandra.metrics:type=ClientRequest,scope=Write,name=Latency". Here I can see oneMinuteRate. I have started a brand new cluster and

Why Cassandra need full repair after incremental repair

2017-11-02 Thread dayu
Hello everyone, I have used cassandra for a while, the version is 3.0.9. I have a question why does cassandra still need full repair after used incremental repair? the full repair takes too long time. And I have searched a lot, but didn’t found any suitable answer. Can anyone answer my

Re: Cassandra 3.10 - Hints & Dropped messages logs Vs Cass 2.x version

2017-11-02 Thread kurt greaves
Well, pretty sure they still are. at least the mutation one is. but you should really use the dedicated metrics for this. On 3 Nov. 2017 01:38, "Anumod Mullachery" wrote: > thanks .. > > so the dropped hints & messages are not captured in cassandra logs, post > 3.x

sstablescrum fails with OOM

2017-11-02 Thread Shashi Yachavaram
We are cassandra 2.0.17 and have corrupted sstables. Ran offline sstablescrub but it fails with OOM. Increased the MAX_HEAP_SIZE to 8G it still fails. Can we move the corrupted sstable file and rerun sstablescrub followed by repair. -shashi..

Cassandra using a ton of native memory

2017-11-02 Thread Austin Sharp
Hi, I have a problem with Cassandra 3.11.0 on Windows. I'm testing a workload w= ith a lot of read-then-writes that had no significant problems on Cassandra= 2.x. However, now when this workload continues for a while (perhaps an hou= r), Cassandra or its JVM effectively use up all of the

Cqlsh unable to switch keyspace after Cassandra upgrade.

2017-11-02 Thread Mikhail Tsaplin
Hi, I've upgraded Cassandra from 2.1.6 to 3.0.9 on three nodes cluster. After upgrade cqlsh shows following error when trying to run "use {keyspace};" command: 'ResponseFuture' object has no attribute 'is_schema_agreed' Actual upgrade was done on Ubuntu 16.04 by running "apt-get upgrade

Re:Re: Why Cassandra need full repair after incremental repair

2017-11-02 Thread dayu
https://docs.datastax.com/en/cassandra/3.0/cassandra/operations/opsRepairNodesWhen.html So you means i am misleading by this statements. The full repair only needed when node failure + replacement, or adding a datacenter. right? At 2017-11-02 15:54:49, "kurt greaves"

During a rolling restart, one node marked as "shutdown" indefinitely by all other nodes

2017-11-02 Thread Yap Sok Ann
We have a large cluster running 2.1.19, with 3 datacenters: - xxx: 220 nodes - yyy: 220 nodes - zzz: 500 nodes The ping time between the datacenters are: - xxx to yyy: 50 ms - xxx to zzz: 240 ms - yyy to zzz: 200 ms There are some added complications such that: - our team is managing xxx and

Re: sstablescrum fails with OOM

2017-11-02 Thread sai krishnam raju potturi
Yes. Move the corrupt sstable, and run a repair on this node, so that it gets in sync with it's peers. On Thu, Nov 2, 2017 at 6:12 PM, Shashi Yachavaram wrote: > We are cassandra 2.0.17 and have corrupted sstables. Ran offline > sstablescrub but it fails with OOM. Increased

Re: Why Cassandra need full repair after incremental repair

2017-11-02 Thread kurt greaves
Where are you seeing this? If your incremental repairs work properly, full repair is only needed in certain situations, like after node failure + replacement, or adding a datacenter.​

Re: Cqlsh unable to switch keyspace after Cassandra upgrade.

2017-11-02 Thread Blake Eggleston
Looks like a bug, could you open a jira? > On Nov 2, 2017, at 2:08 AM, Mikhail Tsaplin wrote: > > Hi, > I've upgraded Cassandra from 2.1.6 to 3.0.9 on three nodes cluster. After > upgrade > cqlsh shows following error when trying to run "use {keyspace};" command: >

Re: Why Cassandra need full repair after incremental repair

2017-11-02 Thread Blake Eggleston
Because in theory, corruption of your repaired dataset is possible, which incremental repair won’t fix. In practice pre-4.0 incremental repair has some flaws that can bring deleted data back to life in some cases, which this would address. You should also evaluate whether pre-4.0 incremental

Re: Cassandra 3.10 - Hints & Dropped messages logs Vs Cass 2.x version

2017-11-02 Thread Anumod Mullachery
thanks .. so the dropped hints & messages are not captured in cassandra logs, post 3.x vs 2.x. -Anumod On Wed, Nov 1, 2017 at 4:50 PM, kurt greaves wrote: > You can get dropped message statistics over JMX. for example nodetool > tpstats has a counter for dropped hints

cassandra gc issue

2017-11-02 Thread Peng Xiao
All, We noticed that the response time jumps very high sometime. The following is from the cassandra gc log. [Eden: 760.0M(760.0M)->0.0B(11.2G) Survivors: 264.0M->96.0M Heap: 7657.7M(20.0G)->6893.3M(20.0G)] Heap after GC invocations=43481 (full 0): garbage-first heap total 20971520K,

Re: sstablescrum fails with OOM

2017-11-02 Thread Jeff Jirsa
This is not guaranteed to be safe If the corrupted sstable has a tombstone past gc grace, and another sstable has shadowed deleted data, removing the corrupt sstable will cause the data to come back to life, and repair will spread it around the ring If that’s problematic to you, you should