Re: Delete-only work loads crash Cassandra

2015-04-15 Thread Robert Wille
I can readily reproduce the bug, and filed a JIRA ticket: https://issues.apache.org/jira/browse/CASSANDRA-9194 I’m posting for posterity On Apr 13, 2015, at 11:59 AM, Robert Wille rwi...@fold3.commailto:rwi...@fold3.com wrote: Unfortunately, I’ve switched email systems and don’t have my

Re: One node misbehaving (lot's of GC), ideas?

2015-04-15 Thread Michal Michalski
Hi Erik, Forgetting for a while that it's only a single row: does this node store any super-long rows? The first things that come to my mind after reading your e-mail is unthrottled compaction (sounds like a possible issue, but it would affect other nodes too) or very large rows. Or a mix of

One node misbehaving (lot's of GC), ideas?

2015-04-15 Thread Erik Forsberg
Hi! We having problems with one node (out of 56 in total) misbehaving. Symptoms are: * High number of full CMS old space collections during early morning when we're doing bulkloads. Yes, bulkloads, not CQL, and only a few thrift insertions. * Really long stop-the-world GC events (I've seen up to

Re: Keyspace Replication changes not synchronized after adding Datacenter

2015-04-15 Thread Paul Leddy
Hello, No, that is not expected at all, to run the alter statement in each DC. Yes, it indicates a larger problem, for sure. Check that ports are open between all nodes, especially 7000, if I recall correctly. We use a simple telnet check. Paul On 04/13/2015 10:22 AM, Thunder Stumpges

Re: How to stop nodetool repair in 2.1.2?

2015-04-15 Thread Sebastian Estevez
nodetool stop *VALIDATION* On Apr 15, 2015 5:16 PM, Benyi Wang bewang.t...@gmail.com wrote: I ran nodetool repair -- keyspace table for a table, and it is still running after 4 days. I knew there is an issue for repair with vnodes https://issues.apache.org/jira/browse/CASSANDRA-5220. My

Delete query range limitation

2015-04-15 Thread Dan Kinder
I understand that range deletes are currently not supported ( http://stackoverflow.com/questions/19390335/cassandra-cql-delete-using-a-less-than-operator-on-a-secondary-key ) Since Cassandra now does have range tombstones is there a reason why it can't be allowed? Is there a ticket for supporting

Re: How to stop nodetool repair in 2.1.2?

2015-04-15 Thread Robert Coli
On Wed, Apr 15, 2015 at 3:30 PM, Benyi Wang bewang.t...@gmail.com wrote: It didn't work. I ran the command on all nodes, but I still can see the repair activities. Your input as an operator who wants a nodetool command to trivially stop repairs is welcome here :

Re: How to stop nodetool repair in 2.1.2?

2015-04-15 Thread Benyi Wang
It didn't work. I ran the command on all nodes, but I still can see the repair activities. On Wed, Apr 15, 2015 at 3:20 PM, Sebastian Estevez sebastian.este...@datastax.com wrote: nodetool stop *VALIDATION* On Apr 15, 2015 5:16 PM, Benyi Wang bewang.t...@gmail.com wrote: I ran nodetool

How to stop nodetool repair in 2.1.2?

2015-04-15 Thread Benyi Wang
I ran nodetool repair -- keyspace table for a table, and it is still running after 4 days. I knew there is an issue for repair with vnodes https://issues.apache.org/jira/browse/CASSANDRA-5220. My question is how I can kill this sequential repair? I killed the process which I ran the repair

Re: How to stop nodetool repair in 2.1.2?

2015-04-15 Thread Benyi Wang
Using JMX worked. Thanks a lot. On Wed, Apr 15, 2015 at 3:57 PM, Robert Coli rc...@eventbrite.com wrote: On Wed, Apr 15, 2015 at 3:30 PM, Benyi Wang bewang.t...@gmail.com wrote: It didn't work. I ran the command on all nodes, but I still can see the repair activities. Your input as an