Re: Questions about cleaning up/purging Hinted Handoffs

2014-09-09 Thread Rahul Menon
Yep, the hinted handoff in 1.0.8 is abysmal at best. What is your replication facter, i have had huge hints pile up, where i had to drop the entire coloumn family and then run a repair. Either that or you can use the JMX HintedHandoffManager and delete hints per endpoint. Also it maybe worthwhile

Re: Questions about cleaning up/purging Hinted Handoffs

2014-09-09 Thread Rahul Menon
DCs. (Going to 1.2.x in a few weeks) What's the procedure to drop via JMX? - Rahul 1-678-451-4545 (US) +91 99018-06625 (India) On Sep 9, 2014, at 9:23 AM, Rahul Menon ra...@apigee.com wrote: Yep, the hinted handoff in 1.0.8 is abysmal at best. What is your replication facter, i have had

Re: A question about using 'update keyspace with strategyoptions' command

2014-08-05 Thread Rahul Menon
Try the show keyspaces command and look for Options under each keyspace. Thanks Rahul On Tue, Aug 5, 2014 at 2:01 PM, Lu, Boying boying...@emc.com wrote: Hi, All, I want to run ‘update keyspace with strategy_options={dc1:3, dc2:3}’ from cassandra-cli to update the strategy options of

Re: Authentication exception

2014-07-21 Thread Rahul Menon
I could you perhaps check your ntp? On Tue, Jul 22, 2014 at 3:35 AM, Jeremy Jongsma jer...@barchart.com wrote: I routinely get this exception from cqlsh on one of my clusters: cql.cassandra.ttypes.AuthenticationException:

Re: Dead node seen as UP by replacement node

2014-03-14 Thread Rahul Menon
Since the older node is not available i would ask you to assassinate the old node and then get the node new node to bootstrap. On Thu, Mar 13, 2014 at 10:56 PM, Paulo Ricardo Motta Gomes paulo.mo...@chaordicsystems.com wrote: Yes, exactly. On Thu, Mar 13, 2014 at 1:27 PM, Rahul Menon ra

Re: Opscenter help?

2014-03-13 Thread Rahul Menon
I have seen the conflicts with sudo error but that was with 3.X rpm on the amazon ami, i was how ever able to install it from the tar ball. As Nick has pointed out, the versions of OS and Opscenter will help in looking at this. Thanks Rahul On Thu, Mar 13, 2014 at 7:56 PM, Nick Bailey

Re: Dead node seen as UP by replacement node

2014-03-13 Thread Rahul Menon
And the token value as suggested is tokenvalueoddeadnode-1 ? On Thu, Mar 13, 2014 at 9:29 PM, Paulo Ricardo Motta Gomes paulo.mo...@chaordicsystems.com wrote: Nope, they have different IPs. I'm using the procedure described here to replace a dead node:

Re: Possibly losing data with corrupted SSTables

2014-01-30 Thread Rahul Menon
at org.apache.cassandra.io.sstable.SSTableIdentityIterator.init(SSTableIdentityIterator.java:123) ... 20 more Regards, Francisco On Jan 29, 2014, at 3:38 PM, Rahul Menon ra...@apigee.com wrote: Francisco, the sstables with *-ib-* is something that was from a previous version of c*. The *-ib

Re: Possibly losing data with corrupted SSTables

2014-01-30 Thread Rahul Menon
Sessions-Users-ic-2933-Summary.db -rw-r--r-- 1 cassandra cassandra 79 Jan 15 10:50 Sessions-Users-ic-2933-TOC.txt Should I delete those 3 files? Should I run nodetool refresh after the operation? Best regards, Francisco. On Jan 30, 2014, at 2:02 PM, Rahul Menon ra...@apigee.com wrote: Looks

Re: Possibly losing data with corrupted SSTables

2014-01-29 Thread Rahul Menon
Francisco, the sstables with *-ib-* is something that was from a previous version of c*. The *-ib-* naming convention started at c* 1.2.1 but 1.2.10 onwards im sure it has the *-ic-* convention. You could try running a nodetool sstableupgrade which should ideally upgrade the sstables with the

Re: Latest Stable version of cassandra in production

2014-01-09 Thread Rahul Menon
You should refer to this https://engineering.eventbrite.com/what-version-of-cassandra-should-i-run/ Thanks Rahul On Thu, Jan 9, 2014 at 8:06 AM, Sanjeeth Kumar sanje...@exotel.in wrote: Hi all, What is the latest stable version of cassandra you have in production ? We are migrating a

Re: Crash with TombstoneOverwhelmingException

2013-12-25 Thread Rahul Menon
Sanjeeth, Looks like the error is being populated from the hintedhandoff, what is the size of your hints cf? Thanks Rahul On Wed, Dec 25, 2013 at 8:54 PM, Sanjeeth Kumar sanje...@exotel.in wrote: Hi all, One of my cassandra nodes crashes with the following exception periodically - ERROR

Re: Bulkoutputformat

2013-12-13 Thread Rahul Menon
Here you go http://thelastpickle.com/blog/2013/01/11/primary-keys-in-cql.html On Fri, Dec 13, 2013 at 7:19 AM, varun allampalli vshoori.off...@gmail.comwrote: Hi Aaron, It seems like you answered the question here. https://groups.google.com/forum/#!topic/nosql-databases/vjZA5vdycWA Can

Re: Write performance with 1.2.12

2013-12-12 Thread Rahul Menon
Quote from http://www.datastax.com/dev/blog/performance-improvements-in-cassandra-1-2 *Murmur3Partitioner is NOT compatible with RandomPartitioner, so if you’re upgrading and using the new cassandra.yaml file, be sure to change the partitioner back to RandomPartitioner* On Thu, Dec 12, 2013 at

Re: nodetool repair keeping an empty cluster busy

2013-12-11 Thread Rahul Menon
Sven So basically when you run a repair you are essentially telling your cluster to run a validation compaction, which generates a merkle tree on all the nodes. These trees are used to identify the inconsistencies. So there is quite a bit of streaming which you see as your network traffic. Rahul

Re: cassandra backup

2013-12-06 Thread Rahul Menon
You should look at this - https://github.com/amorton/cassback i dont believe its setup to use 1.2.10 and above but i believe is just small tweeks to get it running. Thanks Rahul On Fri, Dec 6, 2013 at 7:09 PM, Michael Theroux mthero...@yahoo.com wrote: Hi Marcelo, Cassandra provides and

Re: How to monitor the progress of a HintedHandoff task?

2013-12-06 Thread Rahul Menon
data center. The available bandwidth to that data center might not be able to handle the data quickly enough, resulting in stored hints. The HintedHandoff task that is started is targeting that remote node. Thanks, Tom On Tue, Dec 3, 2013 at 2:22 PM, Rahul Menon ra...@apigee.com wrote

Re: How to monitor the progress of a HintedHandoff task?

2013-12-03 Thread Rahul Menon
Tom, You should check the size of the hints column family to determine how much are present. The hints are a super column family and its keys are destination tokens. You could look at it if you would like. Hints send and timedouts are logged, you should be seeing something like Timed out

Re: How to monitor the progress of a HintedHandoff task?

2013-12-03 Thread Rahul Menon
family then.I'll give that a try. Thanks, Tom On Tue, Dec 3, 2013 at 1:43 PM, Rahul Menon ra...@apigee.com wrote: Tom, You should check the size of the hints column family to determine how much are present. The hints are a super column family and its keys are destination tokens. You could

Re: What is listEndpointsPendingHints?

2013-11-27 Thread Rahul Menon
Tom, Here is the definition List all the endpoints that this node has hints for, and count the number of hints for each such endpoint. Returns:map of endpoint - hint count I would suggest looking at at the gossipinfo to validate if there are any nodes which have that token value. If there is (

Re: 1.1.11: system keyspace is filling up

2013-11-21 Thread Rahul Menon
Oleg, The system keyspace is not replicated it is local to the node. You should check your logs to see if there are Timeouts from streaming hints, i believe the default value to stream hints it 10 seconds. When i ran into this problem i truncated hints to clear out the space and then ran a repair