Re: problem removing dead node from ring

2014-06-07 Thread Curious Patient
Hey all, OK I gave removing the downed node from the cassandra ring another try. To recap what's going on, this is what my ring looks like with nodetool status: [root@beta-new:~] #nodetool status Datacenter: datacenter1 === Status=Up/Down |/

Re: problem removing dead node from ring

2014-06-04 Thread Robert Coli
On Tue, Jun 3, 2014 at 9:03 PM, Matthew Allen matthew.j.al...@gmail.com wrote: Thanks Robert, this makes perfect sense. Do you know if CASSANDRA-6961 will be ported to 1.2.x ? I just asked driftx, he said not gonna happen. And apologies if these appear to be dumb questions, but is a

problem removing dead node from ring

2014-06-03 Thread Curious Patient
One of the nodes in a cassandra cluster has died. I'm using cassandra 2.0.7 throughout. When I do a nodetool status this is what I see (real addresses have been replaced with fake 10 nets) [root@beta-new:/opt] #nodetool status Datacenter: datacenter1 ===

Re: problem removing dead node from ring

2014-06-03 Thread Curious Patient
Hi Rob, If you are replacing an address, you need to use the identical initial_token to the node you are replacing, not the token -1. Thanks, I hope that does the trick. Btw.. was my idea of how to get at the initial token of the missing/dead node correct? .i.e. nodetool ring | grep

Re: problem removing dead node from ring

2014-06-03 Thread Robert Coli
On Tue, Jun 3, 2014 at 10:53 AM, Curious Patient mongoma...@gmail.com wrote: I want to be sure I'm using the right token. In nodetool ring, if you're not using vnodes, only one token should be listed with both the IP of the old node and the status Down. If you are using vnodes, it's a comma

Re: problem removing dead node from ring

2014-06-03 Thread Curious Patient
In nodetool ring, if you're not using vnodes, only one token should be listed with both the IP of the old node and the status Down. If you are using vnodes, it's a comma delimited list in initial_token, which you can get from : nodetool info -T | grep Token | awk '{print $3}' | paste -s -d,

Re: problem removing dead node from ring

2014-06-03 Thread Robert Coli
On Tue, Jun 3, 2014 at 3:48 PM, Matthew Allen matthew.j.al...@gmail.com wrote: Just out of curiosity, for a dead node, would it be possible to just - replace the node (no data in data/commit dirs), same IP Address, same hostname. - restore the cassandra.yaml (initial_token etc) - set

Re: problem removing dead node from ring

2014-06-03 Thread Matthew Allen
| That would work, but until CASSANDRA-6961 [1] there is no way to prevent this node from having a long window where it may serve stale | reads at CLs below QUORUM, until the rebuild completes. Thanks Robert, this makes perfect sense. Do you know if CASSANDRA-6961 will be ported to 1.2.x ? And