Re: Is deleting live sstable safe in this scenario?

2020-05-27 Thread Nitan Kainth
Yeah, I meant the down node can’t participate in repairs Regards, Nitan Cell: 510 449 9629 > On May 27, 2020, at 2:09 PM, Leon Zaruvinsky wrote: > >  > Yep, Jeff is right, the intention would be to run a repair limited to the > available nodes. > >> On Wed, May 27, 2020 at 2:59 PM Jeff

Re: Is deleting live sstable safe in this scenario?

2020-05-27 Thread Leon Zaruvinsky
Yep, Jeff is right, the intention would be to run a repair limited to the available nodes. On Wed, May 27, 2020 at 2:59 PM Jeff Jirsa wrote: > The "-hosts " flag tells cassandra to only compare trees/run repair on the > hosts you specify, so if you have 3 replicas, but 1 replica is down, you >

Re: Is deleting live sstable safe in this scenario?

2020-05-27 Thread Jeff Jirsa
The "-hosts " flag tells cassandra to only compare trees/run repair on the hosts you specify, so if you have 3 replicas, but 1 replica is down, you can provide -hosts with the other two, and it will make sure those two are in sync (via merkle trees, etc), but ignore the third. On Wed, May 27,

Re: Is deleting live sstable safe in this scenario?

2020-05-27 Thread Nitan Kainth
Jeff, If Cassandra is down how will it generate merkle tree to compare? Regards, Nitan Cell: 510 449 9629 > On May 27, 2020, at 11:15 AM, Jeff Jirsa wrote: > >  > You definitely can repair with a node down by passing `-hosts specific_hosts` > >> On Wed, May 27, 2020 at 9:06 AM Nitan Kainth

Re: Is deleting live sstable safe in this scenario?

2020-05-27 Thread Jeff Jirsa
You definitely can repair with a node down by passing `-hosts specific_hosts` On Wed, May 27, 2020 at 9:06 AM Nitan Kainth wrote: > I didn't get you Leon, > > But, the simple thing is just to follow the steps and you will be fine. > You can't run the repair if the node is down. > > On Wed, May

Re: Is deleting live sstable safe in this scenario?

2020-05-27 Thread Nitan Kainth
I didn't get you Leon, But, the simple thing is just to follow the steps and you will be fine. You can't run the repair if the node is down. On Wed, May 27, 2020 at 10:34 AM Leon Zaruvinsky wrote: > Hey Jeff/Nitan, > > 1) this concern should not be a problem if the repair happens before the >

Re: Is deleting live sstable safe in this scenario?

2020-05-27 Thread Leon Zaruvinsky
Hey Jeff/Nitan, 1) this concern should not be a problem if the repair happens before the corrupted node is brought back online, right? 2) in this case, is option (3) equivalent to replacing the node? where we repair the two live nodes and then bring up the third node with no data Leon On Tue,

Re: Is deleting live sstable safe in this scenario?

2020-05-26 Thread Jeff Jirsa
There’s two problems with this approach if you need strict correctness 1) after you delete the sstable and before you repair you’ll violate consistency, so you’ll potentially serve incorrect data for a while 2) The sstable May have a tombstone past gc grace that’s shadowing data in another

Re: Is deleting live sstable safe in this scenario?

2020-05-26 Thread Nitan Kainth
Stop the node Delete as per option 2 Run repair Regards, Nitan Cell: 510 449 9629 > On May 26, 2020, at 6:46 PM, Leon Zaruvinsky wrote: > >  > Hi all, > > I'm looking to understand Cassandra's behavior in an sstable corruption > scenario, and what the minimum amount of work is that needs