Re: Currupt sstables when upgrading from 2.1.8 to 2.1.9

2015-09-30 Thread Robert Coli
On Wed, Sep 30, 2015 at 3:08 AM, George Sigletos wrote: > This is way too many manual steps. I was wondering why not just removing > the entire /var/lib/cassandra/data folder + commitlogs, restart the node > again and wait to catch up > Briefly, there is a low but

Re: Currupt sstables when upgrading from 2.1.8 to 2.1.9

2015-09-30 Thread George Sigletos
Hello again and sorry for the late response, Still having problems with upgrading from 2.1.8 to 2.1.9. I decided to start the problematic nodes with "disk_failure_policy: best_effort" Currently running "nodetool scrub " Then removing the corrupted sstables and planning to run repair

Re: Currupt sstables when upgrading from 2.1.8 to 2.1.9

2015-09-24 Thread Robert Coli
On Tue, Sep 15, 2015 at 9:42 AM, Nate McCall wrote: > Either way, you are going to have to run nodetool scrub. I'm not sure if > it's better to do this from 2.1.8 or from 2.1.9 with "disk_failure_policy: > ignore" > A node which has lost a SSTable also needs to be

Re: Currupt sstables when upgrading from 2.1.8 to 2.1.9

2015-09-24 Thread Robert Coli
On Thu, Sep 24, 2015 at 3:00 PM, Robert Coli wrote: > A node which has lost a SSTable also needs to be repaired immediately. > Forgot to mention, you can repair via this technique : https://issues.apache.org/jira/browse/CASSANDRA-6961 =Rob

Currupt sstables when upgrading from 2.1.8 to 2.1.9

2015-09-15 Thread George Sigletos
Hello, I tried to upgrade two of our clusters from 2.1.8 to 2.1.9. In some, but not all nodes, I got errors about corrupt sstables when restarting. I downgraded back to 2.1.8 for now. Has anybody else faced the same problem? Should sstablescrub fix the problem? I ddin't tried that yet. Kind

Re: Currupt sstables when upgrading from 2.1.8 to 2.1.9

2015-09-15 Thread Nate McCall
You have a/some corrupt SSTables. 2.1.9 is doing strict checking at startup and reacting based on "disk_failure_policy" per the stack trace. For details, see: https://issues.apache.org/jira/browse/CASSANDRA-9686 Either way, you are going to have to run nodetool scrub. I'm not sure if it's better