Re: Possibly losing data with corrupted SSTables

2014-03-14 Thread Robert Coli
On Wed, Feb 12, 2014 at 9:20 AM, Francisco Nogueira Calmon Sobral fsob...@igcorp.com.br wrote: I've removed the corrupted sstables and 'nodetool repair' ran successfully for the column family. I'm not sure whether or not we've lost data. If you read/write at CL.ONE, there is a non-zero

Re: Possibly losing data with corrupted SSTables

2014-02-12 Thread Francisco Nogueira Calmon Sobral
Hi, Rahul. I've removed the corrupted sstables and 'nodetool repair' ran successfully for the column family. I'm not sure whether or not we've lost data. Best regards, Francisco Sobral On Jan 30, 2014, at 3:58 PM, Rahul Menon ra...@apigee.com wrote: Yes should delete all files related to

Re: Possibly losing data with corrupted SSTables

2014-02-12 Thread sankalp kohli
You might want to look at this JIRA i filed today CASSANDRA-6696 https://issues.apache.org/jira/browse/CASSANDRA-6696 You are good if you are fine with data reappearing. On Wed, Feb 12, 2014 at 9:20 AM, Francisco Nogueira Calmon Sobral fsob...@igcorp.com.br wrote: Hi, Rahul. I've removed

Re: Possibly losing data with corrupted SSTables

2014-01-30 Thread Rahul Menon
Looks like the sstables are corrupt. I dont believe there is a method to recover those sstables. I would delete them and run a repair to ensure data consistency. Rahul On Wed, Jan 29, 2014 at 11:29 PM, Francisco Nogueira Calmon Sobral fsob...@igcorp.com.br wrote: Hi, Rahul. I've run

Re: Possibly losing data with corrupted SSTables

2014-01-30 Thread Francisco Nogueira Calmon Sobral
Ok. I'll try this idea with one sstable. But, should I delete all the files associated with it? I mean, there is a difference in the number of files between the BAD sstable and a GOOD one, as I've already shown: BAD -- -rw-r--r-- 8 cassandra cassandra 991M Nov 8 15:11

Re: Possibly losing data with corrupted SSTables

2014-01-30 Thread Rahul Menon
Yes should delete all files related to cfname-ib-num-extension.db Run a repair after deletion On Thu, Jan 30, 2014 at 10:17 PM, Francisco Nogueira Calmon Sobral fsob...@igcorp.com.br wrote: Ok. I'll try this idea with one sstable. But, should I delete all the files associated with it? I

Possibly losing data with corrupted SSTables

2014-01-29 Thread Francisco Nogueira Calmon Sobral
Dear experts, We are facing a annoying problem in our cluster. We have 9 amazon extra large linux nodes, running Cassandra 1.2.11. The short story is that after moving the data from one cluster to another, we've been unable to run 'nodetool repair'. It get stuck due to a

Re: Possibly losing data with corrupted SSTables

2014-01-29 Thread Rahul Menon
Francisco, the sstables with *-ib-* is something that was from a previous version of c*. The *-ib-* naming convention started at c* 1.2.1 but 1.2.10 onwards im sure it has the *-ic-* convention. You could try running a nodetool sstableupgrade which should ideally upgrade the sstables with the

Re: Possibly losing data with corrupted SSTables

2014-01-29 Thread Francisco Nogueira Calmon Sobral
Hi, Rahul. I've run nodetool upgradesstable only in the problematic CF. It throwed the following exception: Error occurred while upgrading the sstables for keyspace Sessions java.util.concurrent.ExecutionException: org.apache.cassandra.io.sstable.CorruptSSTableException: java.io.IOException: