Hi list, It happened that one of our EC2 instance of our cluster got rebooted. Unfortunately when back Cassandra 3.0.6 failed to restart complaining about:
ERROR [NonPeriodicTasks:1] 2017-11-04 03:44:20,019 LogTransaction.java:204 - Unable to delete /<some-path>/system/local/ma-292-big-Data.db as it does not exist ERROR [CompactionExecutor:1] 2017-11-04 03:44:20,136 CassandraDaemon.java:195 - Exception in thread Thread[CompactionExecutor:1,1,main] org.apache.cassandra.io.sstable.CorruptSSTableException: Corrupted: /<some-path>/system/local/ma-294-big-Data.db at org.apache.cassandra.io.sstable.format.big.BigTableScanner$KeyScanningIterator.computeNext(BigTableScanner.java:351) ~[apache-cassandra-3.0.6.jar:3.0.6] I tried to run an sstablescrub system local with no luck as it seems to get stuck in an endless loop: Scrubbing BigTableReader(path='/<some-path>/system/local/ma-293-big-Data.db') (5862 bytes) WARNING: Error reading row (stacktrace follows): Retrying from row index; data is 5855 bytes starting at 7 WARNING: Retry failed too. Skipping to next row (retry's stacktrace follows) WARNING: Error reading row (stacktrace follows): WARNING: Row starting at position 0 is unreadable; skipping to next WARNING: Data file row position 7 differs from index file row position -1 WARNING: Error reading row (stacktrace follows): I do have a snapshot taken 3 hours before the crash. Is it safe to restore that snapshot and run a repair? Is there anything else I can try to recover ? Any help much appreciated, regards
