[
https://issues.apache.org/jira/browse/CASSANDRA-14308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16407376#comment-16407376
]
Stefania commented on CASSANDRA-14308:
--------------------------------------
That's correct, whilst a compaction is in progress a txn log file keep tracks
of the new and old sstables. The txn log is only committed once the compaction
is finished. Upon startup, if txn log files are found, they are examined and if
they were not committed then the compaction leftovers will be cleared.
May I suggest checking if anything prevents the stats component from being
written, or sync-ed to disk, before the compaction txn is committed?
> Remove invalid SSTables from interrupted compaction
> ---------------------------------------------------
>
> Key: CASSANDRA-14308
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14308
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
> Reporter: Jay Zhuang
> Assignee: Jay Zhuang
> Priority: Minor
>
> If the JVM crash while compaction is in progress, the incompleted SSTable
> won't be cleaned up, which causes "{{Stats component is missing for
> sstable}}" error in the startup log:
> {noformat}
> ERROR [SSTableBatchOpen:3] 2018-03-11 00:17:35,597 CassandraDaemon.java:207 -
> Exception in thread Thread[SSTableBatchOpen:3,5,main]
> java.lang.AssertionError: Stats component is missing for sstable
> /cassandra/data/keyspace/table-id/mc-12345-big
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:458)
> ~[apache-cassandra-3.0.14.jar:3.0.14]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:374)
> ~[apache-cassandra-3.0.14.jar:3.0.14]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader$4.run(SSTableReader.java:533)
> ~[apache-cassandra-3.0.14.jar:3.0.14]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[na:1.8.0_121]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ~[na:1.8.0_121]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[na:1.8.0_121]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_121]
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
> [apache-cassandra-3.0.14.jar:3.0.14]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_121]
> {noformat}
> The accumulated incompleted SSTables could take lots of space, especially for
> STCS which could have very large SSTables.
> Here is the script we use to delete the SSTables after node is restarted:
> {noformat}
> grep 'Stats component is missing for sstable' $SYSTEM_LOG | awk '{print $8}'
> > ~/invalid_sstables ; for ss in `cat ~/invalid_sstables`; do echo == $ss; ll
> $ss*; sudo rm $ss* ; done
> {noformat}
> I would suggest to remove these incompleted SSTables while startup.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
