[
https://issues.apache.org/jira/browse/CASSANDRA-6008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13848078#comment-13848078
]
Jonathan Ellis commented on CASSANDRA-6008:
-------------------------------------------
This means that instead of throwing an error if we restart before
removeUnfinishedCompactionLeftovers finishes, we'll leave both old and new
sstables from unfinished compactions live, which defeats the purpose for
counters.
For 2.1 that would be okay (since we're assuming CASSANDRA-4775 will be done
before we release) but for 2.0 it isn't, unfortunately.
I think the alternatives are
# Switch back to delete-first, and add a debug line instead of
IllegalStateException. (Can delete from compaction_log incrementally too to
reduce the window of inconsistency.)
# Do a dance of renaming back to .tmp instead of deleting, then removing
compaction_log entry, then deleting. .tmp will be included in the unfinished
list, but if there is no corresponding compaction_log entry they can just be
deleted
I'd lean towards saying the extra complexity of #2 isn't worth the security
blanket of the ISE.
> Getting 'This should never happen' error at startup due to sstables missing
> ---------------------------------------------------------------------------
>
> Key: CASSANDRA-6008
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6008
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Reporter: John Carrino
> Assignee: Tyler Hobbs
> Fix For: 2.0.4
>
> Attachments: 6008-2.0-part2.patch, 6008-2.0-v1.patch,
> 6008-trunk-v1.patch
>
>
> Exception encountered during startup: "Unfinished compactions reference
> missing sstables. This should never happen since compactions are marked
> finished before we start removing the old sstables"
> This happens when sstables that have been compacted away are removed, but
> they still have entries in the system.compactions_in_progress table.
> Normally this should not happen because the entries in
> system.compactions_in_progress are deleted before the old sstables are
> deleted.
> However at startup recovery time, old sstables are deleted (NOT BEFORE they
> are removed from the compactions_in_progress table) and then after that is
> done it does a truncate using SystemKeyspace.discardCompactionsInProgress
> We ran into a case where the disk filled up and the node died and was bounced
> and then failed to truncate this table on startup, and then got stuck hitting
> this exception in ColumnFamilyStore.removeUnfinishedCompactionLeftovers.
> Maybe on startup we can delete from this table incrementally as we clean
> stuff up in the same way that compactions delete from this table before they
> delete old sstables.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
