[
https://issues.apache.org/jira/browse/CASSANDRA-4411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13423108#comment-13423108
]
Mina Naguib commented on CASSANDRA-4411:
----------------------------------------
Quick follow-up
All the problematic nodes have been offline scrubbed (successfully using
--manifest-check to speed things up). There are no more compaction errors /
pending compactions.
Like Anton, I'm a bit weary and keeping an eye on things - but so far so good.
On a tangent, it occurred to me that the amount of time it takes to run (
sstablescrub --manifest-check ) is mostly reading the sstables - the check
itself and demoting the bad sstables to L0 appears very cheap - would it be a
good idea to perform that check automatically on cassandra startup (after the
sstables have been read) ? It *may* be a quick fix for 1.1.3 to help people
out who have been bitten by this but don't know it yet.
> Assertion with LCS compaction
> -----------------------------
>
> Key: CASSANDRA-4411
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4411
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.1.2
> Reporter: Anton Winter
> Assignee: Sylvain Lebresne
> Fix For: 1.1.3
>
> Attachments: 0001-Add-debugging-info-for-LCS.txt, 4411-followup.txt,
> 4411.txt, assertion-w-more-debugging-info-omid.log,
> assertion.moreinfo.system.log, system.log
>
>
> As instructed in CASSANDRA-4321 I have raised this issue as a continuation of
> that issue as it appears the problem still exists.
> I have repeatedly run sstablescrub across all my nodes after the 1.1.2
> upgrade until sstablescrub shows no errors. The exceptions described in
> CASSANDRA-4321 do not occur as frequently now but the integrity check still
> throws exceptions on a number of nodes. Once those exceptions occur
> compactionstats shows a large number of pending tasks with no progression
> afterwards.
> {code}
> ERROR [CompactionExecutor:150] 2012-07-05 04:26:15,570
> AbstractCassandraDaemon.java (line 134) Exception in thread
> Thread[CompactionExecutor:150,1,main]
> java.lang.AssertionError
> at
> org.apache.cassandra.db.compaction.LeveledManifest.promote(LeveledManifest.java:214)
> at
> org.apache.cassandra.db.compaction.LeveledCompactionStrategy.handleNotification(LeveledCompactionStrategy.java:158)
> at
> org.apache.cassandra.db.DataTracker.notifySSTablesChanged(DataTracker.java:531)
> at
> org.apache.cassandra.db.DataTracker.replaceCompactedSSTables(DataTracker.java:254)
> at
> org.apache.cassandra.db.ColumnFamilyStore.replaceCompactedSSTables(ColumnFamilyStore.java:978)
> at
> org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:200)
> at
> org.apache.cassandra.db.compaction.LeveledCompactionTask.execute(LeveledCompactionTask.java:50)
> at
> org.apache.cassandra.db.compaction.CompactionManager$1.runMayThrow(CompactionManager.java:150)
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:636)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
