[
https://issues.apache.org/jira/browse/CASSANDRA-8191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14249680#comment-14249680
]
Jimmy Mårdell commented on CASSANDRA-8191:
------------------------------------------
We got the same exception and stack trace after running repair where some nodes
got streamed a lot of new data. The table used LCS and got a lot of data in
level 0. The exception kept repeating itself every now and then, and no
progress was made compactions. Restarting the node however seems to have solved
the problem. This was on 2.0.11.
> After sstablesplit all nodes log RuntimeException in CompactionExecutor (Last
> written key DecoratedKey... >= current key DecoratedKey)
> --------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-8191
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8191
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: DSE 4.5.2 (Cassandra 2.0.10)
> Reporter: Nikolai Grigoriev
>
> While recovering the cluster from CASSANDRA-7949 (using the flag from
> CASSANDRA-6621) I had to use sstablesplit tool to split large sstables. Nodes
> were off while using this tool and only one sstablesplit instance was
> running, of course.
> After splitting was done I have restarted the nodes and they all started
> compacting the data. All the nodes are logging the exceptions like this:
> {code}
> ERROR [CompactionExecutor:4028] 2014-10-26 23:14:52,653 CassandraDaemon.java
> (line 199) Exception in thread Thread[CompactionExecutor:4028,1,main]
> java.lang.RuntimeException: Last written key DecoratedKey(425124616570337476,
> 0010000000001111000000000000033523da00001000000000033523da000000001111000000001000000000
> 00004000000000000000000100) >= current key DecoratedKey(-8778432288598355336,
> 0010000000001111000000000000040c7a8f00001000000000040c7a8f000000001111000000001000000000
> 00004000000000000000000100) writing into
> /cassandra-data/disk2/myks/mytable/myks-mytable-tmp-jb-130525-Data.db
> at
> org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:142)
> at
> org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:165)
> at
> org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:160)
> at
> org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60)
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
> at
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:198)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> It seems that scrubbing helps but scrubbing blocks the compactions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
