Nikolai Grigoriev created CASSANDRA-8191:
--------------------------------------------
Summary: All nodes log RuntimeException in CompactionExecutor
(Last written key DecoratedKey... >= current key DecoratedKey)
Key: CASSANDRA-8191
URL: https://issues.apache.org/jira/browse/CASSANDRA-8191
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: DSE 4.5.2 (Cassandra 2.0.10)
Reporter: Nikolai Grigoriev
While recovering the cluster from CASSANDRA-7949 (using the flag from
CASSANDRA-6621) I had to use sstablesplit tool to split large sstables. Nodes
were off while using this tool and only one sstablesplit instance was running,
of course.
After splitting was done I have restarted the nodes and they all started
compacting the data. All the nodes are logging the exceptions like this:
{code}
ERROR [CompactionExecutor:4028] 2014-10-26 23:14:52,653 CassandraDaemon.java
(line 199) Exception in thread Thread[CompactionExecutor:4028,1,main]
java.lang.RuntimeException: Last written key DecoratedKey(425124616570337476,
0010000000001111000000000000033523da00001000000000033523da000000001111000000001000000000
00004000000000000000000100) >= current key DecoratedKey(-8778432288598355336,
0010000000001111000000000000040c7a8f00001000000000040c7a8f000000001111000000001000000000
00004000000000000000000100) writing into
/cassandra-data/disk2/myks/mytable/myks-mytable-tmp-jb-130525-Data.db
at
org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:142)
at
org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:165)
at
org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:160)
at
org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60)
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:198)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
{code}
It seems that scrubbing helps but scrubbing blocks the compactions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
