Dmitry Erokhin created CASSANDRA-13545:
------------------------------------------
Summary: Exception in CompactionExecutor leading to tmplink files
not being removed
Key: CASSANDRA-13545
URL: https://issues.apache.org/jira/browse/CASSANDRA-13545
Project: Cassandra
Issue Type: Bug
Components: Compaction
Reporter: Dmitry Erokhin
We are facing an issue where compactions fail on a few nodes with the following
message
{code}
ERROR [CompactionExecutor:1248] 2017-05-22 15:32:55,390
CassandraDaemon.java:185 - Exception in thread
Thread[CompactionExecutor:1248,1,main]
java.lang.AssertionError: null
at
org.apache.cassandra.io.sstable.IndexSummary.<init>(IndexSummary.java:86)
~[apache-cassandra-2.2.5.jar:2.2.5]
at
org.apache.cassandra.io.sstable.IndexSummaryBuilder.build(IndexSummaryBuilder.java:235)
~[apache-cassandra-2.2.5.jar:2.2.5]
at
org.apache.cassandra.io.sstable.format.big.BigTableWriter.openEarly(BigTableWriter.java:316)
~[apache-cassandra-2.2.5.jar:2.2.5]
at
org.apache.cassandra.io.sstable.SSTableRewriter.maybeReopenEarly(SSTableRewriter.java:170)
~[apache-cassandra-2.2.5.jar:2.2.5]
at
org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:115)
~[apache-cassandra-2.2.5.jar:2.2.5]
at
org.apache.cassandra.db.compaction.writers.DefaultCompactionWriter.append(DefaultCompactionWriter.java:64)
~[apache-cassandra-2.2.5.jar:2.2.5]
at
org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:184)
~[apache-cassandra-2.2.5.jar:2.2.5]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[apache-cassandra-2.2.5.jar:2.2.5]
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:74)
~[apache-cassandra-2.2.5.jar:2.2.5]
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
~[apache-cassandra-2.2.5.jar:2.2.5]
at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:256)
~[apache-cassandra-2.2.5.jar:2.2.5]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[na:1.8.0_121]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
~[na:1.8.0_121]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
~[na:1.8.0_121]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_121]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121]
{code}
Also, the number of tmplink files in /var/lib/cassandra/data/<keyspace
name>blocks/tmplink* is growing constantly until node runs out of space.
Restarting cassandra removes all tmplink files, but the issue still continues.
We are using Cassandra 2.2.5 on Debian 8 with Oracle Java 8
{code}
root@cassandra-p10:/var/lib/cassandra/data/mugenstorage/blocks-33167ef0447a11e68f3e5b42fc45b62f#
dpkg -l | grep -E "java|cassandra"
ii cassandra 2.2.5 all
distributed storage system for structured data
ii cassandra-tools 2.2.5 all
distributed storage system for structured data
ii java-common 0.52 all
Base of all Java packages
ii javascript-common 11 all
Base support for JavaScript library packages
ii oracle-java8-installer 8u121-1~webupd8~0 all
Oracle Java(TM) Development Kit (JDK) 8
ii oracle-java8-set-default 8u121-1~webupd8~0 all
Set Oracle JDK 8 as default Java
{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
