Roman S. Borschel created CASSANDRA-12203:
---------------------------------------------
Summary: AssertionError on compaction after upgrade (2.1.9 -> 3.7)
Key: CASSANDRA-12203
URL: https://issues.apache.org/jira/browse/CASSANDRA-12203
Project: Cassandra
Issue Type: Bug
Components: Compaction
Environment: Cassandra 3.7 (upgrade from 2.1.9)
Java version "1.8.0_91"
Ubuntu 14.04.4 LTS (GNU/Linux 3.13.0-83-generic x86_64)
Reporter: Roman S. Borschel
After upgrading a Cassandra cluster from 2.1.9 to 3.7, one column family (using
SizeTieredCompaction) repeatedly and continuously fails compaction (and thus
also repair) across the cluster, with all nodes producing the following errors
in the logs:
{noformat}
016-07-14T09:29:47.96855 |srv=cassandra|ERROR: Exception in thread
Thread[CompactionExecutor:3,1,main]
2016-07-14T09:29:47.96858 |srv=cassandra|java.lang.AssertionError: null
2016-07-14T09:29:47.96859 |srv=cassandra| at
org.apache.cassandra.db.UnfilteredDeserializer$OldFormatDeserializer$TombstoneTracker.openNew(UnfilteredDeserializer.java:650)
~[apache-cassandra-3.7.jar:3.7]
2016-07-14T09:29:47.96860 |srv=cassandra| at
org.apache.cassandra.db.UnfilteredDeserializer$OldFormatDeserializer$UnfilteredIterator.hasNext(UnfilteredDeserializer.java:423)
~[apache-cassandra-3.7.jar:3.7]
2016-07-14T09:29:47.96860 |srv=cassandra| at
org.apache.cassandra.db.UnfilteredDeserializer$OldFormatDeserializer.hasNext(UnfilteredDeserializer.java:298)
~[apache-cassandra-3.7.jar:3.7]
2016-07-14T09:29:47.96860 |srv=cassandra| at
org.apache.cassandra.io.sstable.SSTableSimpleIterator$OldFormatIterator.readStaticRow(SSTableSimpleIterator.java:133)
~[apache-cassandra-3.7.jar:3.7]
2016-07-14T09:29:47.96861 |srv=cassandra| at
org.apache.cassandra.io.sstable.SSTableIdentityIterator.<init>(SSTableIdentityIterator.java:57)
~[apache-cassandra-3.7.jar:3.7]
2016-07-14T09:29:47.96861 |srv=cassandra| at
org.apache.cassandra.io.sstable.format.big.BigTableScanner$KeyScanningIterator$1.initializeIterator(BigTableScanner.java:334)
~[apache-cassandra-3.7.jar:3.7]
2016-07-14T09:29:47.96862 |srv=cassandra| at
org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.maybeInit(LazilyInitializedUnfilteredRowIterator.java:48)
~[apache-cassandra-3.7.jar:3.7]
2016-07-14T09:29:47.96862 |srv=cassandra| at
org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.isReverseOrder(LazilyInitializedUnfilteredRowIterator.java:70)
~[apache-cassandra-3.7.jar:3.7]
2016-07-14T09:29:47.96863 |srv=cassandra| at
org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$1.reduce(UnfilteredPartitionIterators.java:109)
~[apache-cassandra-3.7.jar:3.7]
2016-07-14T09:29:47.96863 |srv=cassandra| at
org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$1.reduce(UnfilteredPartitionIterators.java:100)
~[apache-cassandra-3.7.jar:3.7]
2016-07-14T09:29:47.96864 |srv=cassandra| at
org.apache.cassandra.utils.MergeIterator$Candidate.consume(MergeIterator.java:408)
~[apache-cassandra-3.7.jar:3.7]
2016-07-14T09:29:47.96864 |srv=cassandra| at
org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:203)
~[apache-cassandra-3.7.jar:3.7]
2016-07-14T09:29:47.96865 |srv=cassandra| at
org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:156)
~[apache-cassandra-3.7.jar:3.7]
2016-07-14T09:29:47.96865 |srv=cassandra| at
org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
~[apache-cassandra-3.7.jar:3.7]
2016-07-14T09:29:47.96866 |srv=cassandra| at
org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$2.hasNext(UnfilteredPartitionIterators.java:150)
~[apache-cassandra-3.7.jar:3.7]
2016-07-14T09:29:47.96866 |srv=cassandra| at
org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:72)
~[apache-cassandra-3.7.jar:3.7]
2016-07-14T09:29:47.96867 |srv=cassandra| at
org.apache.cassandra.db.compaction.CompactionIterator.hasNext(CompactionIterator.java:226)
~[apache-cassandra-3.7.jar:3.7]
2016-07-14T09:29:47.96867 |srv=cassandra| at
org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:182)
~[apache-cassandra-3.7.jar:3.7]
2016-07-14T09:29:47.96867 |srv=cassandra| at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[apache-cassandra-3.7.jar:3.7]
2016-07-14T09:29:47.96868 |srv=cassandra| at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:82)
~[apache-cassandra-3.7.jar:3.7]
2016-07-14T09:29:47.96868 |srv=cassandra| at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:60)
~[apache-cassandra-3.7.jar:3.7]
2016-07-14T09:29:47.96869 |srv=cassandra| at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:264)
~[apache-cassandra-3.7.jar:3.7]
2016-07-14T09:29:47.96870 |srv=cassandra| at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[na:1.8.0_91]
2016-07-14T09:29:47.96870 |srv=cassandra| at
java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_91]
2016-07-14T09:29:47.96870 |srv=cassandra| at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
~[na:1.8.0_91]
2016-07-14T09:29:47.96871 |srv=cassandra| at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_91]
2016-07-14T09:29:47.96871 |srv=cassandra| at
java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]
{noformat}
Only 1 out of 14 tables was affected by this problem after the upgrade.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
