Thanks, reported here https://issues.apache.org/jira/browse/CASSANDRA-20976
Regards From: Ivan Zalozhnykh <[email protected]> Date: Wednesday, 15 October 2025 at 14:07 To: [email protected] <[email protected]> Subject: AssertionError when reading BTI sstable in 5.0.5 during token range query You don't often get email from [email protected]. Learn why this is important<https://aka.ms/LearnAboutSenderIdentification> EXTERNAL EMAIL - USE CAUTION when clicking links or attachments Hello, We're experiencing a recurring issue in Cassandra 5.0.5 when performing token-range read queries on a table using the BTI sstable format. The issue results in a fatal `AssertionError` during read, which causes read failures in the client with `ReadFailureException`. The exception from Cassandra logs looks like this: ``` ERROR [ReadStage-6] 2025-08-27 19:22:49,331 JVMStabilityInspector.java:70 - Exception in thread Thread[ReadStage-6,5,SharedPool] java.lang.RuntimeException: java.lang.AssertionError: Caught an error while trying to process the command: SELECT * FROM keyspace.table WHERE token(profile_id) > -8987898918496238646 AND token(profile_id) <= -8718169305572166374 LIMIT 1000 ALLOW FILTERING at org.apache.cassandra.net.InboundSink.accept(InboundSink.java:108) at org.apache.cassandra.net.InboundSink.accept(InboundSink.java:45) at org.apache.cassandra.net.InboundMessageHandler$ProcessMessage.run(InboundMessageHandler.java:430) at org.apache.cassandra.concurrent.ExecutionFailure$1.run(ExecutionFailure.java:133) at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:143) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: java.lang.AssertionError: Caught an error while trying to process the command: SELECT * FROM keyspace.table WHERE token(profile_id) > -8987898918496238646 AND token(profile_id) <= -8718169305572166374 LIMIT 1000 ALLOW FILTERING at org.apache.cassandra.db.ReadCommandVerbHandler.doVerb(ReadCommandVerbHandler.java:112) at org.apache.cassandra.net.InboundSink.lambda$new$0(InboundSink.java:78) at org.apache.cassandra.net.InboundSink.accept(InboundSink.java:97) ... 6 common frames omitted Caused by: java.lang.AssertionError: 406005 > 393216 at org.apache.cassandra.io.util.MmappedRegions$State.floor(MmappedRegions.java:362) at org.apache.cassandra.io.util.MmappedRegions.floor(MmappedRegions.java:241) at org.apache.cassandra.io.util.MmapRebufferer.rebuffer(MmapRebufferer.java:40) at org.apache.cassandra.io.tries.Walker.<init>(Walker.java:75) at org.apache.cassandra.io.tries.ValueIterator.<init>(ValueIterator.java:96) at org.apache.cassandra.io.tries.ValueIterator.<init>(ValueIterator.java:80) at org.apache.cassandra.io.sstable.format.bti.PartitionIndex$IndexPosIterator.<init>(PartitionIndex.java:407) at org.apache.cassandra.io.sstable.format.bti.PartitionIterator.<init>(PartitionIterator.java:113) at org.apache.cassandra.io.sstable.format.bti.PartitionIterator.create(PartitionIterator.java:75) at org.apache.cassandra.io.sstable.format.bti.BtiTableReader.coveredKeysIterator(BtiTableReader.java:293) at org.apache.cassandra.io.sstable.format.bti.BtiTableScanner$BtiScanningIterator.prepareToIterateRow(BtiTableScanner.java:93) at org.apache.cassandra.io.sstable.format.SSTableScanner$BaseKeyScanningIterator.computeNext(SSTableScanner.java:248) at org.apache.cassandra.io.sstable.format.SSTableScanner$BaseKeyScanningIterator.computeNext(SSTableScanner.java:228) at org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) at org.apache.cassandra.io.sstable.format.SSTableScanner.hasNext(SSTableScanner.java:190) at org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:90) at org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:375) at org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:187) at org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:156) at org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) at org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$4.hasNext(UnfilteredPartitionIterators.java:264) at org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:90) at org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:334) at org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:201) at org.apache.cassandra.db.ReadResponse$LocalDataResponse.<init>(ReadResponse.java:186) at org.apache.cassandra.db.ReadResponse.createDataResponse(ReadResponse.java:48) at org.apache.cassandra.db.ReadCommand.createResponse(ReadCommand.java:374) at org.apache.cassandra.db.ReadCommandVerbHandler.doVerb(ReadCommandVerbHandler.java:93) ... 8 common frames omitted ``` The query itself is a standard paginated scan over token ranges with a LIMIT. Here's a simplified version of the code performing it: ```kotlin @Query("select profile_id, token(profile_id) as page_token from keyspace.table where token(profile_id) > :pageToken limit :limitCount") fun selectWithToken( pageToken: Long, limitCount: Int, ): MappedReactiveResultSet<MigratorEntityToken> ``` This results in read failures like this on the client side: ``` Caused by: com.datastax.oss.driver.api.core.servererrors.ReadFailureException: Cassandra failure during read query at consistency LOCAL_QUORUM (2 responses were required but only 0 replica responded, 1 failed) ``` The issue occurs with multiple SSTables, across multiple token ranges, on different nodes. All the affected Trie-indexed SSTables, and the error disappears when switching the sstable format back to Big. Has anyone else seen similar behavior with BTI sstables in Cassandra 5.0.x? Is this a known issue or something we should file a Jira ticket for? Thanks in advance, Ivan Zalozhnykh
