[
https://issues.apache.org/jira/browse/CASSANDRA-21390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18082574#comment-18082574
]
MM commented on CASSANDRA-21390:
--------------------------------
Here is what we see in logs. Flush is getting triggered on cass_repair.cluster
table.
2026-05-20 04:29:10,918 [INFO ] [NativePoolCleaner] cluster_id=199
ip_address=10.49.xxx.xx AbstractAllocatorMemtable.java:293 - Flushing largest
CFS(Keyspace='caas_repair', ColumnFamily='cluster') to free up room. Used
total: 0.08/0.96, live: 0.08/0.20, flushing: -0.00/0.76, this: -0.00/0.07
2026-05-20 04:29:10,918 [INFO ] [NativePoolCleaner] cluster_id=199
ip_address=10.49.xxx.xx ColumnFamilyStore.java:1052 - Enqueuing flush of
caas_repair.cluster, Reason: MEMTABLE_LIMIT, Usage: -107.559KiB (-0%) on-heap,
268.210MiB (7%) off-heap
2026-05-20 04:29:10,932 [INFO ] [PerDiskMemtableFlushWriter_0:2351]
cluster_id=199 ip_address=10.49.xxx.xx Flushing.java:153 - Writing
Memtable-cluster@858189062(3.725KiB serialized bytes, 135167 ops, -107.559KiB
(-0%) on-heap, 268.210MiB (7%) off-heap), flushed range =
[min(-9223372036854775808), max(9223372036854775807))
2026-05-20 04:29:10,932 [INFO ] [PerDiskMemtableFlushWriter_0:2351]
cluster_id=199 ip_address=10.49.xxx.xx Flushing.java:179 - Completed flushing
/opt/nosql/data/cluster_199/data/caas_repair/cluster-dcae655014bb11ed8414b52584c330bb/da-27090-bti-Data.db
(1.806KiB) for commitlog position CommitLogPosition(segmentId=1779020363209,
position=24920473)
2026-05-20 04:29:10,966 [ERROR] [MemtableReclaimMemory:1] cluster_id=199
ip_address=10.49.xxx.xx JVMStabilityInspector.java:70 - Exception in thread
Thread[MemtableReclaimMemory:1,5,MemtableReclaimMemory]
> TrieMemtable MemtableReclaimMemory AssertionError: Negative released in
> MemtablePool$SubPool
> --------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-21390
> URL: https://issues.apache.org/jira/browse/CASSANDRA-21390
> Project: Apache Cassandra
> Issue Type: Bug
> Components: Local/Memtable
> Reporter: Praveen Reddy Arra
> Priority: Normal
>
> We have started seeing this fatal exception in Apache Cassandra 5.0.6 on one
> of our clusters.
> {code:java}
> [ERROR] [MemtableReclaimMemory:1] cluster_id=xxx ip_address=xxx.xxx.xxx.xxx
> JVMStabilityInspector.java:70 - Exception in thread
> Thread[MemtableReclaimMemory:1,5,MemtableReclaimMemory]
> java.lang.AssertionError: Negative released: -4332
> at
> org.apache.cassandra.utils.memory.MemtablePool$SubPool.released(MemtablePool.java:194)
> at
> org.apache.cassandra.utils.memory.MemtableAllocator$SubAllocator.releaseAll(MemtableAllocator.java:153)
> at
> org.apache.cassandra.utils.memory.MemtableAllocator$SubAllocator.setDiscarded(MemtableAllocator.java:144)
> at
> org.apache.cassandra.utils.memory.MemtableAllocator.setDiscarded(MemtableAllocator.java:95)
> at
> org.apache.cassandra.utils.memory.NativeAllocator.setDiscarded(NativeAllocator.java:205)
> at
> org.apache.cassandra.db.memtable.AbstractAllocatorMemtable.discard(AbstractAllocatorMemtable.java:171)
> at
> org.apache.cassandra.db.memtable.TrieMemtable.discard(TrieMemtable.java:163)
> at
> org.apache.cassandra.db.ColumnFamilyStore$Flush$1.runMayThrow(ColumnFamilyStore.java:1398)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:26)
> at
> org.apache.cassandra.concurrent.ExecutionFailure$1.run(ExecutionFailure.java:133)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:842)
> {code}
> {code:yaml}
> memtable_allocation_type - heap_buffers
> file_cache_enabled: true
> file_cache_size: 2048MiB
> memtable:
> configurations:
> skiplist:
> class_name: SkipListMemtable
> trie:
> class_name: TrieMemtable
> default:
> inherits: trie
> {code}
> it looks similar to the open MemtableReclaimMemory assert issue in
> [CASSANDRA-18159|https://issues.apache.org/jira/browse/CASSANDRA-18159]
> Environment is RHEL 8.10 with OpenJDK 17 and 16GB heap, -ea enabled.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
