[
https://issues.apache.org/jira/browse/CASSANDRA-20396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17932897#comment-17932897
]
Dmitry Konstantinov commented on CASSANDRA-20396:
-------------------------------------------------
There is one more issue which I noticed in the test. It does not cause the test
failure and it is observed even before CASSANDRA-20092 changes. We have a pool
buffer leak reported:
{code}
ERROR [Reference-Reaper] 2025-03-04 12:47:00,906 LEAK DETECTED: a reference
(@2027340356) to @2027340356 was not released before the reference was garbage
collected
ERROR [Reference-Reaper] 2025-03-04 12:47:00,907 Allocate trace @2027340356:
Thread[main,5,main]
at java.base/java.lang.Thread.getStackTrace(Thread.java:1602)
at org.apache.cassandra.utils.concurrent.Ref$Debug.<init>(Ref.java:273)
at org.apache.cassandra.utils.concurrent.Ref$State.<init>(Ref.java:194)
at org.apache.cassandra.utils.concurrent.Ref.<init>(Ref.java:116)
at
org.apache.cassandra.utils.concurrent.Ref$DirectBufferRef.<init>(Ref.java:810)
at
org.apache.cassandra.utils.memory.BufferPool$Chunk.setAttachment(BufferPool.java:1334)
at
org.apache.cassandra.utils.memory.BufferPool$Chunk.set(BufferPool.java:1474)
at
org.apache.cassandra.utils.memory.BufferPool$Chunk.get(BufferPool.java:1464)
at
org.apache.cassandra.utils.memory.BufferPool$MicroQueueOfChunks.get(BufferPool.java:635)
at
org.apache.cassandra.utils.memory.BufferPool$LocalPool.tryGetInternal(BufferPool.java:963)
at
org.apache.cassandra.utils.memory.BufferPool$LocalPool.tryGet(BufferPool.java:946)
at
org.apache.cassandra.utils.memory.BufferPool$LocalPool.get(BufferPool.java:906)
at
org.apache.cassandra.utils.memory.BufferPool$LocalPool.get(BufferPool.java:896)
at org.apache.cassandra.utils.memory.BufferPool.get(BufferPool.java:211)
at org.apache.cassandra.cache.ChunkCache.load(ChunkCache.java:162)
at org.apache.cassandra.cache.ChunkCache.load(ChunkCache.java:46)
at
com.github.benmanes.caffeine.cache.LocalLoadingCache.lambda$newMappingFunction$3(LocalLoadingCache.java:183)
at
com.github.benmanes.caffeine.cache.LocalCache.lambda$statsAware$2(LocalCache.java:167)
at
com.github.benmanes.caffeine.cache.BoundedLocalCache.lambda$doComputeIfAbsent$14(BoundedLocalCache.java:2688)
at
java.base/java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1908)
at
com.github.benmanes.caffeine.cache.BoundedLocalCache.doComputeIfAbsent(BoundedLocalCache.java:2686)
at
com.github.benmanes.caffeine.cache.BoundedLocalCache.computeIfAbsent(BoundedLocalCache.java:2669)
at
com.github.benmanes.caffeine.cache.LocalCache.computeIfAbsent(LocalCache.java:112)
at
com.github.benmanes.caffeine.cache.LocalLoadingCache.get(LocalLoadingCache.java:58)
at
org.apache.cassandra.cache.ChunkCache$CachingRebufferer.rebuffer(ChunkCache.java:232)
at
org.apache.cassandra.cache.ChunkCache$CachingRebufferer.rebuffer(ChunkCache.java:211)
at
org.apache.cassandra.io.util.RandomAccessReader.reBufferAt(RandomAccessReader.java:67)
at
org.apache.cassandra.io.util.RandomAccessReader.seek(RandomAccessReader.java:209)
at
org.apache.cassandra.io.sstable.format.SSTableSimpleScanner.advanceRange(SSTableSimpleScanner.java:162)
at
org.apache.cassandra.io.sstable.format.SSTableSimpleScanner.hasNext(SSTableSimpleScanner.java:146)
at
org.apache.cassandra.db.compaction.CorruptedSSTablesCompactionsTest.readsWithoutError(CorruptedSSTablesCompactionsTest.java:280)
at
org.apache.cassandra.db.compaction.CorruptedSSTablesCompactionsTest.testCorruptedSSTables(CorruptedSSTablesCompactionsTest.java:241)
at
org.apache.cassandra.db.compaction.CorruptedSSTablesCompactionsTest.testCorruptedSSTablesWithLeveledCompactionStrategy(CorruptedSSTablesCompactionsTest.java:152)
{code}
In org.apache.cassandra.cache.ChunkCache#load if we have an exception thrown by
readChunk method the buffer is leaking:
{code}
ByteBuffer buffer = bufferPool.get(key.file.chunkSize(),
key.file.preferredBufferType());
key.file.readChunk(key.position, buffer);
{code}
I will add handling of this case to the fix as well.
> Test failure: CorruptedSSTablesCompactionsTest
> ----------------------------------------------
>
> Key: CASSANDRA-20396
> URL: https://issues.apache.org/jira/browse/CASSANDRA-20396
> Project: Apache Cassandra
> Issue Type: Bug
> Components: Local/SSTable
> Reporter: Dmitry Konstantinov
> Assignee: Dmitry Konstantinov
> Priority: Normal
> Fix For: 5.x
>
> Attachments:
> TEST-org.apache.cassandra.db.compaction.CorruptedSSTablesCompactionsTest.log.xz,
>
> TEST-org.apache.cassandra.db.compaction.CorruptedSSTablesCompactionsTest.xml.xz,
> image-2025-03-04-21-46-34-829.png
>
>
> CorruptedSSTablesCompactionsTest is failing for trunk on CI:
> * [https://ci-cassandra.apache.org/job/Cassandra-trunk/2049/]
> * [https://ci-cassandra.apache.org/job/Cassandra-trunk/2044/]
> *
> [https://ci-cassandra.apache.org/job/Cassandra-trunk/2043/|https://ci-cassandra.apache.org/job/Cassandra-trunk/2043/#showFailuresLink]
> with errors like:
> {code:java}
> junit.framework.AssertionFailedError: expected:<8> but was:<25>
> at
> org.apache.cassandra.db.compaction.CorruptedSSTablesCompactionsTest.testCorruptedSSTables(CorruptedSSTablesCompactionsTest.java:267)
> at
> org.apache.cassandra.db.compaction.CorruptedSSTablesCompactionsTest.testCorruptedSSTablesWithSizeTieredCompactionStrategy(CorruptedSSTablesCompactionsTest.java:146)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
> note: the failure is observing for test-compression suite
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
