[
https://issues.apache.org/jira/browse/CASSANDRA-17298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17877296#comment-17877296
]
Branimir Lambov commented on CASSANDRA-17298:
---------------------------------------------
Thank you for the very detailed investigation.
This has been a source of annoyance, with many attempts to fix since the test
was first introduced, but it's necessary because before the test we had reached
about 2x difference between the memtable's understanding of its on-heap size
and what it actually used. One consideration we've previously had around this
is that adjusting the memory usage reporting may cause memtables to flush
earlier and change the behavior of existing clusters too much for a patch
release. In this case it looks like the difference is on the order of 10%,
which I personally would not see as a problem.
I wonder if it we shouldn't backport most of that 5.0 patch so that we start
testing all allocation strategies in 4.x as well. Also, am I understanding
correctly that there is also something (EMPTY_LEAF?) that we are not tracking
correctly in 5.0?
On the open question, there may be a reason to use both {{BTree.sizeOnHeapOf}}
and {{BTree.sizeOfStructureOnHeap}} (e.g. some BTree-building methods always
share the size map, others never, and if the caller knows which one it is it
could choose between the two). From a quick glance it looks like we use the
latter version for {{Columns}}, and these are built from sorted using shared
size maps, thus this appears to be the right thing to do. However, the names of
the two methods should reflect this difference and they it should also be
explained in javaDoc.
> Test Failure: org.apache.cassandra.cql3.MemtableSizeTest
> --------------------------------------------------------
>
> Key: CASSANDRA-17298
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17298
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
> Reporter: Josh McKenzie
> Assignee: Dmitry Konstantinov
> Priority: Normal
> Fix For: 4.0.x, 4.1.x
>
> Attachments: analyzed_objects.svg, structure_example.svg
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> [https://ci-cassandra.apache.org/job/Cassandra-4.0/313/testReport/org.apache.cassandra.cql3/MemtableSizeTest/testTruncationReleasesLogSpace_2/]
> Failed 4 times in the last 30 runs. Flakiness: 27%, Stability: 86%
> Error Message
> Expected heap usage close to 49.930MiB, got 41.542MiB.
> {code}
> Stacktrace
> junit.framework.AssertionFailedError: Expected heap usage close to 49.930MiB,
> got 41.542MiB.
> at
> org.apache.cassandra.cql3.MemtableSizeTest.testSize(MemtableSizeTest.java:130)
> at org.apache.cassandra.Util.runCatchingAssertionError(Util.java:644)
> at org.apache.cassandra.Util.flakyTest(Util.java:669)
> at
> org.apache.cassandra.cql3.MemtableSizeTest.testTruncationReleasesLogSpace(MemtableSizeTest.java:61)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
> *UPDATE:* It was discovered that unit tests were running with
> memtable_allocation_type: offheap_objects when we ship C* with heap_buffers.
> So we changed that in CASSANDRA-19326, now we test with
> memtable_allocation_type: heap_buffers. As a result, this test now fails all
> the time on 4.0 and 4.1.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
