[
https://issues.apache.org/jira/browse/CASSANDRA-16318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17266644#comment-17266644
]
Ekaterina Dimitrova commented on CASSANDRA-16318:
-------------------------------------------------
Thank you [~blerer] and [~barnie] for your valuable input an feedback and thank
you [~barnie] for adding an additional test, I believe now we are as close as
possible to the reality. Unfortunately, [ColumnMetadata Is
unmetered|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/schema/ColumnMetadata.java#L39]
The final(I hope?) version of the patch is available [here
|https://github.com/ekaterinadimitrova2/cassandra/pull/80/commits/4f47d2bbd8cf5cca92cb147de09a4fd8ccb3be47]
[Jenkins run|https://ci-cassandra.apache.org/job/Cassandra-devbranch/298/]
shows one failure which seems as a not-related typo.
CircleCI shows no new failures.
Java 8: There are a few in-jvm tests failures which are already investigated
in another ticket and one which failed because of lack of resources.
[Java
8|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/586/workflows/41888c0c-54ba-4db3-860f-05e5a6aee884]
[Java
11|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/586/workflows/9f901b57-67d0-46d1-9392-8f59341474fc]
> Memtable heap size is severely underestimated
> ---------------------------------------------
>
> Key: CASSANDRA-16318
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16318
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Memtable
> Reporter: Branimir Lambov
> Assignee: Ekaterina Dimitrova
> Priority: Normal
> Fix For: 4.0-beta
>
> Attachments: image-2020-12-09-10-57-21-994.png,
> image-2020-12-09-11-01-31-273.png
>
>
> We seem to be estimating the size of the on-heap memtable metadata to be
> around half of what it actually is. For example, during a [read benchmark
> which writes 1 million single-long
> rows|https://github.com/blambov/cassandra/blob/memtable-heap/test/microbench/org/apache/cassandra/test/microbench/instance/ReadTestSmallPartitions.java]
> the memtable reports
> {code}
> 1000000 ops, 58.174MiB serialized bytes, 385.284MiB (19%) on heap, 0.000KiB
> (0%) off-heap
> {code}
> while a heap dump taken at this point:
> !image-2020-12-09-10-57-21-994.png!
> lists an usage of about 666MB altogether.
> Switching to {{offheap_objects}}, the reported numbers are
> {code}
> 1000000 ops, 58.174MiB serialized bytes, 233.650MiB (12%) on heap, 53.406MiB
> (3%) off-heap
> {code}
> while actual heap usage:
> !image-2020-12-09-11-01-31-273.png!
> is about 442MB.
> Looking at the code we definitely are not counting the
> {{AtomicBTreePartition.Holder}}, {{EncodingStats}}, liveness and deletion
> info objects associated with each partition, and most probably others.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
