[
https://issues.apache.org/jira/browse/HBASE-15950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15315213#comment-15315213
]
Andrew Purtell edited comment on HBASE-15950 at 6/4/16 1:14 AM:
----------------------------------------------------------------
bq. Found a very nice library called Java-Object-Layout from OpenJDK. I think
we should switch to using this to estimate the object sizes at runtime.
Nope. JOL is GPL.
What we can do is periodically run it ourselves to get object sizes for JVM
vendor X version Y, update the constants we use to ballpark, and we can vary X
and Y to come up with different sets of constants we can switch to given some
runtime detection.
was (Author: apurtell):
bq. Found a very nice library called Java-Object-Layout from OpenJDK. I think
we should switch to using this to estimate the object sizes at runtime.
Nope. JOL is GPL.
What we can do is periodically run it ourselves to get the estimates for
updating the constants we use to ballpark, and perhaps we can experiment with
several different JVMs to come up with different sets of constants we can
switch to given some runtime detection.
> We are grossly overestimating the memstore size
> -----------------------------------------------
>
> Key: HBASE-15950
> URL: https://issues.apache.org/jira/browse/HBASE-15950
> Project: HBase
> Issue Type: Bug
> Reporter: Enis Soztutar
> Assignee: Enis Soztutar
> Fix For: 2.0.0
>
> Attachments: Screen Shot 2016-06-02 at 8.48.27 PM.png
>
>
> While testing something else, I was loading a region with a lot of data.
> Writing 30M cells in 1M rows, with 1 byte values.
> The memstore size turned out to be estimated as 4.5GB, while with the JFR
> profiling I can see that we are using 2.8GB for all the objects in the
> memstore (KV + KV byte[] + CSLM.Node + CSLM.Index).
> This obviously means that there is room in the write cache that we are not
> effectively using.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
