[
https://issues.apache.org/jira/browse/HBASE-15950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar updated HBASE-15950:
----------------------------------
Attachment: hbase-15950-v0.patch
Here is an initial patch without JOL dependency. We use Unsafe to estimate the
header sizes and oop reference size similar to the way JOL does it. For the
changed ones, the estimates matches what JOL reports.
Apart from small cell size tests where it gives >50% improvement in available
memstore space, I've put this patch up in a YCSB cluster (single node).
Inserted 30M rows of 1K size, with and without the patch with 50 regions. The
number of flushes decreased from 610 to 500, and compactions from 64 to 41.
Overall %20 decrease in flush / compaction IO is pretty significant.
> Fix memstore size estimates to be more tighter
> ----------------------------------------------
>
> Key: HBASE-15950
> URL: https://issues.apache.org/jira/browse/HBASE-15950
> Project: HBase
> Issue Type: Bug
> Reporter: Enis Soztutar
> Assignee: Enis Soztutar
> Fix For: 2.0.0
>
> Attachments: Screen Shot 2016-06-02 at 8.48.27 PM.png,
> hbase-15950-v0.patch
>
>
> While testing something else, I was loading a region with a lot of data.
> Writing 30M cells in 1M rows, with 1 byte values.
> The memstore size turned out to be estimated as 4.5GB, while with the JFR
> profiling I can see that we are using 2.8GB for all the objects in the
> memstore (KV + KV byte[] + CSLM.Node + CSLM.Index).
> This obviously means that there is room in the write cache that we are not
> effectively using.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
