[
https://issues.apache.org/jira/browse/HBASE-21355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zheng Hu updated HBASE-21355:
-----------------------------
Description:
When testing the branch-2's write performance in our internal cluster, we
found that the region will be inexplicably split.
We use the default ConstantSizeRegionSplitPolicy and
hbase.hregion.max.filesize=40G，but the region will be split even if its bytes
size is less than 40G(only ~6G).
Checked the code, I found that the following path will accumulate the store's
storeSize to a very big value, because the path has no reset..
{code}
RsRpcServices#getRegionInfo
-> HRegion#isMergeable
-> HRegion#hasReferences
-> HStore#hasReferences
-> HStore#openStoreFiles
{code}
BTW, we seems forget to maintain the read replica's storeSize when refresh the
store files.
Some comment here, I move the storeSize calculation out of loadStoreFiles()
method, because the secondary read replica's refreshStoreFiles() will also use
loadStoreFiles() to refresh its store files and update the storeSize in the
completeCompaction(..) in the final (just like compaction.) , so no need
calculate the storeSize twice..
was:
When testing the branch-2's write performance in our internal cluster, we
found that the region will be inexplicably split.
We use the default ConstantSizeRegionSplitPolicy and
hbase.hregion.max.filesize=40G，but the region will be split even if its bytes
size is less than 40G(only ~6G).
Checked the code, I found that the following path will accumulate the store's
storeSize to a very big value, because the path has no reset..
{code}
RsRpcServices#getRegionInfo
-> HRegion#isMergeable
-> HRegion#hasReferences
-> HStore#hasReferences
-> HStore#openStoreFiles
{code}
BTW, we seems forget to maintain the read replica's storeSize when refresh the
store files.
> HStore's storeSize is calculated repeatedly which causing the confusing
> region split
> -------------------------------------------------------------------------------------
>
> Key: HBASE-21355
> URL: https://issues.apache.org/jira/browse/HBASE-21355
> Project: HBase
> Issue Type: Bug
> Components: regionserver
> Reporter: Zheng Hu
> Assignee: Zheng Hu
> Priority: Blocker
> Fix For: 3.0.0, 1.5.0, 1.3.3, 2.2.0, 2.1.1, 2.0.3, 1.4.9, 1.2.9
>
> Attachments: HBASE-21355.addendum.patch, HBASE-21355.branch-1.patch,
> HBASE-21355.v1.patch
>
>
> When testing the branch-2's write performance in our internal cluster, we
> found that the region will be inexplicably split.
> We use the default ConstantSizeRegionSplitPolicy and
> hbase.hregion.max.filesize=40G，but the region will be split even if its
> bytes size is less than 40G(only ~6G).
> Checked the code, I found that the following path will accumulate the
> store's storeSize to a very big value, because the path has no reset..
> {code}
> RsRpcServices#getRegionInfo
> -> HRegion#isMergeable
> -> HRegion#hasReferences
> -> HStore#hasReferences
> -> HStore#openStoreFiles
> {code}
> BTW, we seems forget to maintain the read replica's storeSize when refresh
> the store files.
> Some comment here, I move the storeSize calculation out of loadStoreFiles()
> method, because the secondary read replica's refreshStoreFiles() will also
> use loadStoreFiles() to refresh its store files and update the storeSize in
> the completeCompaction(..) in the final (just like compaction.) , so no need
> calculate the storeSize twice..
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
