[
https://issues.apache.org/jira/browse/HBASE-15249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15149247#comment-15149247
]
Siddharth Wagle commented on HBASE-15249:
-----------------------------------------
[~anoop.hbase] AMS (Ambari Metrics system), creates tables with pre-splits
based on the knowledge of how many daemons will be writing metrics to HBase and
the memory available to RS.
What we saw is that on a large cluster we defined close to 10 initial pre-split
Regions for the table that gets heavy Read/Write load and this count dropped
shortly after the system came online. This was during Ambari performance test
runs.
*Note*: The metrics system is a constant write load system, however there is a
bootstrap lag as the cluster comes online. This is the grey area where although
Regions are not getting as many writes, merging them would be a bad idea. We
enable normalizer specifically because the intial splits might not be optimal,
however, the Region count is certainly critical for us to support the volume of
writes that would eventually settle downs to a consistent number.
We will try to get the numbers for you, since normalizer has DEBUG as the log
level, we could not capture this on our intial run.
> Provide lower bound on number of regions in region normalizer for pre-split
> tables
> ----------------------------------------------------------------------------------
>
> Key: HBASE-15249
> URL: https://issues.apache.org/jira/browse/HBASE-15249
> Project: HBase
> Issue Type: Bug
> Reporter: Ted Yu
> Assignee: Ted Yu
> Attachments: HBASE-15249.v1.txt, HBASE-15249.v2.txt
>
>
> AMS (Ambari Metrics System) developer found the following scenario:
> Metrics table was pre-split with many regions on large cluster (1600 nodes).
> After some time, AMS stopped working because region normalizer merged the
> regions into few big regions which were not able to serve high read / write
> load.
> This is a big problem since the write requests flood the regions faster than
> the splits can happen resulting in poor performance.
> We should consider setting reasonable lower bound on region count.
> If the table is pre-split, we can use initial region count as the lower bound.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
