[
https://issues.apache.org/jira/browse/HBASE-17515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15840097#comment-15840097
]
stack commented on HBASE-17515:
-------------------------------
I try to follow your reasoning:
+ You leave out class overhead. And aren't you supposed to multiple by 15, the
number of regions to keep around? (Thats what I did). Regardless, you don't
answer the question which is is 24MB (or 24MB * 15) too much data to keep
around for sake of a single metric?
+ You make some soft argument about 'good practice' cutting down memory in
general but no work to show how this here is even a problem area in master
memory management.
+ I ask if rather than do our own accounting, we should not defer to package
and you turn the question around to have me do the work (Since you ask, Data
Sketches, a stochastic streaming algorithms library looks promising especially
as we need to start using it throughout)
+ You then pervert my general shaving of RL size question into a query about
the four fields extracted here.
Please provide our gracious contributors with better quality input/direction.
> Reduce memory footprint of RegionLoads kept by StochasticLoadBalancer
> ---------------------------------------------------------------------
>
> Key: HBASE-17515
> URL: https://issues.apache.org/jira/browse/HBASE-17515
> Project: HBase
> Issue Type: Improvement
> Reporter: Ted Yu
> Assignee: Tim Brown
> Fix For: 2.0.0, 1.4.0
>
> Attachments: 17515.branch-1.v2.txt, 17515.v1.txt, 17515.v2.txt
>
>
> Currently StochasticLoadBalancer uses the following fields of RegionLoad :
> {code}
> return rl.getReadRequestsCount();
> return rl.getWriteRequestsCount();
> return rl.getMemStoreSizeMB();
> return rl.getStorefileSizeMB();
> {code}
> However, RegionLoad refers to ClusterStatusProtos.RegionLoad which has 18
> fields.
> This means we keep hbase.master.balancer.stochastic.numRegionLoadsToRemember
> (default value 15) RegionLoad's in memory but only use a small portion of
> them.
> This JIRA is to consider creating POJO which wraps the above 4 fields so that
> the memory footprint can be lowered.
> Lowering memory footprint would allow wider sliding window to be configured
> for load balancer while limiting the impact to heap consumption.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
