[
https://issues.apache.org/jira/browse/HBASE-20545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16467984#comment-16467984
]
Thiruvel Thirumoolan commented on HBASE-20545:
----------------------------------------------
Uploaded patch for 1.4, will also upload one for branch 2.x. The approach is
that - for most scenarios, the servers are the same when retainAssignment is
called. But we unnecessarily create Cluster object and populate it for every
region. At scale, this times time (array copies in Cluster.doAssignRegion) and
also lot of garbage. Since thats not common case, I moved out that to a
separate loop and only initialize them when random assignment is required. So
in the worst case, we still take that time, but for common scenarios, its much
faster and less memory garbage. We can make the worst case scenario also
faster, but thats later.
> Improve performance of BaseLoadBalancer.retainAssignment
> --------------------------------------------------------
>
> Key: HBASE-20545
> URL: https://issues.apache.org/jira/browse/HBASE-20545
> Project: HBase
> Issue Type: Improvement
> Components: Balancer
> Affects Versions: 1.4.4, 2.0.0
> Reporter: Thiruvel Thirumoolan
> Assignee: Thiruvel Thirumoolan
> Priority: Major
> Fix For: 1.5.0, 2.0.1, 1.4.5
>
> Attachments: HBASE-20545.branch-1.4.001.patch
>
>
> I was measuring perf at scale with a 1m region table and noticed some
> improvements can be made to BaseLoadBalancer.retainAssignment().
> retainAssignment() spends a few mins to enable a 1m regions table and also
> generates a lot of objects unnecessarily. This jira is to make the most
> common case go faster with very minimal changes. A slightly modified version
> of this patch takes about 5 seconds for a 1m region table ignoring the time
> spent in createCluster(). I think locality can be refreshed during master
> startup in different ways without taking time in retainAssignment, but will
> follow up on that in subsequent jiras. Leaving it untouched here, but wanted
> to call out the time taken without that method.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
