[
https://issues.apache.org/jira/browse/HBASE-9869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13823509#comment-13823509
]
Nicolas Liochon commented on HBASE-9869:
----------------------------------------
Yourkit calculates the retained size, i.e. ??Retained size of an object is its
shallow size plus the shallow sizes of the objects that are accessible,
directly or indirectly, only from this object. In other words, the retained
size represents the amount of memory that will be freed by the garbage
collector when this object is collected.??
With 10 thousand regions, the retained size of the two ConcurrentSkipListMap is
7 mega bytes.
With 100 thousands regions, the retained size is 75 mega bytes. 19 megs are
TableName objects, and this leads to an obvious optimization (I had it in mind
already, to save on 'equals' but the final size is crazy). On the same range,
we have 3.3 mb of ServerName.
Lastly, I don't think that a Map is the best algorithm, a Trie would be much
better. I will have a look at this as well.
With 100k regions, time is:
||#clients|#puts|time without the patch|time with the patch||
||2 clients| 50 million each| 83 seconds|58 seconds||
With these results my opinion is that we should commit this patch as it is,
because:
- 60 Mb is acceptable for a client connected to a cluster with 100K regions.
- In most cases, the weak reference will just make the performance
unpredictable. The remaining cases (regions not used often so we can remove
them under memory pressure) does not justify the noise for the other cases.
- We can lower the memory foot print further if necessary, and it's likely a
better solution than playing with the GC.
> Optimize HConnectionManager#getCachedLocation
> ---------------------------------------------
>
> Key: HBASE-9869
> URL: https://issues.apache.org/jira/browse/HBASE-9869
> Project: HBase
> Issue Type: Bug
> Components: Client
> Affects Versions: 0.98.0, 0.96.0
> Reporter: Nicolas Liochon
> Assignee: Nicolas Liochon
> Fix For: 0.98.0, 0.96.1
>
> Attachments: 9869.v1.patch, 9869.v1.patch, 9869.v2.patch
>
>
> It javadoc says: "TODO: This method during writing consumes 15% of CPU doing
> lookup". This is still true, says Yourkit. With 0.96, we also spend more time
> in these methods. We retry more, and the AsyncProcess calls it in parallel.
> I don't have the patch for this yet, but I will spend some time on it.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
