[
https://issues.apache.org/jira/browse/HDFS-8862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700352#comment-14700352
]
Colin Patrick McCabe commented on HDFS-8862:
--------------------------------------------
I guess the set of datanodes is not going to shrink that much over the life of
the cluster. So the fact that this data structure can't shrink should be OK.
We may want to look into whether that {{LightWeightLinkedSet<BlockInfo>}} can
shrink... but that is outside the scope of this JIRA. +1.
> Improve BlockManager#excessReplicateMap
> ---------------------------------------
>
> Key: HDFS-8862
> URL: https://issues.apache.org/jira/browse/HDFS-8862
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: namenode
> Reporter: Yi Liu
> Assignee: Yi Liu
> Attachments: HDFS-8862.001.patch
>
>
> Per [~cmccabe]'s comments in HDFS-8792, this JIRA is to discuss improving
> {{BlockManager#excessReplicateMap}}.
> That's right HashMap don't ever shrink when elements are removed, but
> TreeMap entry needs to store more (memory) references (left, right, parent)
> than HashMap entry (only one reference next), even when there is element
> removing and cause some entry empty, the empty HashMap entry is just a
> {{null}} reference (4 bytes), so they are close at this point. On the other
> hand, the key of {{excessReplicateMap}} is datanode uuid, so the entries
> number is almost fixed, so HashMap memory is good than TreeMap memory in this
> case. I think the most important is the search/insert/remove performance,
> HashMap is absolutely better than TreeMap. Because we don't need to sort,
> we should use HashMap instead of TreeMap
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
