[
https://issues.apache.org/jira/browse/HDFS-9260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14965242#comment-14965242
]
Staffan Friberg commented on HDFS-9260:
---------------------------------------
The TreeSet reduces the number of reference compared to the Double-LinkedList
currently built using the the triplets datastructure.
If we would move things out of the heap the TreeSet, if still used, would
contain memory addresses rather than longs which are trivial for the GC to
handle (no need to scan the array). Potentially the BlockMap could be the same
way a large long array on heap that contains memory addresses of the blockinfos
that are off heap, and collisions could be handled by the blockinfo's off heap
(linking in the same way they are now).
> Improve performance and GC friendliness of startup and FBRs
> -----------------------------------------------------------
>
> Key: HDFS-9260
> URL: https://issues.apache.org/jira/browse/HDFS-9260
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode, namenode, performance
> Affects Versions: 2.7.1
> Reporter: Staffan Friberg
> Assignee: Staffan Friberg
> Attachments: HDFS Block and Replica Management 20151013.pdf,
> HDFS-7435.001.patch, HDFS-7435.002.patch, HDFS-7435.003.patch
>
>
> This patch changes the datastructures used for BlockInfos and Replicas to
> keep them sorted. This allows faster and more GC friendly handling of full
> block reports.
> Would like to hear peoples feedback on this change and also some help
> investigating/understanding a few outstanding issues if we are interested in
> moving forward with this.
> There seems to be some timing issues I hit when testing the patch, not sure
> if it is a bug in the patch or something else (most likely the earlier)...
> Tests that fail for me:
> The issues seems to be that the blocks are not on any storage, so no
> replication can occur causing the tests to fail in different ways.
> TestDecomission.testDecommision
> If I add a little sleep after the cleanup/delete things seem to work
> TestDFSStripedOutputStreamWithFailure
> A couple of tests fails in this class.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
