[
https://issues.apache.org/jira/browse/HDFS-9260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14963055#comment-14963055
]
Walter Su commented on HDFS-9260:
---------------------------------
Excellent work [~sfriberg]!
1. Per your tests, the result looks good. But I'm more interested how does the
new datastructure incorporate with HDFS-7836.
2. To avoid moving entries in {{DatanodeStorageInfo}}, an easy way is to make
blockReport a serializable HashMap. Instead of iterating blockReport, we
iterate {{DatanodeStorageInfo}} with O(1) comparing block from blockReport.
> Improve performance and GC friendliness of startup and FBRs
> -----------------------------------------------------------
>
> Key: HDFS-9260
> URL: https://issues.apache.org/jira/browse/HDFS-9260
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode, namenode, performance
> Affects Versions: 2.7.1
> Reporter: Staffan Friberg
> Assignee: Staffan Friberg
> Attachments: HDFS Block and Replica Management 20151013.pdf,
> HDFS-7435.001.patch, HDFS-7435.002.patch, HDFS-7435.003.patch
>
>
> This patch changes the datastructures used for BlockInfos and Replicas to
> keep them sorted. This allows faster and more GC friendly handling of full
> block reports.
> Would like to hear peoples feedback on this change and also some help
> investigating/understanding a few outstanding issues if we are interested in
> moving forward with this.
> There seems to be some timing issues I hit when testing the patch, not sure
> if it is a bug in the patch or something else (most likely the earlier)...
> Tests that fail for me:
> The issues seems to be that the blocks are not on any storage, so no
> replication can occur causing the tests to fail in different ways.
> TestDecomission.testDecommision
> If I add a little sleep after the cleanup/delete things seem to work
> TestDFSStripedOutputStreamWithFailure
> A couple of tests fails in this class.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
