[
https://issues.apache.org/jira/browse/HDFS-9260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Staffan Friberg updated HDFS-9260:
----------------------------------
Attachment: HDFS-7435.004.patch
All HDFS tests should now pass.
Fixes include
Correctly handle EC blocks which are negative, and need to be masked
Initial report might sometime report into a storage containing new blocks
reported by incremental block report, addStoredLast will fallback on regular
add if not sorted
remove debugging output
remove unused GSet import
invalidate list must be a Block and not a Replica
Left todo
Handle old nodes which don't send data sorted
Add a 'sorted' field in the report PB
Figure out how how to be able to handle reports when cluster contains
negative entries that are not EC blocks
> Improve performance and GC friendliness of startup and FBRs
> -----------------------------------------------------------
>
> Key: HDFS-9260
> URL: https://issues.apache.org/jira/browse/HDFS-9260
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode, namenode, performance
> Affects Versions: 2.7.1
> Reporter: Staffan Friberg
> Assignee: Staffan Friberg
> Attachments: HDFS Block and Replica Management 20151013.pdf,
> HDFS-7435.001.patch, HDFS-7435.002.patch, HDFS-7435.003.patch,
> HDFS-7435.004.patch
>
>
> This patch changes the datastructures used for BlockInfos and Replicas to
> keep them sorted. This allows faster and more GC friendly handling of full
> block reports.
> Would like to hear peoples feedback on this change and also some help
> investigating/understanding a few outstanding issues if we are interested in
> moving forward with this.
> There seems to be some timing issues I hit when testing the patch, not sure
> if it is a bug in the patch or something else (most likely the earlier)...
> Tests that fail for me:
> The issues seems to be that the blocks are not on any storage, so no
> replication can occur causing the tests to fail in different ways.
> TestDecomission.testDecommision
> If I add a little sleep after the cleanup/delete things seem to work
> TestDFSStripedOutputStreamWithFailure
> A couple of tests fails in this class.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
