[
https://issues.apache.org/jira/browse/HDFS-15140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17026925#comment-17026925
]
Stephen O'Donnell commented on HDFS-15140:
------------------------------------------
All failing tests pass locally so I think they are unrelated.
I am also open to simply switching FoldedTreeSet with TreeMap, and then we can
avoid the need to sort the block reports.
The cost of TreeMap would be about 33MB of extra heap per 1M blocks, but the
access time is similar to FoldedTreeSet, so we would not gain performance with
this change. The hope is that we would avoid the degradation which occurs in
foldedTreeSet for an unknown reason after some time.
Discussing this offline, someone asked me whether this would impact IBRs. I
don't believe this will impact IBRs in any way. The reason, is that the IBR
related blocks are added to the IncrementalBlockReportManager where they are
stored in an unrelated structure and then transmitted to the namenode via the
heartbeat thread (but not as part of the heartbeat).
> Replace FoldedTreeSet in Datanode with SortedSet or TreeMap
> -----------------------------------------------------------
>
> Key: HDFS-15140
> URL: https://issues.apache.org/jira/browse/HDFS-15140
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: datanode
> Affects Versions: 3.3.0
> Reporter: Stephen O'Donnell
> Assignee: Stephen O'Donnell
> Priority: Major
> Attachments: HDFS-15140.001.patch, HDFS-15140.002.patch
>
>
> Based on the problems discussed in HDFS-15131, I would like to explore
> replacing the FoldedTreeSet structure in the datanode with a builtin Java
> equivalent - either SortedSet or TreeMap.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
