[
https://issues.apache.org/jira/browse/HDFS-7435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14223574#comment-14223574
]
Suresh Srinivas commented on HDFS-7435:
---------------------------------------
[~jingzhao] and [~daryn], I think chunking the block reports into a set of
arrays is a good change to avoid possible GC promotion failures due to not
finding contiguous memory in old generation heap. We should consider doing that
along with this change to avoid unnecessary layout change.
> PB encoding of block reports is very inefficient
> ------------------------------------------------
>
> Key: HDFS-7435
> URL: https://issues.apache.org/jira/browse/HDFS-7435
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode, namenode
> Affects Versions: 2.0.0-alpha, 3.0.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Priority: Critical
> Attachments: HDFS-7435.patch
>
>
> Block reports are encoded as a PB repeating long. Repeating fields use an
> {{ArrayList}} with default capacity of 10. A block report containing tens or
> hundreds of thousand of longs (3 for each replica) is extremely expensive
> since the {{ArrayList}} must realloc many times. Also, decoding repeating
> fields will box the primitive longs which must then be unboxed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
