[
https://issues.apache.org/jira/browse/HDFS-13693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16875987#comment-16875987
]
Lisheng Sun commented on HDFS-13693:
------------------------------------
Thank [~jojochuang] for your comments.
{quote}I am not a snapshot expert, but I'd be interested to see if
INodeReference can be optimized the same way. If you don't use snapshot and
don't know it off the top of your head, I can find a snapshotted fsimage and
experiment it.
{quote}
Sorry, XiaoMi HDFS don't use snapshot and don't know it off the top of your
head, unfortunately I can't find a snapshotted fsimage and experiment it.
And I have added javadoc for the addChildAtLoading method and updated the
patch. Could you have time to review this patch? Thank you.
> Remove unnecessary search in INodeDirectory.addChild during image loading
> -------------------------------------------------------------------------
>
> Key: HDFS-13693
> URL: https://issues.apache.org/jira/browse/HDFS-13693
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
> Reporter: zhouyingchao
> Assignee: Lisheng Sun
> Priority: Major
> Attachments: HDFS-13693-001.patch
>
>
> In FSImageFormatPBINode.loadINodeDirectorySection, all child INodes are added
> to their parent INode's map one by one. The adding procedure will search a
> position in the parent's map and then insert the child to the position.
> However, during image loading, the search is unnecessary since the insert
> position should always be at the end of the map given the sequence they are
> serialized on disk.
> Test this patch against a fsimage of a 70PB cluster (200million files and
> 300million blocks), the image loading time be reduced from 1210 seconds to
> 1138 seconds.So it can reduce up to about 10% of time.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
