[
https://issues.apache.org/jira/browse/HDFS-13693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lisheng Sun updated HDFS-13693:
-------------------------------
Description:
In FSImageFormatPBINode.loadINodeDirectorySection, all child INodes are added
to their parent INode's map one by one. The adding procedure will search a
position in the parent's map and then insert the child to the position.
However, during image loading, the search is unnecessary since the insert
position should always be at the end of the map given the sequence they are
serialized on disk.
Test this patch against a fsimage of a 70PB cluster (200million files and
300million blocks), the image loading time be reduced from 1210 seconds to 1138
seconds.So it can reduce up to about 10% of time.
was:
In FSImageFormatPBINode.loadINodeDirectorySection, all child INodes are added
to their parent INode's map one by one. The adding procedure will search a
position in the parent's map and then insert the child to the position.
However, during image loading, the search is unnecessary since the insert
position should always be at the end of the map given the sequence they are
serialized on disk.
Test this patch against a fsimage of a 70PB cluster (200million files and
300million blocks), the image loading time be reduced from 1210 seconds to 1138
seconds.
> Remove unnecessary search in INodeDirectory.addChild during image loading
> -------------------------------------------------------------------------
>
> Key: HDFS-13693
> URL: https://issues.apache.org/jira/browse/HDFS-13693
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
> Reporter: zhouyingchao
> Assignee: Lisheng Sun
> Priority: Major
> Attachments: HDFS-13693-001.patch
>
>
> In FSImageFormatPBINode.loadINodeDirectorySection, all child INodes are added
> to their parent INode's map one by one. The adding procedure will search a
> position in the parent's map and then insert the child to the position.
> However, during image loading, the search is unnecessary since the insert
> position should always be at the end of the map given the sequence they are
> serialized on disk.
> Test this patch against a fsimage of a 70PB cluster (200million files and
> 300million blocks), the image loading time be reduced from 1210 seconds to
> 1138 seconds.So it can reduce up to about 10% of time.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
