[
https://issues.apache.org/jira/browse/HDFS-13693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16890943#comment-16890943
]
Lisheng Sun edited comment on HDFS-13693 at 7/23/19 12:19 PM:
--------------------------------------------------------------
{code:java}
[ERROR] Failed to execute goal
org.apache.hadoop:hadoop-maven-plugins:3.3.0-SNAPSHOT:protoc (compile-protoc)
on project hadoop-common: org.apache.maven.plugin.MojoExecutionException:
protoc version is 'libprotoc 3.0.0', expected version is '2.5.0' -> [Help 1]
{code}
I think this failure is unrelated to this patch. From the output log，it should
be protoc version problem.
was (Author: leosun08):
{code:java}
[ERROR] Failed to execute goal
org.apache.hadoop:hadoop-maven-plugins:3.3.0-SNAPSHOT:protoc (compile-protoc)
on project hadoop-common: org.apache.maven.plugin.MojoExecutionException:
protoc version is 'libprotoc 3.0.0', expected version is '2.5.0' -> [Help 1]
{code}
I think this failure is unrelated to this patch. From the output log，it should
be proto version problem.
> Remove unnecessary search in INodeDirectory.addChild during image loading
> -------------------------------------------------------------------------
>
> Key: HDFS-13693
> URL: https://issues.apache.org/jira/browse/HDFS-13693
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
> Reporter: zhouyingchao
> Assignee: Lisheng Sun
> Priority: Major
> Fix For: 3.3.0
>
> Attachments: HDFS-13693-001.patch, HDFS-13693-002.patch,
> HDFS-13693-003.patch, HDFS-13693-004.patch, HDFS-13693-005.patch
>
>
> In FSImageFormatPBINode.loadINodeDirectorySection, all child INodes are added
> to their parent INode's map one by one. The adding procedure will search a
> position in the parent's map and then insert the child to the position.
> However, during image loading, the search is unnecessary since the insert
> position should always be at the end of the map given the sequence they are
> serialized on disk.
> Test this patch against a fsimage of a 70PB cluster (200million files and
> 300million blocks), the image loading time be reduced from 1210 seconds to
> 1138 seconds.So it can reduce up to about 10% of time.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
