[
https://issues.apache.org/jira/browse/HDFS-16245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17423507#comment-17423507
]
JiangHua Zhu commented on HDFS-16245:
-------------------------------------
Here are some examples when loading the FSImage file is complete.
E.g:
2021-10-02 18:30:29,380 [Listener at localhost/55058] INFO
namenode.FSImageFormatPBINode
(FSImageFormatPBINode.java:loadINodeSectionInParallel(432))-Loading the INode
section in parallel with 4 sub-sections
2021-10-02 18:30:29,381 [Listener at localhost/55058] INFO
namenode.FSImageFormatPBINode
(FSImageFormatPBINode.java:loadINodeSectionHeader(423))-Loading xxxx INodes.
2021-10-02 18:30:29,388 [Listener at localhost/55058] INFO
namenode.FSImageFormatPBINode
(FSImageFormatPBINode.java:loadINodeSectionInParallel(479))-Completed loading
all INode sections. Loaded xxxx inodes.
2021-10-02 18:30:29,388 [Listener at localhost/55058] INFO
namenode.FSImageFormatPBINode
(FSImageFormatPBINode.java:loadINodeDirectorySectionInParallel(227))-Loading
the INodeDirectory section in parallel with 4 sub-sections
2021-10-02 18:30:29,389 [Listener at localhost/55058] INFO
namenode.FSImageFormatPBINode
(FSImageFormatPBINode.java:loadINodeDirectorySectionInParallel(268))-Completed
loading all INodeDirectory sub-sections. Loaded xxxx inodes.
> Record the number of INodeDirectory when loading the FSImage file
> -----------------------------------------------------------------
>
> Key: HDFS-16245
> URL: https://issues.apache.org/jira/browse/HDFS-16245
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
> Reporter: JiangHua Zhu
> Assignee: JiangHua Zhu
> Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When starting parallel loading of FsImage (dfs.image.parallel.load=true), you
> can see in the startup log that the number of loaded INodes has been printed,
> but it is impossible to know the specific number of loaded INodeDirectory,
> because it is not printed now.
> For example, here are some startup information:
> 2021-09-30 19:12:55,012 [15587]-INFO
> [main:FSImageFormatProtobuf$Loader@340]-The fsimage will be loaded in
> parallel using 4 threads
> 2021-09-30 19:12:55,031 [15606]-INFO
> [main:FSImageFormatPBINode$Loader@418]-Loading the INode section in parallel
> with 12 sub-sections
> 2021-09-30 19:12:55,034 [15609]-INFO
> [main:FSImageFormatPBINode$Loader@409]-Loading xxxx INodes.
> ......
> 2021-09-30 19:30:37,080 [1077655]-INFO
> [main:FSImageFormatPBINode$Loader@465]-Completed loading all INode sections.
> Loaded xxxx inodes.
> 2021-09-30 19:30:37,086 [1077661]-INFO
> [main:FSImageFormatPBINode$Loader@222]-Loading the INodeDirectory section in
> parallel with 12 sub-sections
> ......
> 2021-09-30 19:36:58,074 [1458649]-INFO
> [main:FSImageFormatPBINode$Loader@261]-Completed loading all INodeDirectory
> sub-sections
> 2021-09-30 19:36:58,076 [1458651]-INFO
> [main:FSImageFormatPBINode$Loader@339]-Completed update blocks map and name
> cache, total waiting duration 1 ms.
> 2021-09-30 19:36:58,111 [1458686]-INFO
> [main:FSImageFormatProtobuf$Loader@248]-Loaded FSImage in xxxx seconds.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
