[
https://issues.apache.org/jira/browse/HDFS-7784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Walter Su updated HDFS-7784:
----------------------------
Description: When single Namenode has huge amount of files, without using
federation, the startup/restart speed is slow. The fsimage loading step takes
the most of the time. fsimage loading can seperate to two parts,
deserialization and object construction(mostly map insertion). Deserialization
takes the most of CPU time. So we can do deserialization in parallel, and add
to hashmap in serial. It will significantly reduce the NN start time. (was:
When single Namenode has huge amount of files, without using federation, the
startup/restart speed is slow. The fsimage loading step takes the most of the
time. fsimage loading can seperate to two parts, deserialization and object
construction(mostly map insertion). Deserialization takes the most of CPU time.
So we can do deserialization in parallel, and add to hashmap in parallel. It
will significantly reduce the NN start time.)
> load fsimage in parallel
> ------------------------
>
> Key: HDFS-7784
> URL: https://issues.apache.org/jira/browse/HDFS-7784
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
> Reporter: Walter Su
>
> When single Namenode has huge amount of files, without using federation, the
> startup/restart speed is slow. The fsimage loading step takes the most of the
> time. fsimage loading can seperate to two parts, deserialization and object
> construction(mostly map insertion). Deserialization takes the most of CPU
> time. So we can do deserialization in parallel, and add to hashmap in serial.
> It will significantly reduce the NN start time.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
