[
https://issues.apache.org/jira/browse/HDFS-1162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12868871#action_12868871
]
dhruba borthakur commented on HDFS-1162:
----------------------------------------
A cluster with 70 million files. The fsimage is 11GB.
1. The secondary namenode fetches this file from the namenode via http. This
takes about 8 minutes.
2. Then the secondary namenode uploads this file back to the namenode. This
takes another 8 minutes
Thus, a total of 16+ minutes is needed to do a checkpoint. This leads to a
scenario where the checkpointing cannot keep up with the rate of growth of the
transaction log. Very Bad!
> Reduce the time required for a checkpoint
> -----------------------------------------
>
> Key: HDFS-1162
> URL: https://issues.apache.org/jira/browse/HDFS-1162
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: name-node
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
>
> The checkpoint time increases linearly with the number of files in the
> cluster. This is a problem with large clusters.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
