[
https://issues.apache.org/jira/browse/HADOOP-227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12464312
]
dhruba borthakur commented on HADOOP-227:
-----------------------------------------
The namenode is typically bottlenecked on CPU whereas a new checkpoint-upload
is IO bound. A periodic checkpoint acquires the fsnamesystem lock and
switches/renames files. The overhead of doing this operation once every 360000
namenode transactions (in our hour) should be minimal. I would like to make
64MB the default editlogsize and once deployed, I will observe a real life
cluster and then determine if we need to change the default size.
Reagrding the optimization of 'image token': I would defer it till I see
measureable difference in performance on a real cluster. My goal is to keep the
periodic checkpoint "as-simple-as-possible" while achieving the goal of making
the "namenode-restart faster".
> Namespace check pointing is not performed until the namenode restarts.
> ----------------------------------------------------------------------
>
> Key: HADOOP-227
> URL: https://issues.apache.org/jira/browse/HADOOP-227
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.2.0
> Reporter: Konstantin Shvachko
> Assigned To: dhruba borthakur
> Attachments: patch-async-checkpoints-0.9.0,
> patch-async-checkpoints-0.9.0, patch-async-checkpoints-0.9.0,
> periodiccheckpoint.patch, periodiccheckpoint2.patch,
> periodiccheckpoint3.patch, periodiccheckpoint4.patch
>
>
> In current implementation when the name node starts, it reads its image file,
> then
> the edits file, and then saves the updated image back into the image file.
> The image file is never updated after that.
> In order to provide the system reliability reliability the namespace
> information should
> be check pointed periodically, and the edits file should be kept relatively
> small.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
