[
https://issues.apache.org/jira/browse/HADOOP-227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12464301
]
Raghu Angadi commented on HADOOP-227:
-------------------------------------
How much time does it take to merge 512MB? I think as long as time to merge
edit.log is in the order of 30-60 sec, that should be fine. That way if a node
is lightly loaded it will checkpoint every hour and edit.log would still be
much smaller and if it busy, we won't add extra checkpointing load.
Another thing, to implement 'image file token' to avoid steps 1. and 2. above,
we don't need to store any extra state on disk. This would just be a runtime
property. Every time primary gets a new image from secondary, it also exchanges
a 'token/etag'. If primary restarts, first check point will have steps 1. and 2.
> Namespace check pointing is not performed until the namenode restarts.
> ----------------------------------------------------------------------
>
> Key: HADOOP-227
> URL: https://issues.apache.org/jira/browse/HADOOP-227
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.2.0
> Reporter: Konstantin Shvachko
> Assigned To: dhruba borthakur
> Attachments: patch-async-checkpoints-0.9.0,
> patch-async-checkpoints-0.9.0, patch-async-checkpoints-0.9.0,
> periodiccheckpoint.patch, periodiccheckpoint2.patch,
> periodiccheckpoint3.patch, periodiccheckpoint4.patch
>
>
> In current implementation when the name node starts, it reads its image file,
> then
> the edits file, and then saves the updated image back into the image file.
> The image file is never updated after that.
> In order to provide the system reliability reliability the namespace
> information should
> be check pointed periodically, and the edits file should be kept relatively
> small.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
