[
https://issues.apache.org/jira/browse/HDFS-14513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
He Xiaoqiao updated HDFS-14513:
-------------------------------
Description:
Checkpointer/FSImageSaver is regular tasks and dump NameNode meta to disk, at
most per hour by default. If it receive some command (e.g. transition to active
in HA mode) it will cancel checkpoint and delete tmp files using
{{FSImage#deleteCancelledCheckpoint}}. However if NameNode shutdown when
checkpoint, the tmp files will not be cleaned anymore.
Consider there are 500m inodes+blocks, it could cost 5~10min to finish once
checkpoint, if we shutdown NameNode during checkpointing, fsimage checkpoint
file will never be cleaned, after long time, there could be many useless
checkpoint files. So I propose that we should add hook to clean that when
shutdown.
was:FSImage checkpoint file which is saving could not be clean while NameNode
shutdown at the same time.
> FSImage which is saving should be clean while NameNode shutdown
> ---------------------------------------------------------------
>
> Key: HDFS-14513
> URL: https://issues.apache.org/jira/browse/HDFS-14513
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
> Reporter: He Xiaoqiao
> Assignee: He Xiaoqiao
> Priority: Major
> Attachments: HDFS-14513.001.patch, HDFS-14513.002.patch,
> HDFS-14513.003.patch
>
>
> Checkpointer/FSImageSaver is regular tasks and dump NameNode meta to disk, at
> most per hour by default. If it receive some command (e.g. transition to
> active in HA mode) it will cancel checkpoint and delete tmp files using
> {{FSImage#deleteCancelledCheckpoint}}. However if NameNode shutdown when
> checkpoint, the tmp files will not be cleaned anymore.
> Consider there are 500m inodes+blocks, it could cost 5~10min to finish once
> checkpoint, if we shutdown NameNode during checkpointing, fsimage checkpoint
> file will never be cleaned, after long time, there could be many useless
> checkpoint files. So I propose that we should add hook to clean that when
> shutdown.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
