[
https://issues.apache.org/jira/browse/HDFS-11366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiao Chen resolved HDFS-11366.
------------------------------
Resolution: Duplicate
Looks like we already have HDFS-3716 in place to take care of this problem.
Sorry didn't find that earlier.
It's more aggressive than proposed here, but since the purge only happens after
a successful checkpoint, the risk is low.
> Clean up old .ckpt files after saveNamespace
> --------------------------------------------
>
> Key: HDFS-11366
> URL: https://issues.apache.org/jira/browse/HDFS-11366
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs, namenode
> Affects Versions: 2.6.0
> Reporter: Xiao Chen
> Assignee: Xiao Chen
>
> Checkpoints are done in the NN by writing to {{fsimage.ckpt_TXID}} files, and
> rename to {{fsimage_TXID}} files upon success.
> If a checkpoint fails half way, the fsimage.ckpt_ file will be left on disk.
> There is no logic to clean it up at all.
> After talking with [~atm], I understand the historical reason for not
> immediately cleaning up those files, since they maybe useful for disaster
> recovery.
> But feels like cleaning those ckpt files after a successful checkpoint, with
> a larger TXID threshold is also safe to do.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
