[ https://issues.apache.org/jira/browse/HDFS-14513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16850707#comment-16850707 ]
He Xiaoqiao edited comment on HDFS-14513 at 5/29/19 4:40 PM: ------------------------------------------------------------- [~elgoiri] Thanks for taking a look. I would like to offer detail information about this issue: a. Checkpointer/FSImageSaver is regular tasks, at most per hour by default. b. If meta of NameNode is huge, for example 500m inodes+blocks, it will take for a while to persist to fsimage file. c. If we shutdown NameNode during checkpointing, fsimage checkpiont file will be not cleaned. After long time, there could be many useless checkpoint files. [~elgoiri] [^HDFS-14513.002.patch] use {{org.apache.hadoop.util.ShutdownHookManager}} to manage the hook logic, would you take another look. Thanks. was (Author: hexiaoqiao): [~elgoiri] Thanks for taking a look. I would like to offer detail information about this issue: a. Checkpointer/FSImageSaver is regular tasks, at most per hour by default. b. If meta of NameNode is huge, for example 500m inodes+blocks, it will take for a while to persist to fsimage file. c. If we shutdown NameNode during checkpointing, fsimage checkpiont file will be not cleaned. After long time, there could be many useless checkpoint files. [~elgoiri] [^HDFS-14351.002.patch] use {{org.apache.hadoop.util.ShutdownHookManager}} to manage the hook logic, would you take another look. Thanks. > FSImage which is saving should be clean while NameNode shutdown > --------------------------------------------------------------- > > Key: HDFS-14513 > URL: https://issues.apache.org/jira/browse/HDFS-14513 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode > Reporter: He Xiaoqiao > Assignee: He Xiaoqiao > Priority: Major > Attachments: HDFS-14513.001.patch, HDFS-14513.002.patch > > > FSImage checkpoint file which is saving could not be clean while NameNode > shutdown at the same time. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org