Suresh Srinivas created HDFS-6186:
-------------------------------------
Summary: Pause deletion of blocks when the namenode starts up
Key: HDFS-6186
URL: https://issues.apache.org/jira/browse/HDFS-6186
Project: Hadoop HDFS
Issue Type: Sub-task
Components: namenode
Reporter: Suresh Srinivas
HDFS namenode can delete blocks very quickly, given the deletion happens as a
parallel operation spread across many datanodes. One of the frequent anxieties
I see is that a lot of data can be deleted very quickly, when a cluster is
brought up, especially when one of the storage directories has failed and
namenode metadata was copied from another storage. Copying wrong metadata would
results in some of the newer files (if old metadata was copied) being deleted
along with their blocks.
HDFS-5986 now captures the number of pending deletion block on namenode webUI
and JMX. I propose pausing deletion of blocks for a configured period of time
(default 1 hour?) after namenode comes out of safemode. This will give enough
time for the administrator to notice large number of pending deletion blocks
and take corrective action.
Thoughts?
--
This message was sent by Atlassian JIRA
(v6.2#6252)
