[
https://issues.apache.org/jira/browse/HDFS-15764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17269764#comment-17269764
]
Yang Yun commented on HDFS-15764:
---------------------------------
Thanks [~ayushtkn] for your comment.
Yes, if read a bad bock, it will call reportBadBlock immediately. But if delete
a block file and nobody access it, it will wait untill the DirectoryScanner to
find it out and report the missing in next round full report. So the namenode
will know the missing more later. And in big cluster, we often set a long time
interval for the full report period, that increase data loss risks.
If block missing occurs on many datanode simultaneously, it may cause huge
number of calls. I think that's rare case and the namenode is more effective
when deal with the received and deleted block.
Yes, For addition or removal of disk/volume there is a separate block of
code. For failed disk, it will schedule a full block report .
> Notify Namenode missing or new block on disk as soon as possible
> ----------------------------------------------------------------
>
> Key: HDFS-15764
> URL: https://issues.apache.org/jira/browse/HDFS-15764
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
> Reporter: Yang Yun
> Assignee: Yang Yun
> Priority: Minor
> Attachments: HDFS-15764.001.patch, HDFS-15764.002.patch
>
>
> When a bock file is deleted on disk or copied back to the disk, the
> DirectoryScanner can find the change, but the namenode know the change only
> untill the next full report. And in big cluster the period of full report is
> set to long time invterval.
> Call notifyNamenodeDeletedBlock if block files are deleted and call
> notifyNamenodeReceivedBlock if the block files is found agian. So the
> Incremental block report can send the change to namenode in next heartbeat.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
