[
https://issues.apache.org/jira/browse/HDFS-15764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17296826#comment-17296826
]
Xiaoqiao He commented on HDFS-15764:
------------------------------------
Thanks [~hadoop_yangyun] for your ping, TBH, I agree that we should report
immediately if a block is lost, But I am still concerned if there will be RPC
request flood (blockReceivedAndDeleted) to NN when one disk failed and many
read request to this disk before found by FailVolumeScanner when disable IBR.
Considering that there are 100 blocks replica located at disk a which is failed
now, then dispatch 100 read request during short time and try to access these
replicas but not success, then there will be 100 blockReceivedAndDeleted RPC
request to NN when patched if disable IBR feature, right? It will be one
potential issue when we enlarge 100 to 100,000 replicas. cc
[~elgoiri],[~ayushtkn] what do you think?
Thanks [~hadoop_yangyun] for your works again.
> Notify Namenode missing or new block on disk as soon as possible
> ----------------------------------------------------------------
>
> Key: HDFS-15764
> URL: https://issues.apache.org/jira/browse/HDFS-15764
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
> Reporter: Yang Yun
> Assignee: Yang Yun
> Priority: Minor
> Attachments: HDFS-15764.001.patch, HDFS-15764.002.patch,
> HDFS-15764.003.patch
>
>
> When a bock file is deleted on disk or copied back to the disk, the
> DirectoryScanner can find the change, but the namenode know the change only
> untill the next full report. And in big cluster the period of full report is
> set to long time invterval.
> Call notifyNamenodeDeletedBlock if block files are deleted and call
> notifyNamenodeReceivedBlock if the block files is found agian. So the
> Incremental block report can send the change to namenode in next heartbeat.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
