[
https://issues.apache.org/jira/browse/HDFS-15764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17299368#comment-17299368
]
Yang Yun commented on HDFS-15764:
---------------------------------
Thanks [~ayushtkn] for your review. Sorry for not fullly Icatching your meaning.
Yes, this solution is reset every run of DirectoryScanner, the frequency is
about 6 hours. we only notify limted number of replicas to pending IBRs list.
if there are more replicas, the left replicas will be sent in next FBR.
Do you mean to add an upper-bound number to the IncrementalBlockReportManager?
to limit the report number of every IBR or to limit reporting times in a
period? That may impact the normal behavior. Some writer may be waiting the
reporting of block.
Or we add special list in IncrementalBlockReportManager, add a limited number
of replicas foud by scaner in every normal IBR?
> Notify Namenode missing or new block on disk as soon as possible
> ----------------------------------------------------------------
>
> Key: HDFS-15764
> URL: https://issues.apache.org/jira/browse/HDFS-15764
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
> Reporter: Yang Yun
> Assignee: Yang Yun
> Priority: Minor
> Attachments: HDFS-15764.001.patch, HDFS-15764.002.patch,
> HDFS-15764.003.patch, HDFS-15764.004.patch
>
>
> When a bock file is deleted on disk or copied back to the disk, the
> DirectoryScanner can find the change, but the namenode know the change only
> untill the next full report. And in big cluster the period of full report is
> set to long time invterval.
> Call notifyNamenodeDeletedBlock if block files are deleted and call
> notifyNamenodeReceivedBlock if the block files is found agian. So the
> Incremental block report can send the change to namenode in next heartbeat.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
