[
https://issues.apache.org/jira/browse/HDFS-7990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14385230#comment-14385230
]
Hudson commented on HDFS-7990:
------------------------------
FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #146 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/146/])
HDFS-7990. IBR delete ack should not be delayed. Contributed by Daryn Sharp.
(kihwal: rev 60882ab26d49f05cbf0686944af6559f86b3417d)
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPServiceActor.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestIncrementalBlockReports.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DNConf.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
> IBR delete ack should not be delayed
> ------------------------------------
>
> Key: HDFS-7990
> URL: https://issues.apache.org/jira/browse/HDFS-7990
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Affects Versions: 2.0.0-alpha
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Fix For: 2.8.0
>
> Attachments: HDFS-7990.patch
>
>
> HDFS-395 added the incremental BR feature. A concern was avoiding a race
> condition with ack-ing block invalidates followed by the directory scanner
> re-adding the block before the async delete service removes the block,
> possibly resulting in a full BR that includes the previously delete ack-ed
> block.
> The solution was to batch & delay block deletion acks via a hardcoded
> interval of 100 * heartbeat interval (default: 5min). The delay isn't
> required:
> # The {{FSDatasetSpi}} tracks blocks pending deletions precisely so the
> scanner won't re-add the block to the volume map
> # Block receiving, received, and deleted go into the same pending report. A
> block received event will trigger an immediate IBR which includes the
> deletion acks. Ie. The delay is meaningless for all but a quiescent cluster
> # Failing to promptly report deleted blocks on a quiescent cluster prevents
> the NN from updating the block maps to remove the locations
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
