-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/998/
-----------------------------------------------------------
(Updated 2011-07-11 21:06:02.473171)
Review request for hadoop-hdfs, Dhruba Borthakur and Hairong Kuang.
Changes
-------
- I removed renaming of the block and meta file from FSDataset. We should
consider it in a separate diff, as suggested by Hairong. By doing so, I need to
defer the notification until the FSAsyncDiskService deletes the files.
- I added an optimization, which does not procuce ACKs for blocks which come
from entirely deleted files (in this situation, the blocks are instantly
removed from the blocks map at the NN side, and hence we need no ACK).
Summary
-------
This patch introduces explicit ACKs sent form datanodes to the namenode in
order to minimize the difference between NN blocks info and DN state.
This will allow for sending the full block reports less frequently, which in
turn will minimize the overhead of processing them, and blocking the namenode
for extended time.
This addresses bug HDFS-395.
https://issues.apache.org/jira/browse/HDFS-395
Diffs (updated)
-----
trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
1145346
trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
1145346
trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/datanode/FSDataset.java
1145346
trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/datanode/FSDatasetAsyncDiskService.java
1145346
trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
1145346
trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
1145346
trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/protocol/BlockCommand.java
1145346
trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/protocol/DatanodeProtocol.java
1145346
trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/protocol/ReceivedDeletedBlockInfo.java
PRE-CREATION
trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/NNThroughputBenchmark.java
1145346
trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestDeadDatanode.java
1145346
Diff: https://reviews.apache.org/r/998/diff
Testing
-------
Will do JUnit tests.
Thanks,
Tomasz
