zhangshuyan0 commented on code in PR #6176:
URL: https://github.com/apache/hadoop/pull/6176#discussion_r1361730711
##########
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeDescriptor.java:
##########
@@ -1007,6 +1013,7 @@ public void updateRegInfo(DatanodeID nodeReg) {
for(DatanodeStorageInfo storage : getStorageInfos()) {
if (storage.getStorageType() != StorageType.PROVIDED) {
storage.setBlockReportCount(0);
+ storage.setBlockContentsStale(true);
Review Comment:
> namenode thinks that this DN contains this block, but actually the DN
doesn't store this block
Actually, this situation can happen at any time, not just between
"registerDataNode" and "blockReport". Why do you think that after the DN is
re-registered, the probability of the above situation happening will increase,
and it needs to be dealt with specifically?
IMO, the concept stale was born to mark replicas that may have been deleted
by NN commands but do not exist in the ExcessRedundancyMap. It is closely
related to the state of ExcessRedundancyMap.
- After the failover, there is no data in the ExcessRedundancyMap of the new
ANN, so all datanodes need to be marked as stale.
- In this patch, the ExcessRedundancyMap of the corresponding DN is cleared
when re-registered, so it needs to be set to stale.
If the data in the ExcessRedundancyMap is correct, it means that NN clearly
knows which replicas are about to be deleted, which ensures that NN will not
actively delete all replicas of a block.
Looking forward to your reply @ZanderXu .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
