ZanderXu commented on code in PR #6176:
URL: https://github.com/apache/hadoop/pull/6176#discussion_r1360449635
##########
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeDescriptor.java:
##########
@@ -1007,6 +1013,7 @@ public void updateRegInfo(DatanodeID nodeReg) {
for(DatanodeStorageInfo storage : getStorageInfos()) {
if (storage.getStorageType() != StorageType.PROVIDED) {
storage.setBlockReportCount(0);
+ storage.setBlockContentsStale(true);
Review Comment:
Thanks @haiyang1987 for your report. And thanks @zhangshuyan0 for your
review.
I think this modification is a new bug, not related to this case, we need to
fix this bug in a new issue.
We can produce this bug by the following steps:
- Assume that there is block1 contains three replicas, dn1, dn2
- DN1 is shutdown for maintenance for corrupt disk
- Admin removed the corrupted disk and restart datanode
- DN1 try to register it to NameNode through registerDatanode rpc
- End-user try to decrease the replicas of block from 2 to 1 through
setReplication RPC
- Block1 still contains three replicas in namenode, but the dn1 is not
existed because it is stored in a corrupt disk
- NameNode select dn2 as a redundancy replica for this block to delete
- DN1 try to report all stored blocks to namnode through blockreport rpc
- NameNode will remove the dn1 replica for block1 because the blockreport
from DN1 doesn't contains block1
After these two operations(setReplication from end-user and restart from
admin), the block1 may lose all replicas.
So I think we should mark all storage as a stale storage while namenode
processing registerdatanode rpc, so that this case can be fixed.
@zhangshuyan0 @haiyang1987 I'm looking forward your good idea, thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
