farmmamba created HDFS-17050:
--------------------------------
Summary: Erasure coding: invalidate duplicated block when actual
block numbers less than data blocks plus parity blocks.
Key: HDFS-17050
URL: https://issues.apache.org/jira/browse/HDFS-17050
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 3.3.2, 3.4.0
Reporter: farmmamba
Currently, I found a strange phenomenon mentioned in HDFS-17047.
When triggered FBR manually or automatically, we found some warn log like below:
{code:java}
2023-06-14 16:29:36,432 WARN BlockStateChange: BLOCK* addStoredBlock: block
blk_-9223372036578646784_59354864 moved to storageType DISK on node
datanode12:50010
2023-06-14 16:29:36,477 WARN BlockStateChange: BLOCK* addStoredBlock: block
blk_-9223372036578646784_59354864 moved to storageType DISK on node
datanode12:50010{code}
The above logs print the same storedBlock two times. After diving into logs, I
found that there exist two blocks of a same block group due to some unknown
reasons. And one of the two blocks is also exists in other datanode. But fsck
did not print the duplicated replicas info.
additional information： the file is 3MB+, we use RS-6-3-1024K, so the fsck
only print seven blocks information. But indeed, we have eight blocks and one
of them is a duplicated block.
The reason why print above logs is that:
In BlockManager#addStoredBlock method, because a datanode has two blocks of the
same block group, the AddBlockResult would be REPLACED.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
