[
https://issues.apache.org/jira/browse/HDFS-17071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
farmmamba updated HDFS-17071:
-----------------------------
Description:
Currently, I found a strange phenomenon mentioned in HDFS-17047.
When triggered FBR manually or automatically, we found some warn log like below:
{code:java}
2023-06-14 16:29:36,432 WARN BlockStateChange: BLOCK* addStoredBlock: block
blk_-9223372036578646784_59354864 moved to storageType DISK on node
datanode12:50010
2023-06-14 16:29:36,477 WARN BlockStateChange: BLOCK* addStoredBlock: block
blk_-9223372036578646784_59354864 moved to storageType DISK on node
datanode12:50010{code}
The above logs print the same storedBlock two times. After diving into logs, I
found that there exist two blocks of a same block group due to some unknown
reasons. And one of the two blocks is also exists in other datanode. But fsck
did not print the duplicated replicas info.
additional information： the file is 3MB+, we use RS-6-3-1024K, so the fsck
only print seven blocks information. But indeed, we have eight blocks and one
of them is a duplicated block.
The reason why print above logs is that:
In BlockManager#addStoredBlock method, because a datanode has two blocks of the
same block group, the AddBlockResult would be REPLACED.
> Erasure coding: delete duplicated ec block with same block group id on one
> datanode.
> ------------------------------------------------------------------------------------
>
> Key: HDFS-17071
> URL: https://issues.apache.org/jira/browse/HDFS-17071
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: farmmamba
> Assignee: farmmamba
> Priority: Major
>
> Currently, I found a strange phenomenon mentioned in HDFS-17047.
> When triggered FBR manually or automatically, we found some warn log like
> below:
> {code:java}
> 2023-06-14 16:29:36,432 WARN BlockStateChange: BLOCK* addStoredBlock: block
> blk_-9223372036578646784_59354864 moved to storageType DISK on node
> datanode12:50010
> 2023-06-14 16:29:36,477 WARN BlockStateChange: BLOCK* addStoredBlock: block
> blk_-9223372036578646784_59354864 moved to storageType DISK on node
> datanode12:50010{code}
> The above logs print the same storedBlock two times. After diving into logs,
> I found that there exist two blocks of a same block group due to some unknown
> reasons. And one of the two blocks is also exists in other datanode. But fsck
> did not print the duplicated replicas info.
> additional information： the file is 3MB+, we use RS-6-3-1024K, so the fsck
> only print seven blocks information. But indeed, we have eight blocks and one
> of them is a duplicated block.
>
> The reason why print above logs is that:
> In BlockManager#addStoredBlock method, because a datanode has two blocks of
> the same block group, the AddBlockResult would be REPLACED.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
