[
https://issues.apache.org/jira/browse/HDFS-16423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17478200#comment-17478200
]
Wei-Chiu Chuang commented on HDFS-16423:
----------------------------------------
Thanks for filing the bug report!
I just wonder if HDFS-12914 has anything to do with the symptom you described
here and in HDFS-16420.
Right after a NameNode starts, all storages are regarded as stale initially.
There are supposedly FBRs happening afterwards, but for large clusters there
used to be timeouts that caused FBR to expire and some storages become stale
for an extended period of time. This is no more a problem after HDFS-12914, so
I wonder if HDFS-16423 and HDFS-16420 became less likely.
Nothing to discount the importance of these bug reports! This is great to find
out.
> balancer should not get blocks on stale storages
> ------------------------------------------------
>
> Key: HDFS-16423
> URL: https://issues.apache.org/jira/browse/HDFS-16423
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: balancer & mover
> Reporter: qinyuren
> Assignee: qinyuren
> Priority: Major
> Labels: pull-request-available
> Attachments: image-2022-01-13-17-18-32-409.png
>
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> We have met a problems as described in HDFS-16420
> We found that balancer copied a block multi times without deleting the source
> block if this block was placed in a stale storage. And resulting a block with
> many copies, but these redundant copies are not deleted until the storage
> become not stale.
>
> !image-2022-01-13-17-18-32-409.png|width=657,height=275!
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
