[
https://issues.apache.org/jira/browse/HDFS-16016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17782501#comment-17782501
]
Yuanbo Liu commented on HDFS-16016:
-----------------------------------
[~hexiaoqiao] [~vjasani] [~tasanuma]
Sorry to interrrupt. We've used this patch in our production env. We find that
this patch may lead block to be marked as a missing block in this scenario:
1. client write block to dn1 and dn2 as pipeline
2. dn1/dn2 has over 1 million blocks and fbr is reported one by one disk.
3. when fbr is being reported, client finish writing, dn1/dn2 starts to trigger
ibr in different thread.
4. nn receives all the operations and because of the namesystem write lock, the
operations are executed in line as dn1-ibr, dn2-ibr, dn1-fbr-disk, dn2-fbr-disk
then the replicas of the block will be removed from meta in reportDiff
(dn1-fbr-disk, dn2-fbr-disk) and marked as a missing block untils next fbrs
from dn1 and dn2.
!image-2023-11-03-18-11-54-502.png|width=340,height=279!
> BPServiceActor add a new thread to handle IBR
> ---------------------------------------------
>
> Key: HDFS-16016
> URL: https://issues.apache.org/jira/browse/HDFS-16016
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: JiangHua Zhu
> Assignee: Viraj Jasani
> Priority: Minor
> Labels: pull-request-available
> Fix For: 3.4.0, 3.3.6
>
> Attachments: image-2023-11-03-18-11-54-502.png
>
> Time Spent: 5h 20m
> Remaining Estimate: 0h
>
> Now BPServiceActor#offerService() is doing many things, FBR, IBR, heartbeat.
> We can handle IBR independently to improve the performance of heartbeat and
> FBR.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
