[
https://issues.apache.org/jira/browse/HDFS-16016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17783080#comment-17783080
]
Yuanbo Liu commented on HDFS-16016:
-----------------------------------
[~hexiaoqiao]
> I just wonder if meet some performance issue?
Yes indeed we met. The heartbeat thread not only handle FBR but also dispatch
deleting blocks commands. When the disks of DN are quite busy, the whole
execution time of heartbeat thread will be longer than one minute, and it makes
files cannot be closed in time. We've met this performance issue in our
checkpoint-backup clusters of flink. It's necessary to seperate IBR from
heartbeat thread, and NN will benefit as the client close files faster without
useless retrying. Increasing the frequency of ibr doesn't have much side-effect
as blockReceivedAndDeleted has been refactored with async model long time
ago(HDFS-9198)
> BPServiceActor add a new thread to handle IBR
> ---------------------------------------------
>
> Key: HDFS-16016
> URL: https://issues.apache.org/jira/browse/HDFS-16016
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: JiangHua Zhu
> Assignee: Viraj Jasani
> Priority: Minor
> Labels: pull-request-available
> Fix For: 3.4.0, 3.3.6
>
> Attachments: image-2023-11-03-18-11-54-502.png,
> image-2023-11-06-10-53-13-584.png, image-2023-11-06-10-55-50-939.png
>
> Time Spent: 5h 20m
> Remaining Estimate: 0h
>
> Now BPServiceActor#offerService() is doing many things, FBR, IBR, heartbeat.
> We can handle IBR independently to improve the performance of heartbeat and
> FBR.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
