[
https://issues.apache.org/jira/browse/HDFS-12282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16136406#comment-16136406
]
Anu Engineer commented on HDFS-12282:
-------------------------------------
[~cheersyang] My concern comes from the issues I have seen in the HDFS world.
The heartbeat became a scalability bottleneck when it started doing too many
things. Then HDFS eventually had to invent things like "life-line protocol",
which is really Heartbeat which does nothing else. Disk I/O also has a very
weird problem when the disk is slow or busy it will slow down everything in its
path. That in turn, makes the SCM think that data nodes are not HB-ing and
marks them as dead.
This is a first-hand issue from very busy HDFS clusters, so life line protocol
was developed to purposefully make HB handling lightweight. I am just
suggesting that we follow the model, HB processing should be quick -- if you
need to do any work based on HB, I think we should post that to a queue and let
an async system pick up the work.
> Ozone: DeleteKey-4: Block delete via HB between SCM and DN
> ----------------------------------------------------------
>
> Key: HDFS-12282
> URL: https://issues.apache.org/jira/browse/HDFS-12282
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode, ozone, scm
> Reporter: Weiwei Yang
> Assignee: Weiwei Yang
> Attachments: Block delete via HB between SCM and DN.png,
> HDFS-12282-HDFS-7240.001.patch, HDFS-12282-HDFS-7240.002.patch,
> HDFS-12282.WIP.patch
>
>
> This is the task 3 in the design doc, implements the SCM to datanode
> interactions. Including
> # SCM sends block deletion message via HB to datanode
> # datanode changes block state to deleting when processes the HB response
> # datanode sends deletion ACKs back to SCM
> # SCM handles ACKs and removes blocks in DB
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
