[
https://issues.apache.org/jira/browse/HDFS-16348?focusedWorklogId=692594&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-692594
]
ASF GitHub Bot logged work on HDFS-16348:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 08/Dec/21 14:12
Start Date: 08/Dec/21 14:12
Worklog Time Spent: 10m
Work Description: symious commented on pull request #3704:
URL: https://github.com/apache/hadoop/pull/3704#issuecomment-988848771
@tasanuma Thanks for the review.
> I have one question. This feature is similar to ECN. The configuration of
ECN is dfs.pipeline.ecn, which is for the DataNode configuration.
dfs.client.mark.slownode.as.badnode seems the client configuration
I tried to digest the design of the code again. IMHO, the config of
"dfs.pipeline.ecn" is like "dfs.pipeline.ecn.enable", the value of "true" means
datanode should add the ECN information in PipelineAck, so this configuration
is for DataNode to add or not add ECN. On Client side, there is no config for
clients to choose if she should use ECN information, since it will use the
information by default.
In this ticket, I think a similar config of "dfs.pipeline.ecn" should be
"dfs.pipeline.reply_slownode_in_pipeline_ack.enable", based on which DataNodes
will choose to add or skip the slownode information in PipelineAck.
On Client side, "dfs.client.mark.slownode.as.badnode" is more like
"dfs.client.handle.ECN_from_pipelineACK".
I think it's more depend on the subject of the operation, that is the
subject of "dfs.pipeline.ecn" is DataNode, while the subject of
"dfs.client.mark.slownode.as.badnode" is Client.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 692594)
Time Spent: 1h 10m (was: 1h)
> Mark slownode as badnode to recover pipeline
> --------------------------------------------
>
> Key: HDFS-16348
> URL: https://issues.apache.org/jira/browse/HDFS-16348
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: Janus Chow
> Assignee: Janus Chow
> Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> In HDFS-16320, the DataNode can retrieve the SLOW status from each NameNode.
> This ticket is to send this information back to Clients who are writing
> blocks. If a Clients noticed the pipeline is build on a slownode, he/she can
> choose to mark the slownode as a badnode to exclude the node or rebuild a
> pipeline.
> In order to avoid the false positives, we added a config of "threshold", only
> clients continuously receives slownode reply from the same node will the node
> be marked as SLOW.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
