[
https://issues.apache.org/jira/browse/HDFS-7270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14309397#comment-14309397
]
Daryn Sharp commented on HDFS-7270:
-----------------------------------
> Both enum and a uint32 are encoded as a varint32 over the wire. ... Can you
> clarify what compatibility means in your mind?
Yes, I understand how/why that technically works over the wire. Clever, but
bad because modifying an existing protobuf tag to changing a public API's
method return signature. We don't do that, unless perhaps there's an
extraordinary circumstance where the API is insecure or completely broken.
This isn't an extraordinary case. The feature provides a client advisory which
newer clients aren't obligated honor. Older clients are broken just to save 1
byte per datanode in the pipeline ack due to the overloading of the Status
field to utilize more bits.
Simply adding a new optional tag, whose bits can be used to encode more than
ECN in the future, is a completely compatible change.
> Add congestion signaling capability to DataNode write protocol
> --------------------------------------------------------------
>
> Key: HDFS-7270
> URL: https://issues.apache.org/jira/browse/HDFS-7270
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
> Reporter: Haohui Mai
> Assignee: Haohui Mai
> Attachments: HDFS-7270.000.patch, HDFS-7270.001.patch,
> HDFS-7270.002.patch, HDFS-7270.003.patch, HDFS-7270.004.patch
>
>
> When a client writes to HDFS faster than the disk bandwidth of the DNs, it
> saturates the disk bandwidth and put the DNs unresponsive. The client only
> backs off by aborting / recovering the pipeline, which leads to failed writes
> and unnecessary pipeline recovery.
> This jira proposes to add explicit congestion control mechanisms in the
> writing pipeline.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
