[
https://issues.apache.org/jira/browse/HDFS-7270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14308262#comment-14308262
]
Daryn Sharp commented on HDFS-7270:
-----------------------------------
I've looked at the patch again. The problem is changing the existing protobuf
Status enum record tag from an enum to a uint32. That pretty much violates the
compatibility promise that protobufs are supposed to provide.
The reason the tag was changed was to allow masking the ECN bit atop the status
value. If it's off, 0 is masked onto the status which is why it "works".
The compatible solution is do what we always do: Add a new optional protobuf
tag for ECN. It's a dangerous precedent to allow breaking compatibility just
to save nominally 3 bytes across the wire.
> Add congestion signaling capability to DataNode write protocol
> --------------------------------------------------------------
>
> Key: HDFS-7270
> URL: https://issues.apache.org/jira/browse/HDFS-7270
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
> Reporter: Haohui Mai
> Assignee: Haohui Mai
> Attachments: HDFS-7270.000.patch, HDFS-7270.001.patch,
> HDFS-7270.002.patch, HDFS-7270.003.patch, HDFS-7270.004.patch
>
>
> When a client writes to HDFS faster than the disk bandwidth of the DNs, it
> saturates the disk bandwidth and put the DNs unresponsive. The client only
> backs off by aborting / recovering the pipeline, which leads to failed writes
> and unnecessary pipeline recovery.
> This jira proposes to add explicit congestion control mechanisms in the
> writing pipeline.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
