[
https://issues.apache.org/jira/browse/HDFS-14790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16918150#comment-16918150
]
Wei-Chiu Chuang commented on HDFS-14790:
----------------------------------------
Actually, I think this is is the project you're looking at HDFS-13572 (check
out the design doc)
> Support Client Write Fan-Out
> ----------------------------
>
> Key: HDFS-14790
> URL: https://issues.apache.org/jira/browse/HDFS-14790
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: block placement, hdfs-client
> Affects Versions: 3.3.0
> Reporter: David Mollitor
> Priority: Major
>
> The default behavior of an HDFS write is to setup a pipeline. A file is
> broken into packets and sent through the pipeline. Pipelining provides good
> throughput, but latency suffers.
> Allowing a client to specify a fan-out strategy allows the client to send the
> packets to the DataNodes concurrently instead of passing the packet through a
> pipeline serially.
> {code:none}
> # Pipeline
> C |-------> DN -------> DN -------> DN
> # Fan Out
> |-------> DN
> C |-------> DN
> |-------> DN
> {code}
> Also, if there's a 'min replication' of, for example, 2. The client only
> needs to wait for the first 2 ACKs before writing the next packet as long as
> the 2 ACKs are from different racks. The block placement rules may need to
> support this.
> HBase requires this improved latency.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
