[ https://issues.apache.org/jira/browse/HDFS-14790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16918150#comment-16918150 ]
Wei-Chiu Chuang commented on HDFS-14790: ---------------------------------------- Actually, I think this is is the project you're looking at HDFS-13572 (check out the design doc) > Support Client Write Fan-Out > ---------------------------- > > Key: HDFS-14790 > URL: https://issues.apache.org/jira/browse/HDFS-14790 > Project: Hadoop HDFS > Issue Type: Improvement > Components: block placement, hdfs-client > Affects Versions: 3.3.0 > Reporter: David Mollitor > Priority: Major > > The default behavior of an HDFS write is to setup a pipeline. A file is > broken into packets and sent through the pipeline. Pipelining provides good > throughput, but latency suffers. > Allowing a client to specify a fan-out strategy allows the client to send the > packets to the DataNodes concurrently instead of passing the packet through a > pipeline serially. > {code:none} > # Pipeline > C |-------> DN -------> DN -------> DN > # Fan Out > |-------> DN > C |-------> DN > |-------> DN > {code} > Also, if there's a 'min replication' of, for example, 2. The client only > needs to wait for the first 2 ACKs before writing the next packet as long as > the 2 ACKs are from different racks. The block placement rules may need to > support this. > HBase requires this improved latency. -- This message was sent by Atlassian Jira (v8.3.2#803003) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org