[
https://issues.apache.org/jira/browse/HDFS-1783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13291861#comment-13291861
]
Todd Lipcon commented on HDFS-1783:
-----------------------------------
{code}
As for the complexity, I find it manageable... The pipelining as such has not
changed, only that the client opens up N pipelines on length 1.
Once this change is in, one could get fancier (for example 2 pipelines of
length 2 for 4 replicas, etc, or maybe we could open pipelines to multiple
clusters, etc).
{code}
My concern is less with the complexity of the actual writing and more with the
changes that will probably have to be done for failure handling, for the append
path, etc. At the least I'd probably want all of our write-pipeline tests to be
duplicated to run once in pipelined-write mode and once in parallel-write mode.
Maybe I'm just being paranoid, happy for more opinions here :)
> Ability for HDFS client to write replicas in parallel
> -----------------------------------------------------
>
> Key: HDFS-1783
> URL: https://issues.apache.org/jira/browse/HDFS-1783
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs client
> Reporter: dhruba borthakur
> Assignee: Lars Hofhansl
> Attachments: HDFS-1783-trunk-v2.patch, HDFS-1783-trunk-v3.patch,
> HDFS-1783-trunk-v4.patch, HDFS-1783-trunk.patch
>
>
> The current implementation of HDFS pipelines the writes to the three
> replicas. This introduces some latency for realtime latency sensitive
> applications. An alternate implementation that allows the client to write all
> replicas in parallel gives much better response times to these applications.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
