[
https://issues.apache.org/jira/browse/HDFS-6581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14146647#comment-14146647
]
Andrew Wang commented on HDFS-6581:
-----------------------------------
Thanks for running these numbers Arpit. To convert into MB/s, it seems like
we're going from 280 or 292 MB/s baseline to 455 or 487MBs with csums off.
Are you aware of any further copies that can be eliminated in DTP, or via
something like short-circuit writes? I'd be satisfied regarding the merge if we
had a pretty good estimate of the perf after such improvements are made. Saying
that we could go from e.g. 4 copies to 2 copies would suffice for this, better
would be results from a prototype.
> Write to single replica in memory
> ---------------------------------
>
> Key: HDFS-6581
> URL: https://issues.apache.org/jira/browse/HDFS-6581
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: datanode
> Reporter: Arpit Agarwal
> Assignee: Arpit Agarwal
> Attachments: HDFS-6581.merge.01.patch, HDFS-6581.merge.02.patch,
> HDFS-6581.merge.03.patch, HDFS-6581.merge.04.patch, HDFS-6581.merge.05.patch,
> HDFS-6581.merge.06.patch, HDFS-6581.merge.07.patch, HDFS-6581.merge.08.patch,
> HDFS-6581.merge.09.patch, HDFS-6581.merge.10.patch,
> HDFSWriteableReplicasInMemory.pdf, Test-Plan-for-HDFS-6581-Memory-Storage.pdf
>
>
> Per discussion with the community on HDFS-5851, we will implement writing to
> a single replica in DN memory via DataTransferProtocol.
> This avoids some of the issues with short-circuit writes, which we can
> revisit at a later time.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
