[
https://issues.apache.org/jira/browse/HDFS-6581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14138335#comment-14138335
]
Arpit Agarwal commented on HDFS-6581:
-------------------------------------
bq. Before we merged HDFS-4949 we did substantial benchmarking to show
performance improvements. I'd like to see some similar benchmarks here before
we can consider merging this branch.
I don't recall any performance numbers were published before merging to trunk.
Looking through the Jira even now I don't see any numbers. Maybe it's there -
feel free to give a pointer.
The only brief mention of performance is in the test plan where you cite
marginal improvement for MapReduce but no numbers. That said I don't want the
conversation to get stuck on performance numbers. I am also surprised to hear
you talking about design issues at this stage. The current direction is based
on broad discussion with the community here and on HDFS-5851, and the overall
consensus was to keep DataTransferProtocol in the picture for now. We cannot
get close to memory bandwidth without 'short-circuit writes', which is our
eventual plan, and the current design is a step in that direction.
> Write to single replica in memory
> ---------------------------------
>
> Key: HDFS-6581
> URL: https://issues.apache.org/jira/browse/HDFS-6581
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: datanode
> Reporter: Arpit Agarwal
> Assignee: Arpit Agarwal
> Attachments: HDFS-6581.merge.01.patch, HDFS-6581.merge.02.patch,
> HDFS-6581.merge.03.patch, HDFSWriteableReplicasInMemory.pdf,
> Test-Plan-for-HDFS-6581-Memory-Storage.pdf
>
>
> Per discussion with the community on HDFS-5851, we will implement writing to
> a single replica in DN memory via DataTransferProtocol.
> This avoids some of the issues with short-circuit writes, which we can
> revisit at a later time.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
