[
https://issues.apache.org/jira/browse/HDFS-6581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14138223#comment-14138223
]
Arpit Agarwal commented on HDFS-6581:
-------------------------------------
Thanks for the note. We have been busy getting the basic feature in and
stabilizing the branch. I intend to call a merge vote for the branch later this
week. Most of the things you mention are somewhat open ended tasks and we plan
to address post-merge:
# Benchmarking writes to memory. This might suggest further work to improve the
data write pipeline.
# Benchmark the impact of CRC computation, evaluate moving it off the hot path.
# Improve eviction, there's multiple ideas floating around, including
integration with CCM.
Contributions would be welcome on any of these.
bq. It would be nice to see that would could get a substantial fraction of
memory bandwidth when writing to a single replica in-memory.
The comparison will be interesting but I can tell you without measurement it is
not going to be a substantial fraction of memory bandwidth. We are still going
through DataTransferProtocol with all the copies and overhead that involves.
> Write to single replica in memory
> ---------------------------------
>
> Key: HDFS-6581
> URL: https://issues.apache.org/jira/browse/HDFS-6581
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: datanode
> Reporter: Arpit Agarwal
> Assignee: Arpit Agarwal
> Attachments: HDFS-6581.merge.01.patch, HDFS-6581.merge.02.patch,
> HDFS-6581.merge.03.patch, HDFSWriteableReplicasInMemory.pdf,
> Test-Plan-for-HDFS-6581-Memory-Storage.pdf
>
>
> Per discussion with the community on HDFS-5851, we will implement writing to
> a single replica in DN memory via DataTransferProtocol.
> This avoids some of the issues with short-circuit writes, which we can
> revisit at a later time.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
