[
https://issues.apache.org/jira/browse/HDFS-6581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14106593#comment-14106593
]
Arpit Agarwal commented on HDFS-6581:
-------------------------------------
bq. Memory pressure is worse with mmaped files because we cannot control the
timing of when the pages will be freed. We can evict pages from memory via
unmap faster than the memory manager can write them to disk. tmpfs has better
characteristics, once we run into the configured limit we can just stop
allocating more blocks in memory. A related optimization I'd really like to
have is to use unbuffered IO when writing to block files on disk so we don't
churn buffer cache.
Also our initial proposal on HDFS-5851 was to use mmapped files. However using
a RAM disk allows fairly good control over memory usage on the write path with
the least effort.
> Write to single replica in memory
> ---------------------------------
>
> Key: HDFS-6581
> URL: https://issues.apache.org/jira/browse/HDFS-6581
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: datanode
> Reporter: Arpit Agarwal
> Assignee: Arpit Agarwal
> Attachments: HDFSWriteableReplicasInMemory.pdf
>
>
> Per discussion with the community on HDFS-5851, we will implement writing to
> a single replica in DN memory via DataTransferProtocol.
> This avoids some of the issues with short-circuit writes, which we can
> revisit at a later time.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
