[
https://issues.apache.org/jira/browse/HDFS-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sanjay Radia updated HDFS-5851:
-------------------------------
Attachment: SupportingMemoryStorageinHDFSPersistentandDiscardableMemory.pdf
Please see attached document that identifies some use cases and a proposal for
using memory for intermediate data. We introduce the notion of Discardable
Distributed Memory (DDM) that exploit the property the data can be
reconstructed. Further, by using HDFS files as a backing store to which DDM
data is lazily written, we give the impression of much larger memory size and
also give the system an additional degree of freedom to manage the scarce
memory resource. The main implementation mechanism is memory-mapped files that
are lazily replicated; this mechanism provides weak-persistence which may have
other direct use cases beyond DDMs.
> Support memory as a storage medium
> ----------------------------------
>
> Key: HDFS-5851
> URL: https://issues.apache.org/jira/browse/HDFS-5851
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode
> Affects Versions: 3.0.0
> Reporter: Arpit Agarwal
> Assignee: Arpit Agarwal
> Attachments:
> SupportingMemoryStorageinHDFSPersistentandDiscardableMemory.pdf
>
>
> Memory can be used as a storage medium for smaller/transient files for fast
> write throughput.
> More information/design will be added later.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
