[
https://issues.apache.org/jira/browse/HDFS-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14007896#comment-14007896
]
Colin Patrick McCabe commented on HDFS-5851:
--------------------------------------------
Todd is correct here. It's the DataNode which pins the block file and metadata
file, and it can un-pin them if the client takes too long.
A few concerns:
* why a separate namespace under {{hdfs://<namespace>/.reserved/ddm}} ? We
have xattrs now, so files which have been created in the write cache could be
identified with a given (system) xattr.
* discarding files when memory gets tight. spilling them to disk is another.
we should have both policies available so that this can be useful in things
like Spark.
> Support memory as a storage medium
> ----------------------------------
>
> Key: HDFS-5851
> URL: https://issues.apache.org/jira/browse/HDFS-5851
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode
> Affects Versions: 3.0.0
> Reporter: Arpit Agarwal
> Assignee: Arpit Agarwal
> Attachments:
> SupportingMemoryStorageinHDFSPersistentandDiscardableMemory.pdf,
> SupportingMemoryStorageinHDFSPersistentandDiscardableMemory.pdf
>
>
> Memory can be used as a storage medium for smaller/transient files for fast
> write throughput.
> More information/design will be added later.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
