[
https://issues.apache.org/jira/browse/HDFS-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14046247#comment-14046247
]
Henry Saputra commented on HDFS-5851:
-------------------------------------
HI [~sanjay.radia], I was looking at the JIRA and proposal and I have some
questions related to it:
1. I did not see where the memory will be allocated for the DDM proposal. Is it
similar to HDFS-4949 to use the memory from Datanode?
2. As for the APIs, would it be new Hadoop FS (Java) APIs or higher level
construct to store data in memory because it seemed that the proposal only
relying on file path to indicate trying to use in-memory cache
3. In the problem statement of the proposal seemed like there would be policy
to manage how data should be store in memory per application but I could not
find details about how to achieve it. Some applications may need to have quick
access to some small portion of data more significant (eg: newer time series
data) whereas some others may be need to store more (eg: large Hive query)
4. In term of discardability, what is the "eviction" policy for such data and
how control or fine tune it if needed.
Maybe it was discussed in the in-person happened before but I could not find it
in the meet summary.
Thanks for driving this new feature.
> Support memory as a storage medium
> ----------------------------------
>
> Key: HDFS-5851
> URL: https://issues.apache.org/jira/browse/HDFS-5851
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: datanode
> Affects Versions: 3.0.0
> Reporter: Arpit Agarwal
> Assignee: Arpit Agarwal
> Attachments:
> SupportingMemoryStorageinHDFSPersistentandDiscardableMemory.pdf,
> SupportingMemoryStorageinHDFSPersistentandDiscardableMemory.pdf,
> SupportingMemoryStorageinHDFSPersistentandDiscardableMemory.pdf
>
>
> Memory can be used as a storage medium for smaller/transient files for fast
> write throughput.
> More information/design will be added later.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
