[
https://issues.apache.org/jira/browse/HDFS-6919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14148608#comment-14148608
]
Arpit Agarwal commented on HDFS-6919:
-------------------------------------
Colin, thinking ahead to this integration. Could you please describe what you
have in mind?
I think as a first step it would be good to have the caching logic on the DN
know the current RAM disk usage, as well as the RAM disk replica management
logic know how much memory is in use by caching. So if one feature is not in
use the other gets access to the entire memory. We'd at least avoid any memory
fragmentation issues. The total limit would be the
{{dfs.datanode.max.locked.memory}} or whatever we call it.
Longer term perhaps we can look at RAM disk manager evicting cold data from the
DN cache to make room for writes, and vice versa.
> Enforce a single limit for RAM disk usage and replicas cached via locking
> -------------------------------------------------------------------------
>
> Key: HDFS-6919
> URL: https://issues.apache.org/jira/browse/HDFS-6919
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Arpit Agarwal
> Assignee: Colin Patrick McCabe
> Priority: Blocker
>
> The DataNode can have a single limit for memory usage which applies to both
> replicas cached via CCM and replicas on RAM disk.
> See comments
> [1|https://issues.apache.org/jira/browse/HDFS-6581?focusedCommentId=14106025&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14106025],
>
> [2|https://issues.apache.org/jira/browse/HDFS-6581?focusedCommentId=14106245&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14106245]
> and
> [3|https://issues.apache.org/jira/browse/HDFS-6581?focusedCommentId=14106575&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14106575]
> for discussion.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
