[
https://issues.apache.org/jira/browse/HDFS-13762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16789264#comment-16789264
]
Feilong He edited comment on HDFS-13762 at 3/13/19 6:36 AM:
------------------------------------------------------------
Thanks everyone for your valuable comments on this work.
After discussed with [~umamaheswararao] & [~rakeshr] offline, we thought of
splitting this JIRA into 4 subtasks as the above subtask section shows. A big
change to this work is that we are considering to add an implementation of SCM
cache by using mapped byte buffer, which will not depend on PMDK. And the PMDK
based implementation as the posted patch for this JIRA shows will still be
kept. Using which implementation can be configurable. Currently, the code is
under last refining.
Please feel free to comment if you have any other suggestions. Thanks!
was (Author: philohe):
Thanks everyone for your valuable comments on this work. After discussed with
[~umamaheswararao] & [~rakeshr], we decide to split this JIRA into 4 subtasks
as the above subtask section shows. A big change to this work is that we will
add an implementation of SCM cache by using mapped byte buffer, which will not
depend on PMDK. And the previous implementation with PMDK is still kept. Using
which implementation is configurable. Currently, the code is under refining. I
will upload the patches in these days.
> Support non-volatile storage class memory(SCM) in HDFS cache directives
> -----------------------------------------------------------------------
>
> Key: HDFS-13762
> URL: https://issues.apache.org/jira/browse/HDFS-13762
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: caching, datanode
> Reporter: Sammi Chen
> Assignee: Feilong He
> Priority: Major
> Attachments: HDFS-13762.000.patch, HDFS-13762.001.patch,
> HDFS-13762.002.patch, HDFS-13762.003.patch, HDFS-13762.004.patch,
> HDFS-13762.005.patch, HDFS-13762.006.patch, HDFS-13762.007.patch,
> HDFS-13762.008.patch, SCMCacheDesign-2018-11-08.pdf, SCMCacheTestPlan.pdf
>
>
> No-volatile storage class memory is a type of memory that can keep the data
> content after power failure or between the power cycle. Non-volatile storage
> class memory device usually has near access speed as memory DIMM while has
> lower cost than memory. So today It is usually used as a supplement to
> memory to hold long tern persistent data, such as data in cache.
> Currently in HDFS, we have OS page cache backed read only cache and RAMDISK
> based lazy write cache. Non-volatile memory suits for both these functions.
> This Jira aims to enable storage class memory first in read cache. Although
> storage class memory has non-volatile characteristics, to keep the same
> behavior as current read only cache, we don't use its persistent
> characteristics currently.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
