[
https://issues.apache.org/jira/browse/HBASE-21874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16770097#comment-16770097
]
Wellington Chevreuil commented on HBASE-21874:
----------------------------------------------
Thanks for clarifying this, [~anoop.hbase] and [~ram_krish]! Summarising some
of the comments so far to see if we are getting to a common understanding:
{quote}
As a 1st step we thought of adding it an IOEngine almost same perf with off
heap engine and with lower cost!
{quote}
{quote}
Memory mode does not give us control as where the cache or the address space
could reside.
{quote}
If "Memory Mode" was used, current *ByteBufferIOEngine" could or could not
access the extra memory "transparently", but there would be no control over
whether it's storing on DRAM or PMem spaces.
Then:
{quote}
Anyway the cache persistence is valid if RS down gracefully. So that we can
persist the blocks info meta and on restart use that to rebuild the cache. So
short ans is yes still the persistence is there.
{quote}
{quote}
But here we specifically ask our cache to reside only on the Pmem area and once
it is mapped in the Pmem address space everything is transparent to us.
{quote}
{quote}
All the file based OS calls work normally here. The extra is we can do mmap and
the OS will not try to bring the pages into its Page cache (DRAM). Instead the
CPU can directly access bytes over the device with its load/store instructions.
For this the pmem library APIs also not a mandatory item.
{quote}
Thus, main goal of setting "Direct Mode" here for now is not to use the
persistence capabilities (although it's probably already working), but just
have a mean to guarantee we use space from PMem device for caching (and not
DRAM at all, which can't be guaranteed with "Memory Mode"). So theoretically,
we can already do this with the current FileMmapEngine, no?
> Bucket cache on Persistent memory
> ---------------------------------
>
> Key: HBASE-21874
> URL: https://issues.apache.org/jira/browse/HBASE-21874
> Project: HBase
> Issue Type: New Feature
> Components: BucketCache
> Affects Versions: 3.0.0
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-21874.patch, HBASE-21874.patch,
> HBASE-21874_V2.patch, Pmem_BC.png
>
>
> Non volatile persistent memory devices are byte addressable like DRAM (for
> eg. Intel DCPMM). Bucket cache implementation can take advantage of this new
> memory type and can make use of the existing offheap data structures to serve
> data directly from this memory area without having to bring the data to
> onheap.
> The patch is a new IOEngine implementation that works with the persistent
> memory.
> Note : Here we don't make use of the persistence nature of the device and
> just make use of the big memory it provides.
> Performance numbers to follow.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
