[
https://issues.apache.org/jira/browse/HBASE-20003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16373420#comment-16373420
]
Andrew Purtell commented on HBASE-20003:
----------------------------------------
Can't assume in a failure the primary (or even any replica at all) will
survive.
> WALLess HBase on Persistent Memory
> ----------------------------------
>
> Key: HBASE-20003
> URL: https://issues.apache.org/jira/browse/HBASE-20003
> Project: HBase
> Issue Type: New Feature
> Reporter: Anoop Sam John
> Assignee: Anoop Sam John
> Priority: Major
>
> This JIRA aims to make use of persistent memory (pmem) technologies in HBase.
> One such usage is to make the Memstore to reside on pmem. Making a persistent
> memstore would remove the need for WAL and paves way for a WALLess HBase.
> The existing region replica feature could be used here and ensure the data
> written to memstores are synchronously replicated to the replicas and ensure
> strong consistency of the data. (pipeline model)
> Advantages :
> - Data Availability : Since the data across replicas are consistent
> (synchronously written) our data is always 100 % available.
> - Lower MTTR : It becomes easier/faster to switch over to the replicas on a
> primary region failure as there is no WAL replay involved. Building the
> memstore map data also is much faster than reading the WAL and replaying the
> WAL.
> - Possibility of bigger memstores : These pmems are designed to have more
> memory than DRAMs so it would also enable us to have bigger sized memstores
> which leads to lesser flushes/compaction IO.
> - Removes the dependency of HDFS on the write path
> Initial PoC has been designed and developed. Testing is underway and we would
> publish the PoC results along with the design doc sooner. The PoC doc will
> talk about the design decisions, the libraries considered to work with these
> pmem devices, pros and cons of those libraries and the performance results.
> Note : Next gen memory technologies using 3DXPoint gives persistent memory
> feature. Such memory DIMMs are soon to appear in the market. The PoC is done
> around Intel's ApachePass (AEP)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
