Hi Dmitriy, We're still working on the documentation of LLPL. You may forward me your questions in the meantime and I'll try to answer them.
Thanks, Mulugeta -----Original Message----- From: Dmitriy Setrakyan [mailto:dsetrak...@apache.org] Sent: Thursday, November 16, 2017 1:26 PM To: dev@ignite.apache.org Subject: Re: Adding Persistent Memory Support for Ignite Hi Mulugeta, Where can I find documentation about LLPL to understand how memory and persistence are handled there? D. On Thu, Nov 16, 2017 at 7:16 AM, Mammo, Mulugeta <mulugeta.ma...@intel.com> wrote: > Hi all, > > > > Ignite, when persistence mode is enabled, stores data and indexes on disk. > To minimize the latency of disks, several tuning options can be applied. > Setting the page size of a memory region to match the page size of the > underlying storage, using a separate disk for the WAL, and using > production-level SSDs are just a few of them [ > https://apacheignite.readme.io/docs/durable-memory-tuning# > section-native-persistence-related-tuning ]. > > A persistent memory store with low latency and high capacity offers a > viable alternative to disks. In light of this, we are proposing to > make use of our Low Level Persistent Library (LLPL), > https://github.com/pmem/pcj/ tree/master/LLPL, to offer a persistent memory > storage for Ignite. > > At this point, we envision two distinct implementation options: > > 1. Data and indexes will continue to be stored in the off-heap memory > but the disk will be replaced by a persistent memory. Since > persistence memory in this option is not a file system, the logic > currently offered by WAL file and the partition files would have to be > implemented from scratch. > > 2. In this option, we eliminate the current check-point process and > the WAL file. We will use a memory region defined by LLPL to store > data and indexes. There will be no off-heap memory. DRAM will be > exclusively used to store hot cache entries just like the on-heap > cache is in the current implementation. > > In both cases, there are more details and subtleties that have to > handled > - e.g. the atomic and transactional guarantees offered. More > clarifications will be given as we go along. And, feel free to provide your > thoughts. > > Thanks, > Mulugeta > >
