[jira] [Updated] (IGNITE-6854) Enabling Persistent Memory for Ignite
[ https://issues.apache.org/jira/browse/IGNITE-6854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Plekhanov updated IGNITE-6854: -- Fix Version/s: (was: 2.9) 2.10 > Enabling Persistent Memory for Ignite > - > > Key: IGNITE-6854 > URL: https://issues.apache.org/jira/browse/IGNITE-6854 > Project: Ignite > Issue Type: New Feature >Affects Versions: 2.3 >Reporter: Mulugeta Mammo >Priority: Major > Fix For: 2.10 > > > Ignite, when persistence mode is enabled, stores data and indexes on disk. To > minimize the latency of disks, several tuning options can be applied. Setting > the page size of a memory region to match the page size of the underlying > storage, using a separate disk for the WAL, and using production-level SSDs > are just a few of them [ > https://apacheignite.readme.io/docs/durable-memory-tuning#section-native-persistence-related-tuning > ]. > > A persistent memory store with low latency and high capacity offers a viable > alternative to disks. In light of this, we are proposing to make use of our > Low Level Persistent Library (LLPL), > https://github.com/pmem/pcj/tree/master/LLPL, to offer a persistent memory > storage for Ignite. > > At this point, we envision two distinct implementation options: > # Data and indexes will continue to be stored in the off-heap memory but the > disk will be replaced by a persistent memory. Since persistence memory in > this option is not a file system, the logic currently offered by WAL file and > the partition files would have to be implemented from scratch. > # In this option, we eliminate the current check-point process and the WAL > file. We will use a memory region defined by LLPL to store data and indexes. > There will be no off-heap memory. DRAM will be exclusively used to store hot > cache entries just like the on-heap cache is in the current implementation. > > > In both cases, there are more details and subtleties that have to handled – > e.g. the atomic and transactional guarantees offered. More clarifications > will be given as we go along. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-6854) Enabling Persistent Memory for Ignite
[ https://issues.apache.org/jira/browse/IGNITE-6854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-6854: Fix Version/s: (was: 2.8) 2.9 > Enabling Persistent Memory for Ignite > - > > Key: IGNITE-6854 > URL: https://issues.apache.org/jira/browse/IGNITE-6854 > Project: Ignite > Issue Type: New Feature >Affects Versions: 2.3 >Reporter: Mulugeta Mammo >Priority: Major > Fix For: 2.9 > > > Ignite, when persistence mode is enabled, stores data and indexes on disk. To > minimize the latency of disks, several tuning options can be applied. Setting > the page size of a memory region to match the page size of the underlying > storage, using a separate disk for the WAL, and using production-level SSDs > are just a few of them [ > https://apacheignite.readme.io/docs/durable-memory-tuning#section-native-persistence-related-tuning > ]. > > A persistent memory store with low latency and high capacity offers a viable > alternative to disks. In light of this, we are proposing to make use of our > Low Level Persistent Library (LLPL), > https://github.com/pmem/pcj/tree/master/LLPL, to offer a persistent memory > storage for Ignite. > > At this point, we envision two distinct implementation options: > # Data and indexes will continue to be stored in the off-heap memory but the > disk will be replaced by a persistent memory. Since persistence memory in > this option is not a file system, the logic currently offered by WAL file and > the partition files would have to be implemented from scratch. > # In this option, we eliminate the current check-point process and the WAL > file. We will use a memory region defined by LLPL to store data and indexes. > There will be no off-heap memory. DRAM will be exclusively used to store hot > cache entries just like the on-heap cache is in the current implementation. > > > In both cases, there are more details and subtleties that have to handled – > e.g. the atomic and transactional guarantees offered. More clarifications > will be given as we go along. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-6854) Enabling Persistent Memory for Ignite
[ https://issues.apache.org/jira/browse/IGNITE-6854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Ozerov updated IGNITE-6854: Fix Version/s: (was: 2.7) 2.8 > Enabling Persistent Memory for Ignite > - > > Key: IGNITE-6854 > URL: https://issues.apache.org/jira/browse/IGNITE-6854 > Project: Ignite > Issue Type: New Feature >Affects Versions: 2.3 >Reporter: Mulugeta Mammo >Priority: Major > Fix For: 2.8 > > > Ignite, when persistence mode is enabled, stores data and indexes on disk. To > minimize the latency of disks, several tuning options can be applied. Setting > the page size of a memory region to match the page size of the underlying > storage, using a separate disk for the WAL, and using production-level SSDs > are just a few of them [ > https://apacheignite.readme.io/docs/durable-memory-tuning#section-native-persistence-related-tuning > ]. > > A persistent memory store with low latency and high capacity offers a viable > alternative to disks. In light of this, we are proposing to make use of our > Low Level Persistent Library (LLPL), > https://github.com/pmem/pcj/tree/master/LLPL, to offer a persistent memory > storage for Ignite. > > At this point, we envision two distinct implementation options: > # Data and indexes will continue to be stored in the off-heap memory but the > disk will be replaced by a persistent memory. Since persistence memory in > this option is not a file system, the logic currently offered by WAL file and > the partition files would have to be implemented from scratch. > # In this option, we eliminate the current check-point process and the WAL > file. We will use a memory region defined by LLPL to store data and indexes. > There will be no off-heap memory. DRAM will be exclusively used to store hot > cache entries just like the on-heap cache is in the current implementation. > > > In both cases, there are more details and subtleties that have to handled – > e.g. the atomic and transactional guarantees offered. More clarifications > will be given as we go along. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-6854) Enabling Persistent Memory for Ignite
[ https://issues.apache.org/jira/browse/IGNITE-6854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov updated IGNITE-6854: Fix Version/s: (was: 2.8) 2.7 > Enabling Persistent Memory for Ignite > - > > Key: IGNITE-6854 > URL: https://issues.apache.org/jira/browse/IGNITE-6854 > Project: Ignite > Issue Type: New Feature >Affects Versions: 2.3 >Reporter: Mulugeta Mammo >Priority: Major > Fix For: 2.7 > > > Ignite, when persistence mode is enabled, stores data and indexes on disk. To > minimize the latency of disks, several tuning options can be applied. Setting > the page size of a memory region to match the page size of the underlying > storage, using a separate disk for the WAL, and using production-level SSDs > are just a few of them [ > https://apacheignite.readme.io/docs/durable-memory-tuning#section-native-persistence-related-tuning > ]. > > A persistent memory store with low latency and high capacity offers a viable > alternative to disks. In light of this, we are proposing to make use of our > Low Level Persistent Library (LLPL), > https://github.com/pmem/pcj/tree/master/LLPL, to offer a persistent memory > storage for Ignite. > > At this point, we envision two distinct implementation options: > # Data and indexes will continue to be stored in the off-heap memory but the > disk will be replaced by a persistent memory. Since persistence memory in > this option is not a file system, the logic currently offered by WAL file and > the partition files would have to be implemented from scratch. > # In this option, we eliminate the current check-point process and the WAL > file. We will use a memory region defined by LLPL to store data and indexes. > There will be no off-heap memory. DRAM will be exclusively used to store hot > cache entries just like the on-heap cache is in the current implementation. > > > In both cases, there are more details and subtleties that have to handled – > e.g. the atomic and transactional guarantees offered. More clarifications > will be given as we go along. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-6854) Enabling Persistent Memory for Ignite
[ https://issues.apache.org/jira/browse/IGNITE-6854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov updated IGNITE-6854: Fix Version/s: (was: 2.7) 2.8 > Enabling Persistent Memory for Ignite > - > > Key: IGNITE-6854 > URL: https://issues.apache.org/jira/browse/IGNITE-6854 > Project: Ignite > Issue Type: New Feature >Affects Versions: 2.3 >Reporter: Mulugeta Mammo >Priority: Major > Fix For: 2.8 > > > Ignite, when persistence mode is enabled, stores data and indexes on disk. To > minimize the latency of disks, several tuning options can be applied. Setting > the page size of a memory region to match the page size of the underlying > storage, using a separate disk for the WAL, and using production-level SSDs > are just a few of them [ > https://apacheignite.readme.io/docs/durable-memory-tuning#section-native-persistence-related-tuning > ]. > > A persistent memory store with low latency and high capacity offers a viable > alternative to disks. In light of this, we are proposing to make use of our > Low Level Persistent Library (LLPL), > https://github.com/pmem/pcj/tree/master/LLPL, to offer a persistent memory > storage for Ignite. > > At this point, we envision two distinct implementation options: > # Data and indexes will continue to be stored in the off-heap memory but the > disk will be replaced by a persistent memory. Since persistence memory in > this option is not a file system, the logic currently offered by WAL file and > the partition files would have to be implemented from scratch. > # In this option, we eliminate the current check-point process and the WAL > file. We will use a memory region defined by LLPL to store data and indexes. > There will be no off-heap memory. DRAM will be exclusively used to store hot > cache entries just like the on-heap cache is in the current implementation. > > > In both cases, there are more details and subtleties that have to handled – > e.g. the atomic and transactional guarantees offered. More clarifications > will be given as we go along. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-6854) Enabling Persistent Memory for Ignite
[ https://issues.apache.org/jira/browse/IGNITE-6854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mulugeta Mammo updated IGNITE-6854: --- Attachment: (was: ignite_3DXPoint.patch) > Enabling Persistent Memory for Ignite > - > > Key: IGNITE-6854 > URL: https://issues.apache.org/jira/browse/IGNITE-6854 > Project: Ignite > Issue Type: New Feature >Affects Versions: 2.3 >Reporter: Mulugeta Mammo >Priority: Major > Fix For: 2.7 > > > Ignite, when persistence mode is enabled, stores data and indexes on disk. To > minimize the latency of disks, several tuning options can be applied. Setting > the page size of a memory region to match the page size of the underlying > storage, using a separate disk for the WAL, and using production-level SSDs > are just a few of them [ > https://apacheignite.readme.io/docs/durable-memory-tuning#section-native-persistence-related-tuning > ]. > > A persistent memory store with low latency and high capacity offers a viable > alternative to disks. In light of this, we are proposing to make use of our > Low Level Persistent Library (LLPL), > https://github.com/pmem/pcj/tree/master/LLPL, to offer a persistent memory > storage for Ignite. > > At this point, we envision two distinct implementation options: > # Data and indexes will continue to be stored in the off-heap memory but the > disk will be replaced by a persistent memory. Since persistence memory in > this option is not a file system, the logic currently offered by WAL file and > the partition files would have to be implemented from scratch. > # In this option, we eliminate the current check-point process and the WAL > file. We will use a memory region defined by LLPL to store data and indexes. > There will be no off-heap memory. DRAM will be exclusively used to store hot > cache entries just like the on-heap cache is in the current implementation. > > > In both cases, there are more details and subtleties that have to handled – > e.g. the atomic and transactional guarantees offered. More clarifications > will be given as we go along. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-6854) Enabling Persistent Memory for Ignite
[ https://issues.apache.org/jira/browse/IGNITE-6854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitriy Pavlov updated IGNITE-6854: --- Fix Version/s: (was: 2.6) 2.7 > Enabling Persistent Memory for Ignite > - > > Key: IGNITE-6854 > URL: https://issues.apache.org/jira/browse/IGNITE-6854 > Project: Ignite > Issue Type: New Feature >Affects Versions: 2.3 >Reporter: Mulugeta Mammo >Priority: Major > Fix For: 2.7 > > Attachments: ignite_3DXPoint.patch > > > Ignite, when persistence mode is enabled, stores data and indexes on disk. To > minimize the latency of disks, several tuning options can be applied. Setting > the page size of a memory region to match the page size of the underlying > storage, using a separate disk for the WAL, and using production-level SSDs > are just a few of them [ > https://apacheignite.readme.io/docs/durable-memory-tuning#section-native-persistence-related-tuning > ]. > > A persistent memory store with low latency and high capacity offers a viable > alternative to disks. In light of this, we are proposing to make use of our > Low Level Persistent Library (LLPL), > https://github.com/pmem/pcj/tree/master/LLPL, to offer a persistent memory > storage for Ignite. > > At this point, we envision two distinct implementation options: > # Data and indexes will continue to be stored in the off-heap memory but the > disk will be replaced by a persistent memory. Since persistence memory in > this option is not a file system, the logic currently offered by WAL file and > the partition files would have to be implemented from scratch. > # In this option, we eliminate the current check-point process and the WAL > file. We will use a memory region defined by LLPL to store data and indexes. > There will be no off-heap memory. DRAM will be exclusively used to store hot > cache entries just like the on-heap cache is in the current implementation. > > > In both cases, there are more details and subtleties that have to handled – > e.g. the atomic and transactional guarantees offered. More clarifications > will be given as we go along. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-6854) Enabling Persistent Memory for Ignite
[ https://issues.apache.org/jira/browse/IGNITE-6854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mulugeta Mammo updated IGNITE-6854: --- Attachment: ignite_3DXPoint.patch > Enabling Persistent Memory for Ignite > - > > Key: IGNITE-6854 > URL: https://issues.apache.org/jira/browse/IGNITE-6854 > Project: Ignite > Issue Type: New Feature >Affects Versions: 2.3 >Reporter: Mulugeta Mammo >Priority: Major > Fix For: 2.6 > > Attachments: ignite_3DXPoint.patch > > > Ignite, when persistence mode is enabled, stores data and indexes on disk. To > minimize the latency of disks, several tuning options can be applied. Setting > the page size of a memory region to match the page size of the underlying > storage, using a separate disk for the WAL, and using production-level SSDs > are just a few of them [ > https://apacheignite.readme.io/docs/durable-memory-tuning#section-native-persistence-related-tuning > ]. > > A persistent memory store with low latency and high capacity offers a viable > alternative to disks. In light of this, we are proposing to make use of our > Low Level Persistent Library (LLPL), > https://github.com/pmem/pcj/tree/master/LLPL, to offer a persistent memory > storage for Ignite. > > At this point, we envision two distinct implementation options: > # Data and indexes will continue to be stored in the off-heap memory but the > disk will be replaced by a persistent memory. Since persistence memory in > this option is not a file system, the logic currently offered by WAL file and > the partition files would have to be implemented from scratch. > # In this option, we eliminate the current check-point process and the WAL > file. We will use a memory region defined by LLPL to store data and indexes. > There will be no off-heap memory. DRAM will be exclusively used to store hot > cache entries just like the on-heap cache is in the current implementation. > > > In both cases, there are more details and subtleties that have to handled – > e.g. the atomic and transactional guarantees offered. More clarifications > will be given as we go along. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-6854) Enabling Persistent Memory for Ignite
[ https://issues.apache.org/jira/browse/IGNITE-6854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Gura updated IGNITE-6854: Fix Version/s: (was: 2.5) 2.6 > Enabling Persistent Memory for Ignite > - > > Key: IGNITE-6854 > URL: https://issues.apache.org/jira/browse/IGNITE-6854 > Project: Ignite > Issue Type: New Feature >Affects Versions: 2.3 >Reporter: Mulugeta Mammo >Priority: Major > Fix For: 2.6 > > > Ignite, when persistence mode is enabled, stores data and indexes on disk. To > minimize the latency of disks, several tuning options can be applied. Setting > the page size of a memory region to match the page size of the underlying > storage, using a separate disk for the WAL, and using production-level SSDs > are just a few of them [ > https://apacheignite.readme.io/docs/durable-memory-tuning#section-native-persistence-related-tuning > ]. > > A persistent memory store with low latency and high capacity offers a viable > alternative to disks. In light of this, we are proposing to make use of our > Low Level Persistent Library (LLPL), > https://github.com/pmem/pcj/tree/master/LLPL, to offer a persistent memory > storage for Ignite. > > At this point, we envision two distinct implementation options: > # Data and indexes will continue to be stored in the off-heap memory but the > disk will be replaced by a persistent memory. Since persistence memory in > this option is not a file system, the logic currently offered by WAL file and > the partition files would have to be implemented from scratch. > # In this option, we eliminate the current check-point process and the WAL > file. We will use a memory region defined by LLPL to store data and indexes. > There will be no off-heap memory. DRAM will be exclusively used to store hot > cache entries just like the on-heap cache is in the current implementation. > > > In both cases, there are more details and subtleties that have to handled – > e.g. the atomic and transactional guarantees offered. More clarifications > will be given as we go along. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-6854) Enabling Persistent Memory for Ignite
[ https://issues.apache.org/jira/browse/IGNITE-6854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Ozerov updated IGNITE-6854: Fix Version/s: (was: 2.4) 2.5 > Enabling Persistent Memory for Ignite > - > > Key: IGNITE-6854 > URL: https://issues.apache.org/jira/browse/IGNITE-6854 > Project: Ignite > Issue Type: New Feature >Affects Versions: 2.3 >Reporter: Mulugeta Mammo >Priority: Major > Fix For: 2.5 > > > Ignite, when persistence mode is enabled, stores data and indexes on disk. To > minimize the latency of disks, several tuning options can be applied. Setting > the page size of a memory region to match the page size of the underlying > storage, using a separate disk for the WAL, and using production-level SSDs > are just a few of them [ > https://apacheignite.readme.io/docs/durable-memory-tuning#section-native-persistence-related-tuning > ]. > > A persistent memory store with low latency and high capacity offers a viable > alternative to disks. In light of this, we are proposing to make use of our > Low Level Persistent Library (LLPL), > https://github.com/pmem/pcj/tree/master/LLPL, to offer a persistent memory > storage for Ignite. > > At this point, we envision two distinct implementation options: > # Data and indexes will continue to be stored in the off-heap memory but the > disk will be replaced by a persistent memory. Since persistence memory in > this option is not a file system, the logic currently offered by WAL file and > the partition files would have to be implemented from scratch. > # In this option, we eliminate the current check-point process and the WAL > file. We will use a memory region defined by LLPL to store data and indexes. > There will be no off-heap memory. DRAM will be exclusively used to store hot > cache entries just like the on-heap cache is in the current implementation. > > > In both cases, there are more details and subtleties that have to handled – > e.g. the atomic and transactional guarantees offered. More clarifications > will be given as we go along. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-6854) Enabling Persistent Memory for Ignite
[ https://issues.apache.org/jira/browse/IGNITE-6854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mulugeta Mammo updated IGNITE-6854: --- Description: Ignite, when persistence mode is enabled, stores data and indexes on disk. To minimize the latency of disks, several tuning options can be applied. Setting the page size of a memory region to match the page size of the underlying storage, using a separate disk for the WAL, and using production-level SSDs are just a few of them [ https://apacheignite.readme.io/docs/durable-memory-tuning#section-native-persistence-related-tuning ]. A persistent memory store with low latency and high capacity offers a viable alternative to disks. In light of this, we are proposing to make use of our Low Level Persistent Library (LLPL), https://github.com/pmem/pcj/tree/master/LLPL, to offer a persistent memory storage for Ignite. At this point, we envision two distinct implementation options: # Data and indexes will continue to be stored in the off-heap memory but the disk will be replaced by a persistent memory. Since persistence memory in this option is not a file system, the logic currently offered by WAL file and the partition files would have to be implemented from scratch. # In this option, we eliminate the current check-point process and the WAL file. We will use a memory region defined by LLPL to store data and indexes. There will be no off-heap memory. DRAM will be exclusively used to store hot cache entries just like the on-heap cache is in the current implementation. In both cases, there are more details and subtleties that have to handled – e.g. the atomic and transactional guarantees offered. More clarifications will be given as we go along. was: Ignite, when persistence mode is enabled, stores data and indexes on disk. To minimize the latency of disks, several tuning options can be applied. Setting the page size of a memory region to match the page size of the underlying storage, using a separate disk for the WAL, and using production-level SSDs are just a few of them [ https://apacheignite.readme.io/docs/durable-memory-tuning#section-native-persistence-related-tuning ]. A persistent memory store with low latency and high capacity offers a viable alternative to disks. In light of this, we are proposing to make use of our Low Level Persistent Library (LLPL), https://github.com/pmem/pcj/tree/master/LLPL, to offer a persistent memory storage for Ignite. At this point, we envision two distinct implementation options: # Data and indexes will continue to be stored in the off-heap memory but the disk will be replaced by a persistent memory. Since persistence memory in this option is not a file system, the logic currently offered by WAL file and the partition files would have to be implemented from scratch. # In this option, we eliminate the current check-point process and the WAL file. We will use a memory region defined by LLPL to store data and indexes. There will be no off-heap memory. DRAM will be exclusively used to store hot cache entries just like the on-heap cache is in the current implementation. In both cases, there are more details and subtleties that have to handled – e.g. the atomic and transactional guarantees offered. More clarifications will be offered as we go along. > Enabling Persistent Memory for Ignite > - > > Key: IGNITE-6854 > URL: https://issues.apache.org/jira/browse/IGNITE-6854 > Project: Ignite > Issue Type: New Feature > Security Level: Public(Viewable by anyone) >Affects Versions: 2.3 >Reporter: Mulugeta Mammo > Fix For: 2.4 > > > Ignite, when persistence mode is enabled, stores data and indexes on disk. To > minimize the latency of disks, several tuning options can be applied. Setting > the page size of a memory region to match the page size of the underlying > storage, using a separate disk for the WAL, and using production-level SSDs > are just a few of them [ > https://apacheignite.readme.io/docs/durable-memory-tuning#section-native-persistence-related-tuning > ]. > > A persistent memory store with low latency and high capacity offers a viable > alternative to disks. In light of this, we are proposing to make use of our > Low Level Persistent Library (LLPL), > https://github.com/pmem/pcj/tree/master/LLPL, to offer a persistent memory > storage for Ignite. > > At this point, we envision two distinct implementation options: > # Data and indexes will continue to be stored in the off-heap memory but the > disk will be replaced by a persistent memory. Since persistence memory in > this option is not a file system, the logic currently offered by WAL file and > the partition files would have to be implemented from scratch. > # In this option, we eliminate the current
[jira] [Updated] (IGNITE-6854) Enabling Persistent Memory for Ignite
[ https://issues.apache.org/jira/browse/IGNITE-6854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mulugeta Mammo updated IGNITE-6854: --- Description: Ignite, when persistence mode is enabled, stores data and indexes on disk. To minimize the latency of disks, several tuning options can be applied. Setting the page size of a memory region to match the page size of the underlying storage, using a separate disk for the WAL, and using production-level SSDs are just a few of them [ https://apacheignite.readme.io/docs/durable-memory-tuning#section-native-persistence-related-tuning ]. A persistent memory store with low latency and high capacity offers a viable alternative to disks. In light of this, we are proposing to make use of our Low Level Persistent Library (LLPL), https://github.com/pmem/pcj/tree/master/LLPL, to offer a persistent memory storage for Ignite. At this point, we envision two distinct implementation options: # Data and indexes will continue to be stored in the off-heap memory but the disk will be replaced by a persistent memory. Since persistence memory in this option is not a file system, the logic currently offered by WAL file and the partition files would have to be implemented from scratch. # In this option, we eliminate the current check-point process and the WAL file. We will use a memory region defined by LLPL to store data and indexes. There will be no off-heap memory. DRAM will be exclusively used to store hot cache entries just like the on-heap cache is in the current implementation. In both cases, there are more details and subtleties that have to handled – e.g. the atomic and transactional guarantees offered. More clarifications will be offered as we go along. was: Ignite, when persistence mode is enabled, stores data and indexes on disk. To minimize the latency of disks, several tuning options can be applied. Setting the page size of a memory region to match the page size of the underlying storage, using a separate disk for the WAL, and using production-level SSDs are just a few of them [ https://apacheignite.readme.io/docs/durable-memory-tuning#section-native-persistence-related-tuning ]. A persistent memory store with low latency and high capacity offers a viable alternative to disks. In light of this, we are proposing to make use of our Low Level Persistent Library (LLPL), https://github.com/pmem/pcj/tree/master/LLPL, to offer a persistent memory storage for Ignite. At this point, we envision two distinct implementation options: # Data and indexes will continue to be stored in the off-heap memory but the disk will be replaced by a persistent memory. Since persistence memory in this option is not a file system, the logic currently offered by WAL file and the partition files would have to be implemented from scratch. # In this option, we eliminate the current check-point process and the WAL file. We will use a memory region defined by LLPL to store data and indexes. There will be no off-heap memory. DRAM will be exclusively used to store hot cache entries just like the on-heap cache is in the current implementation. In both cases, there are more details and subtleties that have to handled – e.g. the atomic and transactional guarantees offered. More clarifications will be offered as we go along. > Enabling Persistent Memory for Ignite > - > > Key: IGNITE-6854 > URL: https://issues.apache.org/jira/browse/IGNITE-6854 > Project: Ignite > Issue Type: New Feature > Security Level: Public(Viewable by anyone) >Affects Versions: 2.3 >Reporter: Mulugeta Mammo > Fix For: 2.4 > > > Ignite, when persistence mode is enabled, stores data and indexes on disk. To > minimize the latency of disks, several tuning options can be applied. Setting > the page size of a memory region to match the page size of the underlying > storage, using a separate disk for the WAL, and using production-level SSDs > are just a few of them [ > https://apacheignite.readme.io/docs/durable-memory-tuning#section-native-persistence-related-tuning > ]. > > A persistent memory store with low latency and high capacity offers a viable > alternative to disks. In light of this, we are proposing to make use of our > Low Level Persistent Library (LLPL), > https://github.com/pmem/pcj/tree/master/LLPL, to offer a persistent memory > storage for Ignite. > > At this point, we envision two distinct implementation options: > # Data and indexes will continue to be stored in the off-heap memory but the > disk will be replaced by a persistent memory. Since persistence memory in > this option is not a file system, the logic currently offered by WAL file and > the partition files would have to be implemented from scratch. > # In this option, we eliminate the current
[jira] [Updated] (IGNITE-6854) Enabling Persistent Memory for Ignite
[ https://issues.apache.org/jira/browse/IGNITE-6854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mulugeta Mammo updated IGNITE-6854: --- Description: Ignite, when persistence mode is enabled, stores data and indexes on disk. To minimize the latency of disks, several tuning options can be applied. Setting the page size of a memory region to match the page size of the underlying storage, using a separate disk for the WAL, and using production-level SSDs are just a few of them [ https://apacheignite.readme.io/docs/durable-memory-tuning#section-native-persistence-related-tuning ]. A persistent memory store with low latency and high capacity offers a viable alternative to disks. In light of this, we are proposing to make use of our Low Level Persistent Library (LLPL), https://github.com/pmem/pcj/tree/master/LLPL, to offer a persistent memory storage for Ignite. At this point, we envision two distinct implementation options: # Data and indexes will continue to be stored in the off-heap memory but the disk will be replaced by a persistent memory. Since persistence memory in this option is not a file system, the logic currently offered by WAL file and the partition files would have to be implemented from scratch. # In this option, we eliminate the current check-point process and the WAL file. We will use a memory region defined by LLPL to store data and indexes. There will be no off-heap memory. DRAM will be exclusively used to store hot cache entries just like the on-heap cache is in the current implementation. In both cases, there are more details and subtleties that have to handled – e.g. the atomic and transactional guarantees offered. More clarifications will be offered as we go along. was: Ignite, when persistence mode is enabled, stores data and indexes on disk. To minimize the latency of disks, several tuning options can be applied. Setting the page size of a memory region to match the page size of the underlying storage, using a separate disk for the WAL, and using production-level SSDs are just a few of them [ https://apacheignite.readme.io/docs/durable-memory-tuning#section-native-persistence-related-tuning ]. A persistent memory store with low latency and high capacity offers a viable alternative to disks. In light of this, we are proposing to make use of our Low Level Persistent Library (LLPL), https://github.com/pmem/pcj/tree/master/LLPL, to offer a persistent memory storage for Ignite. At this point, we envision two distinct implementation options: # Data and indexes will continue to be stored in the off-heap memory but the disk will be replaced by a persistent memory. Since persistence memory in this option is not a file system, the logic currently offered by WAL file and the partition files would have to be implemented from scratch. # # In this option, we eliminate the current check-point process and the WAL file. We will use a memory region defined by LLPL to store data and indexes. There will be no off-heap memory. DRAM will be exclusively used to store hot cache entries just like the on-heap cache is in the current implementation. # In both cases, there are more details and subtleties that have to handled – e.g. the atomic and transactional guarantees offered. More clarifications will be offered as we go along. > Enabling Persistent Memory for Ignite > - > > Key: IGNITE-6854 > URL: https://issues.apache.org/jira/browse/IGNITE-6854 > Project: Ignite > Issue Type: New Feature > Security Level: Public(Viewable by anyone) >Affects Versions: 2.3 >Reporter: Mulugeta Mammo > Fix For: 2.4 > > > Ignite, when persistence mode is enabled, stores data and indexes on disk. To > minimize the latency of disks, several tuning options can be applied. Setting > the page size of a memory region to match the page size of the underlying > storage, using a separate disk for the WAL, and using production-level SSDs > are just a few of them [ > https://apacheignite.readme.io/docs/durable-memory-tuning#section-native-persistence-related-tuning > ]. > > A persistent memory store with low latency and high capacity offers a viable > alternative to disks. In light of this, we are proposing to make use of our > Low Level Persistent Library (LLPL), > https://github.com/pmem/pcj/tree/master/LLPL, to offer a persistent memory > storage for Ignite. > > At this point, we envision two distinct implementation options: > > # Data and indexes will continue to be stored in the off-heap memory but the > disk will be replaced by a persistent memory. Since persistence memory in > this option is not a file system, the logic currently offered by WAL file and > the partition files would have to be implemented from scratch. > # In this option, we eliminate the
