[
https://issues.apache.org/jira/browse/IGNITE-6854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mulugeta Mammo updated IGNITE-6854:
-----------------------------------
Description:
Ignite, when persistence mode is enabled, stores data and indexes on disk. To
minimize the latency of disks, several tuning options can be applied. Setting
the page size of a memory region to match the page size of the underlying
storage, using a separate disk for the WAL, and using production-level SSDs are
just a few of them [
https://apacheignite.readme.io/docs/durable-memory-tuning#section-native-persistence-related-tuning
].
A persistent memory store with low latency and high capacity offers a viable
alternative to disks. In light of this, we are proposing to make use of our Low
Level Persistent Library (LLPL), https://github.com/pmem/pcj/tree/master/LLPL,
to offer a persistent memory storage for Ignite.
At this point, we envision two distinct implementation options:
# Data and indexes will continue to be stored in the off-heap memory but the
disk will be replaced by a persistent memory. Since persistence memory in this
option is not a file system, the logic currently offered by WAL file and the
partition files would have to be implemented from scratch.
# In this option, we eliminate the current check-point process and the WAL
file. We will use a memory region defined by LLPL to store data and indexes.
There will be no off-heap memory. DRAM will be exclusively used to store hot
cache entries just like the on-heap cache is in the current implementation.
In both cases, there are more details and subtleties that have to handled –
e.g. the atomic and transactional guarantees offered. More clarifications will
be offered as we go along.
was:
Ignite, when persistence mode is enabled, stores data and indexes on disk. To
minimize the latency of disks, several tuning options can be applied. Setting
the page size of a memory region to match the page size of the underlying
storage, using a separate disk for the WAL, and using production-level SSDs are
just a few of them [
https://apacheignite.readme.io/docs/durable-memory-tuning#section-native-persistence-related-tuning
].
A persistent memory store with low latency and high capacity offers a viable
alternative to disks. In light of this, we are proposing to make use of our Low
Level Persistent Library (LLPL), https://github.com/pmem/pcj/tree/master/LLPL,
to offer a persistent memory storage for Ignite.
At this point, we envision two distinct implementation options:
# Data and indexes will continue to be stored in the off-heap memory but the
disk will be replaced by a persistent memory. Since persistence memory in this
option is not a file system, the logic currently offered by WAL file and the
partition files would have to be implemented from scratch.
#
# In this option, we eliminate the current check-point process and the WAL
file. We will use a memory region defined by LLPL to store data and indexes.
There will be no off-heap memory. DRAM will be exclusively used to store hot
cache entries just like the on-heap cache is in the current implementation.
#
In both cases, there are more details and subtleties that have to handled –
e.g. the atomic and transactional guarantees offered. More clarifications will
be offered as we go along.
> Enabling Persistent Memory for Ignite
> -------------------------------------
>
> Key: IGNITE-6854
> URL: https://issues.apache.org/jira/browse/IGNITE-6854
> Project: Ignite
> Issue Type: New Feature
> Security Level: Public(Viewable by anyone)
> Affects Versions: 2.3
> Reporter: Mulugeta Mammo
> Fix For: 2.4
>
>
> Ignite, when persistence mode is enabled, stores data and indexes on disk. To
> minimize the latency of disks, several tuning options can be applied. Setting
> the page size of a memory region to match the page size of the underlying
> storage, using a separate disk for the WAL, and using production-level SSDs
> are just a few of them [
> https://apacheignite.readme.io/docs/durable-memory-tuning#section-native-persistence-related-tuning
> ].
>
> A persistent memory store with low latency and high capacity offers a viable
> alternative to disks. In light of this, we are proposing to make use of our
> Low Level Persistent Library (LLPL),
> https://github.com/pmem/pcj/tree/master/LLPL, to offer a persistent memory
> storage for Ignite.
>
> At this point, we envision two distinct implementation options:
>
> # Data and indexes will continue to be stored in the off-heap memory but the
> disk will be replaced by a persistent memory. Since persistence memory in
> this option is not a file system, the logic currently offered by WAL file and
> the partition files would have to be implemented from scratch.
> # In this option, we eliminate the current check-point process and the WAL
> file. We will use a memory region defined by LLPL to store data and indexes.
> There will be no off-heap memory. DRAM will be exclusively used to store hot
> cache entries just like the on-heap cache is in the current implementation.
>
>
> In both cases, there are more details and subtleties that have to handled –
> e.g. the atomic and transactional guarantees offered. More clarifications
> will be offered as we go along.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
