[jira] [Updated] (IGNITE-3477) Rework offheap storage
[
https://issues.apache.org/jira/browse/IGNITE-3477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Goncharuk updated IGNITE-3477:
-
Description:
Current (Ignite 1.x) implementation of cache storage relies on a tiered
approach where cache entry can be located in one of the three storage areas:
on-heap, off-heap memory and swap. It leads to the following disadvantages:
* Entry constantly moves from one storage area to another which leads to a
complex code (for example, swap/unswap listeners for queries and rebalancing)
* It is not possible to set per-cache memory limit
* Off-heap indexes require on-heap row cache. If this cache is small,
performance becomes very poor
* Continuous put/remove operations with OFFHEAP_TIERED mode lead to
uncontrollable memory fragmentation
We can reapproach the cache storage and base it on a concept of page memory. We
will allocate several memory pools of pages of fixed size and assign each cache
to one of the memory pools. All storage data structures should operate on
memory pages, the main storage will be always off-heap with an optional on-heap
cache.
This gives us the following benefits:
* Flexible and precise per-cache memory limit
* Transparent swap: we can allocate page memory over a memory-mapped file and
OS will take care of swapping
* Getting rid of on-heap cache for off-heap SQL indexes
* Ability to take a cluster-wide data snapshot
I suggest the following API for memory policy configuration:
{code}
class MemoryPolicy {
boolean default; // Require at least one memory policy to be default.
private String name;
private long size;
private String tmpFsPath; // If memory-mapped file should be used.
}
class CacheConfiguration {
private String memoryPolicyName;
}
{code}
was:
Current (Ignite 1.x) implementation of cache storage relies on a tiered
approach where cache entry can be located in one of the three storage areas:
on-heap, off-heap memory and swap. It leads to the following disadvantages:
* Entry constantly moves from one storage area to another which leads to a
complex code (for example, swap/unswap listeners for queries and rebalancing)
* It is not possible to set per-cache memory limit
* Off-heap indexes require on-heap row cache. If this cache is small,
performance becomes very poor
* Continuous put/remove operations with OFFHEAP_TIERED mode lead to
uncontrollable memory fragmentation
We can reapproach the cache storage and base it on a concept of page memory. We
will allocate several memory pools of pages of fixed size and assign each cache
to one of the memory pools. All storage data structures should operate on
memory pages, the main storage will be always off-heap with an optional on-heap
cache.
This gives us the following benefits:
* Flexible and precise per-cache memory limit
* Transparent swap: we can allocate page memory over a memory-mapped file and
OS will take care of swapping
* Getting rid of on-heap cache for off-heap SQL indexes
* Ability to take a cluster-wide data snapshot
> Rework offheap storage
> --
>
> Key: IGNITE-3477
> URL: https://issues.apache.org/jira/browse/IGNITE-3477
> Project: Ignite
> Issue Type: Task
> Components: cache
>Reporter: Alexey Goncharuk
> Labels: important
> Fix For: 2.0
>
>
> Current (Ignite 1.x) implementation of cache storage relies on a tiered
> approach where cache entry can be located in one of the three storage areas:
> on-heap, off-heap memory and swap. It leads to the following disadvantages:
> * Entry constantly moves from one storage area to another which leads to a
> complex code (for example, swap/unswap listeners for queries and rebalancing)
> * It is not possible to set per-cache memory limit
> * Off-heap indexes require on-heap row cache. If this cache is small,
> performance becomes very poor
> * Continuous put/remove operations with OFFHEAP_TIERED mode lead to
> uncontrollable memory fragmentation
> We can reapproach the cache storage and base it on a concept of page memory.
> We will allocate several memory pools of pages of fixed size and assign each
> cache to one of the memory pools. All storage data structures should operate
> on memory pages, the main storage will be always off-heap with an optional
> on-heap cache.
> This gives us the following benefits:
> * Flexible and precise per-cache memory limit
> * Transparent swap: we can allocate page memory over a memory-mapped file
> and OS will take care of swapping
> * Getting rid of on-heap cache for off-heap SQL indexes
> * Ability to take a cluster-wide data snapshot
> I suggest the following API for memory policy configuration:
> {code}
> class MemoryPolicy {
> boolean default; // Require at least one memory policy to be default.
> private String name;
[jira] [Updated] (IGNITE-3477) Rework offheap storage
[ https://issues.apache.org/jira/browse/IGNITE-3477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Goncharuk updated IGNITE-3477: - Assignee: (was: Alexey Goncharuk) > Rework offheap storage > -- > > Key: IGNITE-3477 > URL: https://issues.apache.org/jira/browse/IGNITE-3477 > Project: Ignite > Issue Type: Task > Components: cache >Reporter: Alexey Goncharuk > Labels: important > Fix For: 2.0 > > > Current (Ignite 1.x) implementation of cache storage relies on a tiered > approach where cache entry can be located in one of the three storage areas: > on-heap, off-heap memory and swap. It leads to the following disadvantages: > * Entry constantly moves from one storage area to another which leads to a > complex code (for example, swap/unswap listeners for queries and rebalancing) > * It is not possible to set per-cache memory limit > * Off-heap indexes require on-heap row cache. If this cache is small, > performance becomes very poor > * Continuous put/remove operations with OFFHEAP_TIERED mode lead to > uncontrollable memory fragmentation > We can reapproach the cache storage and base it on a concept of page memory. > We will allocate several memory pools of pages of fixed size and assign each > cache to one of the memory pools. All storage data structures should operate > on memory pages, the main storage will be always off-heap with an optional > on-heap cache. > This gives us the following benefits: > * Flexible and precise per-cache memory limit > * Transparent swap: we can allocate page memory over a memory-mapped file > and OS will take care of swapping > * Getting rid of on-heap cache for off-heap SQL indexes > * Ability to take a cluster-wide data snapshot -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (IGNITE-3477) Rework offheap storage
[ https://issues.apache.org/jira/browse/IGNITE-3477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denis Magda updated IGNITE-3477: Labels: important (was: ) > Rework offheap storage > -- > > Key: IGNITE-3477 > URL: https://issues.apache.org/jira/browse/IGNITE-3477 > Project: Ignite > Issue Type: Task > Components: cache >Reporter: Alexey Goncharuk >Assignee: Alexey Goncharuk > Labels: important > Fix For: 2.0 > > > Current (Ignite 1.x) implementation of cache storage relies on a tiered > approach where cache entry can be located in one of the three storage areas: > on-heap, off-heap memory and swap. It leads to the following disadvantages: > * Entry constantly moves from one storage area to another which leads to a > complex code (for example, swap/unswap listeners for queries and rebalancing) > * It is not possible to set per-cache memory limit > * Off-heap indexes require on-heap row cache. If this cache is small, > performance becomes very poor > * Continuous put/remove operations with OFFHEAP_TIERED mode lead to > uncontrollable memory fragmentation > We can reapproach the cache storage and base it on a concept of page memory. > We will allocate several memory pools of pages of fixed size and assign each > cache to one of the memory pools. All storage data structures should operate > on memory pages, the main storage will be always off-heap with an optional > on-heap cache. > This gives us the following benefits: > * Flexible and precise per-cache memory limit > * Transparent swap: we can allocate page memory over a memory-mapped file > and OS will take care of swapping > * Getting rid of on-heap cache for off-heap SQL indexes > * Ability to take a cluster-wide data snapshot -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (IGNITE-3477) Rework offheap storage
[ https://issues.apache.org/jira/browse/IGNITE-3477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Goncharuk updated IGNITE-3477: - Summary: Rework offheap storage (was: Rework offheap approach) > Rework offheap storage > -- > > Key: IGNITE-3477 > URL: https://issues.apache.org/jira/browse/IGNITE-3477 > Project: Ignite > Issue Type: Task > Components: cache >Reporter: Alexey Goncharuk > Fix For: 2.0 > > > Current (Ignite 1.x) implementation of cache storage relies on a tiered > approach where cache entry can be located in one of the three storage areas: > on-heap, off-heap memory and swap. It leads to the following disadvantages: > * Entry constantly moves from one storage area to another which leads to a > complex code (for example, swap/unswap listeners for queries and rebalancing) > * It is not possible to set per-cache memory limit > * Off-heap indexes require on-heap row cache. If this cache is small, > performance becomes very poor > * Continuous put/remove operations with OFFHEAP_TIERED mode lead to > uncontrollable memory fragmentation > We can reapproach the cache storage and base it on a concept of page memory. > We will allocate several memory pools of pages of fixed size and assign each > cache to one of the memory pools. All storage data structures should operate > on memory pages, the main storage will be always off-heap with an optional > on-heap cache. > This gives us the following benefits: > * Flexible and precise per-cache memory limit > * Transparent swap: we can allocate page memory over a memory-mapped file > and OS will take care of swapping > * Getting rid of on-heap cache for off-heap SQL indexes > * Ability to take a cluster-wide data snapshot -- This message was sent by Atlassian JIRA (v6.3.4#6332)
