Cache groups were implemented for a sole purpose - to hide internal
inefficiencies. Namely (add more if I missed something):
1) Excessive heap usage for affinity/partition data
2) Too much data files as we employ file-per-partition approach.

These problems were resolved, but now cache groups are a great source of
confusion both for users and us - hard to understand, no way to configure
it in deterministic way. Should we resolve mentioned performance issues we
would never had cache groups. I propose to think we would it take for us to
get rid of cache groups.

Please provide your inputs to suggestions below.

1) "Merge" partition data from different caches
Consider that we start a new cache with the same affinity configuration
(cache mode, partition number, affinity function) as some of already
existing caches, Is it possible to re-use partition distribution and
history of existing cache for a new cache? Think of it as a kind of
automatic cache grouping which is transparent to the user. This would
remove heap pressure. Also it could resolve our long-standing issue with
FairAffinityFunction when tow caches with the same affinity configuration
are not co-located when started on different topology versions.

2) Employ segment-extent based approach instead of file-per-partition
- Every object (cache, index) reside in dedicated segment
- Segment consists of extents (minimal allocation units)
- Extents are allocated and deallocated as needed
- *Ignite specific*: particular extent can be used by only one partition
- Segments may be located in any number of data files we find convenient
With this approach "too many fsyncs" problem goes away automatically. At
the same time it would be possible to implement efficient rebalance still
as partition data will be split across moderate number of extents, not

Once we have p.1 and p.2 ready cache groups could be removed, couldn't they?


Reply via email to