Igniters, Cache groups were implemented for a sole purpose - to hide internal inefficiencies. Namely (add more if I missed something): 1) Excessive heap usage for affinity/partition data 2) Too much data files as we employ file-per-partition approach.
These problems were resolved, but now cache groups are a great source of confusion both for users and us - hard to understand, no way to configure it in deterministic way. Should we resolve mentioned performance issues we would never had cache groups. I propose to think we would it take for us to get rid of cache groups. Please provide your inputs to suggestions below. 1) "Merge" partition data from different caches Consider that we start a new cache with the same affinity configuration (cache mode, partition number, affinity function) as some of already existing caches, Is it possible to re-use partition distribution and history of existing cache for a new cache? Think of it as a kind of automatic cache grouping which is transparent to the user. This would remove heap pressure. Also it could resolve our long-standing issue with FairAffinityFunction when tow caches with the same affinity configuration are not co-located when started on different topology versions. 2) Employ segment-extent based approach instead of file-per-partition - Every object (cache, index) reside in dedicated segment - Segment consists of extents (minimal allocation units) - Extents are allocated and deallocated as needed - *Ignite specific*: particular extent can be used by only one partition - Segments may be located in any number of data files we find convenient With this approach "too many fsyncs" problem goes away automatically. At the same time it would be possible to implement efficient rebalance still as partition data will be split across moderate number of extents, not chaotically. Once we have p.1 and p.2 ready cache groups could be removed, couldn't they? Vladimir.
