Re: Remove cache groups in AI 3.0

Vova, thanks for explanations. Comments really valuable to me. >> 1) Please see my original message explaining how this could be fixed >> without cache groups. I have questions about you initial statements. >> 1) "Merge" partition data from different caches Ptoposal is just to automate

Re: Remove cache groups in AI 3.0

Anton, Your example is extremely unlikely use case which we've never seen in a wild. But nevertheless: 1) Please see my original message explaining how this could be fixed without cache groups. 2) Logical cache creation also causes PME. 3) Yes, it is real. No fundamental limitations. In addition,

Re: Remove cache groups in AI 3.0

Dima, The question is: would we need cache groups if physical caches have the same performance as logical? On Wed, Apr 11, 2018 at 1:45 PM, Dmitry Pavlov wrote: > Hi Igniters, > > Actually I do not understand both points of view: we need to (keep/remove) > cache groups.

Re: Remove cache groups in AI 3.0

Vova, 1) Each real cache have some megabytes overhead of memory on affinity each node. Virtual cache inside cache group consumes much less memory (~ 0mb). 2) Real cache creation cause PME, Virtual cache creation just cause minor topology increment and do not stops tx. Not sure about this

Re: Remove cache groups in AI 3.0

Hi Igniters, Actually I do not understand both points of view: we need to (keep/remove) cache groups. Only one reason for refactoring I see : 'too much fsyncs', but it may be solved at level of FilePageStoreV2 with new virtual FS for partitions/index data, without any other changes. Sincerely,

Re: Remove cache groups in AI 3.0

Denis, Normally, every database object, whether it is a table or an index, is kept in it's own exclusive segment. Segment can span one or more real files. Segment always have a kind of allocation map allowing to quickly get number of allocated pages for specific object. On Wed, Apr 11, 2018 at

Re: Remove cache groups in AI 3.0

Anton, I do not see the point. What is the problem with creation or removal of real cache? On Wed, Apr 11, 2018 at 1:05 PM, Anton Vinogradov wrote: > Vova, > > Cache groups are very useful. > > For example, you can develop multi-tenant applications using cache groups > as a

Re: Remove cache groups in AI 3.0

Dmitry, If you do this, why would you need cache groups at all? On Tue, Apr 10, 2018 at 1:58 PM, Dmitry Pavlov wrote: > Hi Vladimir, > > We can solve "too many fsyncs" or 'too many small files' by placing several > partitions of cache group in one file. > > We don't need

Re: Remove cache groups in AI 3.0

Vova, Cache groups are very useful. For example, you can develop multi-tenant applications using cache groups as a templates. In case you have some cache groups, eg. Users, Loans, Deposits, you can keep records for Organisation_A, Organisation_B and Organisation_C at same data sctuctures, but

Re: Remove cache groups in AI 3.0

Vladimir, - Data size per-cache Could you elaborate how the data size per-cache/table task will be addressed with proposed architecture? Are you going to store data of a specific cache in dedicated pages/segments? What's about index size? -- Denis On Tue, Apr 10, 2018 at 2:31 AM, Vladimir

Re: Remove cache groups in AI 3.0

Hi Vladimir, We can solve "too many fsyncs" or 'too many small files' by placing several partitions of cache group in one file. We don't need to get rid from cache groups in this case. It is not trivial task, but it is doable. We need to create simplest FS for paritition chunks inside one file.

Re: Remove cache groups in AI 3.0

Dima, 1) Easy to understand for users AI 2.x: cluster -> cache group -> cache -> table AI 3.x: cluster -> cache(==table) 2) Fine grained cache management - MVCC on/off per-cache - WAL mode on/off per-cache - Data size per-cache 3) Performance: - Efficient scans are not possible with cache

Re: Remove cache groups in AI 3.0

Vladimir, sounds like a huge refactoring. Other than "cache groups are confusing", are we solving any other big issues with the new proposed approach? (every time we try to refactor rebalancing, I get goose bumps) D. On Tue, Apr 10, 2018 at 1:32 AM, Vladimir Ozerov wrote: