Re: The future of Affinity / Topology concepts and possible PME optimizations.

2018-09-18 Thread Dmitriy Setrakyan
Ilya, I would suggest that you discuss everything here on the dev@ list, including suggestions how to split the work. D. On Tue, Sep 18, 2018 at 6:34 PM Ilya Lantukh wrote: > Thanks for the feedback! > > I agree that we should start with the simplest optimizations, but it seems > that

Re: The future of Affinity / Topology concepts and possible PME optimizations.

2018-09-18 Thread Ilya Lantukh
Thanks for the feedback! I agree that we should start with the simplest optimizations, but it seems that decoupling affinity/topology versions is necessary before we can make any progress here, and this is a rather complex change all over the code. If anyone wants to help, please contact me

Re: The future of Affinity / Topology concepts and possible PME optimizations.

2018-09-18 Thread Maxim Muzafarov
Ilya, > 3. Start node in baseline: both affinity and topology versions should be incremented, but it might be possible to optimize PME for such case and avoid cluster-wide freeze. Partition assignments for such node are already calculated, so we can simply put them all into MOVING state.

Re: The future of Affinity / Topology concepts and possible PME optimizations.

2018-09-18 Thread Alexey Goncharuk
Ilya, This is a great idea, but before we can ultimately decouple the affinity version from the topology version, we need to fix a few things with baseline topology first. Currently the in-memory caches are not using the baseline topology. We are going to fix this as a part of IEP-4 Phase II

Re: The future of Affinity / Topology concepts and possible PME optimizations.

2018-09-18 Thread Anton Vinogradov
Ilya, I like your idea. Let's create IEP and jira issues. I will be glad to take a part in this journey once we'll discuss every optimization in details. чт, 13 сент. 2018 г. в 22:51, Dmitriy Setrakyan : > Ilya, > > Thanks for the detailed explanation. Everything you suggested makes sense. >

Re: The future of Affinity / Topology concepts and possible PME optimizations.

2018-09-13 Thread Dmitriy Setrakyan
Ilya, Thanks for the detailed explanation. Everything you suggested makes sense. Needless to say, PME effect on the grid should be minimized. Let's start with the simpler and less risky changes first. D. On Thu, Sep 13, 2018 at 5:58 AM Ilya Lantukh wrote: > Igniters, > > As most of you know,

The future of Affinity / Topology concepts and possible PME optimizations.

2018-09-13 Thread Ilya Lantukh
Igniters, As most of you know, Ignite has a concept of AffinityTopologyVersion, which is associated with nodes that are currently present in topology and a global cluster state (active/inactive, baseline topology, started caches). Modification of either of them involves process called Partition