Re: Cluster auto activation design proposal

Sounds good to me. 2017-11-08 10:48 GMT+03:00 Dmitriy Setrakyan : > How about BaseNode? > > On Wed, Nov 8, 2017 at 2:40 PM, Alexey Goncharuk < > alexey.goncha...@gmail.com > > wrote: > > > Dmitriy, > > > > An offline node is a node that was present in a cluster, but either

Re: Cluster auto activation design proposal

How about BaseNode? On Wed, Nov 8, 2017 at 2:40 PM, Alexey Goncharuk wrote: > Dmitriy, > > An offline node is a node that was present in a cluster, but either > disconnected or failed and the corresponding event is fired by the > discovery SPI. In this case, the

Re: Cluster auto activation design proposal

Dmitriy, An offline node is a node that was present in a cluster, but either disconnected or failed and the corresponding event is fired by the discovery SPI. In this case, the cluster will not attempt to rebalance data and change affinity assignments until it is either confirmed manually via

Re: Cluster auto activation design proposal

Alexey, can you explain what an offline node means, if it is not an endpoint? Can it become an online node? If you could describe the transition steps of how a node goes offline and online, it would help. D. On Tue, Nov 7, 2017 at 4:01 PM, Alexey Goncharuk wrote: >

Re: Cluster auto activation design proposal

Why not? To me, the endpoint in the name suggests that this is something intended to connect to, however, the interface just denotes a node that can be offline, so I think it still has to be a 'Node'. 2017-11-06 22:52 GMT+03:00 Dmitriy Setrakyan : > Got it. In that case, I

Re: Cluster auto activation design proposal

Got it. In that case, I do not think that BaselineNode is a correct abstraction. How about ClusterEndpoint? D. On Sun, Nov 5, 2017 at 2:12 AM, Alexey Goncharuk wrote: > Dmitriy, > > The main point of having a BaselineNode interface is that baseline topology > will

Re: Cluster auto activation design proposal

Dmitriy, The main point of having a BaselineNode interface is that baseline topology will contain nodes even when they are offline. When a node is offline, most of the methods on ClusterNode are meaningless, thus the new interface (for example, node ID). I left on the interface only methods which

Re: Cluster auto activation design proposal

I assume that BaselineNode is intended to be used instead of ClusterNode during affinity assignment calculation. However, it will require that we change signature of AffinityFunction and AffinityFunctionContext methods, which are part of public API. Can we actually do it? On Sat, Nov 4, 2017 at

Re: Cluster auto activation design proposal

Alexey, what is the point of BaselineNode interface? Why not just have ClusterNode? On Fri, Nov 3, 2017 at 11:26 PM, Alexey Goncharuk < alexey.goncha...@gmail.com> wrote: > Guys, > > We are getting closer to the baseline topology finalization. As a first > step, I would like to request a review

Re: Cluster auto activation design proposal

Guys, We are getting closer to the baseline topology finalization. As a first step, I would like to request a review of the baseline topology management API. The changes are summarized in [1]. In my opinion, changes are quite simple and concise. Also, as a side note, I suggest moving cluster

Re: Cluster auto activation design proposal

On Mon, Sep 4, 2017 at 6:13 AM, Sergey Chugunov wrote: > Dmitriy, > > I like the idea of ClusterActivator interface. > > From user perspective it provides the same functionality as the setter but > in more clear and intuitive way. > BTW, I made a naming mistake in the

Re: Cluster auto activation design proposal

Dmitriy, I like the idea of ClusterActivator interface. >From user perspective it provides the same functionality as the setter but in more clear and intuitive way. Also it gives us a good place to put all the documentation about the feature. Any other opinions? On Fri, Sep 1, 2017 at 2:35

Re: Cluster auto activation design proposal

How about this: > *interface ClusterActivator {* > *boolean activate(Collection nodes);**}* Out of the box, we can provide this implementation of the activation filter: > > *ClusterInitialActiveSet implements ClusterActivator { * > *InigeInitialActiveSet(String... addresses);**}*

Re: Cluster auto activation design proposal

Dmitriy, The idea is interesting however I cannot come up with a clear use case which can be widely adopted. I would give users a simple API at first to cover 80% of their needs and then collect some feedback and start thinking about adding new functionality. Makes sense? Sergey. On Thu, Aug

Re: Cluster auto activation design proposal

Hm... Can we also ask user to optionally provide a predicate which will receive a collection of nodes started so far and return true if the activation should happen? Will it be useful? On Wed, Aug 30, 2017 at 6:28 PM, Sergey Chugunov wrote: > Nick, > > As I summed up

Re: Cluster auto activation design proposal

Nick, As I summed up in this thread above, calling setter for initial activation nodes is not the only option: 1. user starts up new cluster of desired number of nodes and activates it using existing API. BLT is created with all nodes presented in the cluster at the moment of

Re: Cluster auto activation design proposal

How is a user expected to produce a collection of ClusterNode prior to all of the expected nodes joining? Users don't create instances of this, so as far as I can tell it would have to be retrieved from IgniteCluster. However, would doing that and calling the proposed method be really any

Re: Cluster auto activation design proposal

Dmitriy, Now I see your point and I think you're right. We can give end-user a simple setter like Ignite::activation::setInitialActivationNodes(Collection nodes) to provide collection of nodes that grid must reach to activate automatically. And then using the collection we'll create

Re: Cluster auto activation design proposal

Sergey, the interface you are suggesting is internal, not external. Why should user ever see it or care about it? D. On Mon, Aug 28, 2017 at 3:18 PM, Sergey Chugunov wrote: > Dmitriy, > > It was my misunderstanding, I believe that setter is not enough and we need > a

Re: Cluster auto activation design proposal

Dmitriy, It was my misunderstanding, I believe that setter is not enough and we need a full-fledged entity. We should also be able to check if BLTs are compatible. Interface looks like this and use case for this functionality is described below. interface BaselineTopology { Collection

Re: Cluster auto activation design proposal

Sergey, I am still confused. What is the BaselineTopology interface in your example? I thought that you agreed with me that we simply need a setter for activation nodes, no? D. On Tue, Aug 22, 2017 at 4:47 AM, Sergey Chugunov wrote: > Dmitriy, > > As I understand you

Re: Cluster auto activation design proposal

Dmitriy, As I understand you use the term "minimalActivationNodes" as a synonym for BaselineTopology concept. In that case I agree with you that we can replace both "establish*" methods with a simple setter method (see below in summary). Summing up the whole discussion I see the functionality as

Re: Cluster auto activation design proposal

I still do not see why anyone would explicitly call these 2 methods: *Ignite::activation::establishBaselineTopology();* *Ignite::activation::establishBaselineTopology(BaselineTopology bltTop);* For example, if a web console, or some other admin process, want to automatically set currently

Re: Cluster auto activation design proposal

API is proposed in the head of the thread by Sergey, as I understood: __ API for BaselineTopology manipulation may look like this: *Ignite::activation::establishBaselineTopology();*

Re: Cluster auto activation design proposal

On Thu, Aug 17, 2017 at 7:34 AM, Sergey Chugunov wrote: > Dmitriy, > > As automatic creation of Baseline Topology is just another use case about > the whole concept, lets discuss it in this thread and forget about parallel > one. > > *initialActivationNodes* for

Re: Cluster auto activation design proposal

Dmitriy, As automatic creation of Baseline Topology is just another use case about the whole concept, lets discuss it in this thread and forget about parallel one. *initialActivationNodes* for desired configuration of Baseline Topology sounds good to me, I would like to hear from other Igniters.

Re: Cluster auto activation design proposal

Sergey, As it is becoming clear from another thread, the only configuration users will need to provide is the initial set of nodes. Everything else will be handled by Ignite automatically. In this case, the name Baseline Topology will appear only in documentation, in which case the name is OK

Re: Cluster auto activation design proposal

More options to name the concept (one may put "node set" or "topology" at the end - for me these are interchangeable): 1. essential 2. basic 3. completed 4. prepared 5. solid 6. enduring To me the name must stress the fact that "node set" or "topology" contains all nodes user

Re: Cluster auto activation design proposal

Going down "node set" road: -fixed node set -established node set -base node set On Thu, Aug 10, 2017 at 5:23 PM, Dmitriy Setrakyan wrote: > Can we brainstorm on the names again, I am not sure we have a consensus on > the name "baseline topology". This will be included

Re: Cluster auto activation design proposal

Can we brainstorm on the names again, I am not sure we have a consensus on the name "baseline topology". This will be included in Ignite configuration, so the name has to be clear. Some of the proposals were: - baseline topology - minimal node set - node restart set - minimal topology Any other

Re: Cluster auto activation design proposal

Denis, This should be handled by the BT triggers. If I have 3 backups configured, I actually won't care if my cluster will live 6 hours without an additional backup. If for a partition there is only one backup left - a new BT should be triggered automatically. 2017-08-10 0:33 GMT+03:00 Denis

Re: Cluster auto activation design proposal

Sergey, That’s the only concern I have: * 5. User takes out nodes from cluster (e.g. for maintenance purposes): no rebalance happens until user recreates BLT on new cluster topology.* What if a node is crashed (or some other kind of outage) in the middle of the night and the user has to be

Re: Cluster auto activation design proposal

Folks, I've summarized all results from our discussion so far on wiki page: https://cwiki.apache.org/confluence/display/IGNITE/Automatic+activation+design+-+draft I hope I reflected the most important details and going to add API suggestions for all use cases soon. Feel free to give feedback

Re: Cluster auto activation design proposal

I also would like to provide more use cases of how BLT is supposed to work (let me call it this way until we come up with a better one): 1. User creates new BLT using WebConsole or other tool and "applies" it to brand-new cluster. 2. User starts up brand-new cluster with desired amount

Re: Cluster auto activation design proposal

My understanding of Baseline Topology is the set of nodes which are *expected* to be in the cluster. Let me go a little bit further because BT (or whatever name we choose) may and will solve more issues than just auto-activation: 1) More graceful control over rebalancing than just rebalance

Re: Cluster auto activation design proposal

>Obvious connotation of "minimal set" is a set that cannot be decreased. >But lets consider the following case: user has a cluster of 50 nodes and >decides to switch off 3 nodes for maintenance for a while. Ok, user just >does it and then recreates this "minimal node set" to only 47 nodes. >So

Re: Cluster auto activation design proposal

>I think it is not just restarts, this set of nodes is minimally required for the cluster to function, no? I don't think so. Cluster can function if there is no data loss. --Yakov

Re: Cluster auto activation design proposal

Igniters, what about Target Node Set? Complete Node Set? As when we reach this topology, we can activate cluster. чт, 3 авг. 2017 г. в 12:58, Sergey Chugunov : > Dmitriy, > > Obvious connotation of "minimal set" is a set that cannot be decreased. > > But lets consider

Re: Cluster auto activation design proposal

Dmitriy, Obvious connotation of "minimal set" is a set that cannot be decreased. But lets consider the following case: user has a cluster of 50 nodes and decides to switch off 3 nodes for maintenance for a while. Ok, user just does it and then recreates this "minimal node set" to only 47 nodes.

Re: Cluster auto activation design proposal

Yakov, I think it is not just restarts, this set of nodes is minimally required for the cluster to function, no? ⁣D.​ On Aug 3, 2017, 11:23 AM, at 11:23 AM, Yakov Zhdanov wrote: >Ю> How about naming it "minimal node set" or "required node set"? > >Required for what? I

Re: Cluster auto activation design proposal

>From my standpoint name for the concept should emphasize that nodes from the set constitute a target topology - the place where user wants to be. If we go in a "node set" way, what about FixedNodeSet or BaseNodeSet? "restart node set" also is a bit confusing because this concept works not only

Re: Cluster auto activation design proposal

Ю> How about naming it "minimal node set" or "required node set"? Required for what? I would add restart if there are no confusion. --Yakov

Re: Cluster auto activation design proposal

How about naming it "minimal node set" or "required node set"? ⁣D.​ On Aug 3, 2017, 11:15 AM, at 11:15 AM, Yakov Zhdanov wrote: >> * Based on some sort of policies when the actual cluster topology >differs >too much from the baseline or when some critical condition happens

Re: Cluster auto activation design proposal

> * Based on some sort of policies when the actual cluster topology differs too much from the baseline or when some critical condition happens (e.g., when there are no more backups for a partition) Good point, Alex! I would even go further. If cluster is active and under load and nodes continue

Re: Cluster auto activation design proposal

First of all, this sounds a bit too scientific and will likely be misunderstood and misused. Is there a chance to give it a better name and a simpler use case and description? Secondly, I do agree that it should be mandatory to update the BT. For example, what if I want to add several nodes to

Re: Cluster auto activation design proposal

I think we should be able to change the BT in the runtime, and a user should have several ways to do this: * programmatically via the API suggested by Sergey * Using management tools (console visor on Web Console) * Based on some sort of policies when the actual cluster topology differs too

Re: Cluster auto activation design proposal

Sergey, Is it assumed that the baseline topology can be updated in runtime? Like, initially I had a cluster of 10 nodes but in a couple of weeks it was expanded to 15 nodes. How the baseline topology should be updated in this way? Will it happen automatically? — Denis > On Aug 1, 2017, at