Denis,

In general, service initialization shouldn't prevent a node from joining
the cluster or slowing down that process. Thus, I would start the
initialization routines only after the node is accepted by the cluster. If
the initialization fails then we need to report a respective message to the
logs and, probably, generate a system event the user can be subscribed to.

Regardless, of the service initialization time, I think we still need to
utilize discovery SPI to avoid problems discussed later.

Val, others, what do you think about that?


--
Denis

On Mon, Apr 16, 2018 at 10:29 AM, Denis Mekhanikov <dmekhani...@gmail.com>
wrote:

> Basically, my question is: at which moment services should be deployed on
> connecting nodes?
> Should we reject a node from being included into a topology, if services,
> that are assigned to it, fail to deploy?
>
> It would be good to be sure, that all assigned services are initialised and
> working, when node start is finished.
> Otherwise it's unclear, how to notify a user about failures in service
> initialisation on new nodes.
>
> If we decide to provide such guarantee, then how are we going to do that?
> Is procedure, that I described, viable?
> It requires hacking through the discovery protocol, which is a thing, that
> should be avoided.
> So, maybe there is another way to achieve the same thing?
>
> Denis
>
> сб, 14 апр. 2018 г. в 1:48, Denis Magda <dma...@apache.org>:
>
> > It sounds like it's not a trivial thing to support the automatic services
> > redeployment after a restart. Let's postpone it for now, guys
> concentrating
> > on more urgent things related to the services.
> >
> > Alex, Vladimir,
> >
> > Could you have a look at Denis question about the discovery-based
> > deployment? Guess it's the only one thing that prevents us from the IEP
> > finalization.
> >
> > --
> > Denis
> >
> > On Fri, Apr 13, 2018 at 5:30 AM, Denis Mekhanikov <dmekhani...@gmail.com
> >
> > wrote:
> >
> > > Vladimir,
> > >
> > > Currently we don't save binary metadata to disk, when persistence is
> > > disabled.
> > > But we still persist marshaller mappings for some reason, and I
> > personally
> > > believe, that we shouldn't.
> > >
> > > But I agree, that we should separate data and service persistence
> > > configuration.
> > > Right now persistence of services is configured in a pretty non-obvious
> > > manner.
> > > It should be a clear way to tell Ignite, whether you want services to
> be
> > > persisted or not.
> > >
> > > I'm not sure, that we should make "statefullness" in general
> > configurable.
> > > Users don't care much, whether metadata is preserved on restarts, or
> not.
> > >
> > > Denis
> > >
> > > пт, 13 апр. 2018 г. в 14:29, Vladimir Ozerov <voze...@gridgain.com>:
> > >
> > > > Alex,
> > > >
> > > > I would say that we've already had this behavior for years -
> marshaller
> > > > cache. I think it is time to agree that "in-memory" != stateless.
> > Instead
> > > > "in-memory" means "data is not stored on disk".
> > > > May be we can have a flag which will wipe out all metadata on node
> > > restart
> > > > (e.g. it could make sense for embedded clients)?
> > > >
> > > > On Fri, Apr 13, 2018 at 12:48 PM, Alexey Goncharuk <
> > > > alexey.goncha...@gmail.com> wrote:
> > > >
> > > > > Denis,
> > > > >
> > > > > This is a subtle question. It looks like we have now a number of
> > > > use-cases
> > > > > when persistent storage is required even for a pure in-memory mode.
> > One
> > > > of
> > > > > the use-cases is thin client authentication, the other is service
> > grid
> > > > > configuration persistence.
> > > > >
> > > > > Generally, I would agree that this is an expected behavior.
> However,
> > > this
> > > > > means that a user cannot simply start and stop nodes randomly
> > anymore.
> > > > > Ignite start will require some sort of installation or work folder
> > > > > initialization (sort of initdb in postgres) which is ok for
> > > > > persistence-enabled modes, but I am not sure if this is expected
> for
> > > > > in-memory. Of course, we can run this initialization automatically,
> > but
> > > > it
> > > > > is not always a good idea.
> > > > >
> > > > > If we are ok to have this restrictions for in-memory mode, then
> > service
> > > > > persistence makes sense.
> > > > >
> > > > > --AG
> > > > >
> > > > > 2018-04-11 22:36 GMT+03:00 Denis Magda <dma...@apache.org>:
> > > > >
> > > > >> Denis,
> > > > >>
> > > > >> I think that the service deployment state needs be persisted
> > > > cluster-wide.
> > > > >> I guess that our meta-store is capable of doing so. Alex G.,
> > Vladimir,
> > > > >> could you confirm?
> > > > >>
> > > > >> As for the split-brain scenarios, I would put them aside for now
> > > > because,
> > > > >> anyway, they have to be solved at lower levels (meta store,
> > discovery,
> > > > >> etc.).
> > > > >>
> > > > >> Also, I heard that presently we store a service configuration in
> the
> > > > >> system
> > > > >> cache that doesn't give us a way to deploy a new version of a
> > service
> > > > >> without undeployment of the previous one. Will this issue be
> > addressed
> > > > by
> > > > >> the new deployment approach?
> > > > >>
> > > > >> --
> > > > >> Denis
> > > > >>
> > > > >> On Wed, Apr 11, 2018 at 1:28 AM, Denis Mekhanikov <
> > > > dmekhani...@gmail.com>
> > > > >> wrote:
> > > > >>
> > > > >> > Denis,
> > > > >> >
> > > > >> > Sounds reasonable. It's not clear, though, what should happen,
> if
> > a
> > > > >> joining
> > > > >> > node has some services persisted, that are missing on other
> nodes.
> > > > >> > Should we deploy them?
> > > > >> > If we do so, it could lead to surprising behaviour. For example
> > you
> > > > >> could
> > > > >> > kill a node, undeploy a service, then bring back an old node,
> and
> > it
> > > > >> would
> > > > >> > make the service resurrect.
> > > > >> > We could store some deployment counter along with the service
> > > > >> > configurations on all nodes, that would show how many times the
> > > > service
> > > > >> > state has changed, i.e. it has been undeployed/redeployed. It
> > should
> > > > be
> > > > >> > kept for undeployed services as well to avoid situations like I
> > > > >> described.
> > > > >> >
> > > > >> > But it still leaves a possibility of incorrect behaviour, if
> there
> > > > was a
> > > > >> > split-brain situation at some point. I don't think we should
> > precess
> > > > it
> > > > >> > somehow, though. If we choose to tackle it, it will
> overcomplicate
> > > > >> things
> > > > >> > for a sake of a minor improvement.
> > > > >> >
> > > > >> > Denis
> > > > >> >
> > > > >> > вт, 10 апр. 2018 г. в 0:55, Valentin Kulichenko <
> > > > >> > valentin.kuliche...@gmail.com>:
> > > > >> >
> > > > >> > > I was responding to another Denis :) Agree with you on your
> > point
> > > > >> though.
> > > > >> > >
> > > > >> > > -Val
> > > > >> > >
> > > > >> > > On Mon, Apr 9, 2018 at 2:48 PM, Denis Magda <
> dma...@apache.org>
> > > > >> wrote:
> > > > >> > >
> > > > >> > > > Val,
> > > > >> > > >
> > > > >> > > > Guess we're talking about other situations. I'm bringing up
> > the
> > > > case
> > > > >> > > when a
> > > > >> > > > service was deployed dynamically and has to be brought up
> > after
> > > a
> > > > >> full
> > > > >> > > > cluster restart w/o user intervention. To achieve this we
> need
> > > to
> > > > >> > persist
> > > > >> > > > the service's configuration somewhere.
> > > > >> > > >
> > > > >> > > > --
> > > > >> > > > Denis
> > > > >> > > >
> > > > >> > > > On Mon, Apr 9, 2018 at 1:42 PM, Valentin Kulichenko <
> > > > >> > > > valentin.kuliche...@gmail.com> wrote:
> > > > >> > > >
> > > > >> > > > > Denis,
> > > > >> > > > >
> > > > >> > > > > EVT_CLASS_DEPLOYED should be fired every time a class is
> > > > deployed
> > > > >> or
> > > > >> > > > > redeployed. If this doesn't happen in some cases, I
> believe
> > > this
> > > > >> > would
> > > > >> > > > be a
> > > > >> > > > > bug. I don't think we need to add any new events.
> > > > >> > > > >
> > > > >> > > > > -Val
> > > > >> > > > >
> > > > >> > > > > On Mon, Apr 9, 2018 at 10:50 AM, Denis Magda <
> > > dma...@apache.org
> > > > >
> > > > >> > > wrote:
> > > > >> > > > >
> > > > >> > > > > > Denis,
> > > > >> > > > > >
> > > > >> > > > > > I would encourage us to persist a service configuration
> in
> > > the
> > > > >> meta
> > > > >> > > > store
> > > > >> > > > > > and have this capability enabled by default. That's
> > > essential
> > > > >> for
> > > > >> > > > > services
> > > > >> > > > > > started dynamically. Moreover, we support similar
> behavior
> > > for
> > > > >> > > caches,
> > > > >> > > > > > indexes, and other DDL changes happened at runtime.
> > > > >> > > > > >
> > > > >> > > > > > --
> > > > >> > > > > > Denis
> > > > >> > > > > >
> > > > >> > > > > > On Mon, Apr 9, 2018 at 9:34 AM, Denis Mekhanikov <
> > > > >> > > > dmekhani...@gmail.com>
> > > > >> > > > > > wrote:
> > > > >> > > > > >
> > > > >> > > > > > > Another question, that I would like to discuss is
> > whether
> > > > >> > services
> > > > >> > > > > should
> > > > >> > > > > > > be preserved on cluster restarts.
> > > > >> > > > > > >
> > > > >> > > > > > > Currently it depends on persistence configuration. If
> > > > >> persistence
> > > > >> > > for
> > > > >> > > > > any
> > > > >> > > > > > > data region is enabled, then services will be
> persisted
> > as
> > > > >> well.
> > > > >> > > This
> > > > >> > > > > is
> > > > >> > > > > > a
> > > > >> > > > > > > pretty strange way of configuring this behaviour.
> > > > >> > > > > > > I'm not sure, if anybody relies on this functionality
> > > right
> > > > >> now.
> > > > >> > > > Should
> > > > >> > > > > > we
> > > > >> > > > > > > support it at all? If yes, should we make it
> > configurable?
> > > > >> > > > > > >
> > > > >> > > > > > > Denis
> > > > >> > > > > > >
> > > > >> > > > > > > пн, 9 апр. 2018 г. в 19:27, Denis Mekhanikov <
> > > > >> > > dmekhani...@gmail.com
> > > > >> > > > >:
> > > > >> > > > > > >
> > > > >> > > > > > > > Val,
> > > > >> > > > > > > >
> > > > >> > > > > > > > Sounds reasonable. I just think, that user should
> have
> > > > some
> > > > >> way
> > > > >> > > to
> > > > >> > > > > > know,
> > > > >> > > > > > > > that new version of a service class was deployed.
> > > > >> > > > > > > > One way to do it is to listen to
> *EVT_CLASS_DEPLOYED.
> > > *I'm
> > > > >> not
> > > > >> > > > sure,
> > > > >> > > > > > > > whether it is triggered on class redeployment,
> though.
> > > If
> > > > >> not,
> > > > >> > > then
> > > > >> > > > > > > another
> > > > >> > > > > > > > event type should be added.
> > > > >> > > > > > > >
> > > > >> > > > > > > > I don't think, that a lot of people will implement
> > their
> > > > own
> > > > >> > > > > > > > *DeploymentSpi*-s, so we should make work with
> > > > >> > *UriDeploymentSpi*
> > > > >> > > > as
> > > > >> > > > > > > > comfortable as possible.
> > > > >> > > > > > > >
> > > > >> > > > > > > > Denis
> > > > >> > > > > > > >
> > > > >> > > > > > > > пт, 6 апр. 2018 г. в 23:40, Valentin Kulichenko <
> > > > >> > > > > > > > valentin.kuliche...@gmail.com>:
> > > > >> > > > > > > >
> > > > >> > > > > > > >> Yes, the class deployment itself has to be
> explicit.
> > > > I.e.,
> > > > >> > there
> > > > >> > > > has
> > > > >> > > > > > to
> > > > >> > > > > > > be
> > > > >> > > > > > > >> a manual step where user updates the class, and the
> > > exact
> > > > >> step
> > > > >> > > > > > required
> > > > >> > > > > > > >> would depend on DeploymentSpi implementation. But
> > then
> > > > >> Ignite
> > > > >> > > > takes
> > > > >> > > > > > care
> > > > >> > > > > > > >> of
> > > > >> > > > > > > >> everything else - service redeployment and restart
> is
> > > > >> > automatic.
> > > > >> > > > > > > >>
> > > > >> > > > > > > >> Dmitriy Pavlov, all this is going to be disabled if
> > > > >> > > DeploymentSpi
> > > > >> > > > is
> > > > >> > > > > > not
> > > > >> > > > > > > >> configured. In this case service class definitions
> > have
> > > > to
> > > > >> be
> > > > >> > > > > deployed
> > > > >> > > > > > > on
> > > > >> > > > > > > >> local classpath and can't be updated in runtime.
> Just
> > > > like
> > > > >> it
> > > > >> > > > works
> > > > >> > > > > > > right
> > > > >> > > > > > > >> now.
> > > > >> > > > > > > >>
> > > > >> > > > > > > >> -Val
> > > > >> > > > > > > >>
> > > > >> > > > > > > >> On Fri, Apr 6, 2018 at 10:20 AM, Dmitriy Setrakyan
> <
> > > > >> > > > > > > dsetrak...@apache.org
> > > > >> > > > > > > >> >
> > > > >> > > > > > > >> wrote:
> > > > >> > > > > > > >>
> > > > >> > > > > > > >> > On Fri, Apr 6, 2018 at 9:13 AM, Dmitry Pavlov <
> > > > >> > > > > > dpavlov....@gmail.com>
> > > > >> > > > > > > >> > wrote:
> > > > >> > > > > > > >> >
> > > > >> > > > > > > >> > > Hi Igniters,
> > > > >> > > > > > > >> > >
> > > > >> > > > > > > >> > > I like automatic redeploy which can be disabled
> > by
> > > > >> config
> > > > >> > if
> > > > >> > > > > user
> > > > >> > > > > > > >> wants
> > > > >> > > > > > > >> > to
> > > > >> > > > > > > >> > > control this process. What do you think?
> > > > >> > > > > > > >> > >
> > > > >> > > > > > > >> >
> > > > >> > > > > > > >> > I do not think we should have anything automatic
> > when
> > > > it
> > > > >> > comes
> > > > >> > > > to
> > > > >> > > > > > > >> > deployment, everything should be explicit.
> However,
> > > if
> > > > we
> > > > >> > use
> > > > >> > > > the
> > > > >> > > > > > > >> > deployment SPI, then a user should be able to do
> > > "hot"
> > > > >> > > redeploy,
> > > > >> > > > > > > where a
> > > > >> > > > > > > >> > new service will be deployed if the user drops an
> > > > updated
> > > > >> > jar.
> > > > >> > > > > > > >> >
> > > > >> > > > > > > >> > We should not create anything new here. Ignite
> > > already
> > > > >> has a
> > > > >> > > > > > > deployment
> > > > >> > > > > > > >> SPI
> > > > >> > > > > > > >> > and it already works in a certain way. Let's not
> > > change
> > > > >> it.
> > > > >> > > > > > > >> >
> > > > >> > > > > > > >> > D.
> > > > >> > > > > > > >> >
> > > > >> > > > > > > >>
> > > > >> > > > > > > >
> > > > >> > > > > > >
> > > > >> > > > > >
> > > > >> > > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > > >
> > > > >
> > > >
> > >
> >
>

Reply via email to