If grid hangs, graceful shutdown would most likely hang as well. Almost
never you can recover from a bad state using graceful procedures.

I agree that we should not create two defaults, especially in this case.
It's not even strictly defined what is embedded node in Ignite. For
example, if I start it using a custom main class and/or custom script
instead of ignite.sh, would it be embedded or standalone node?


On Tue, Mar 13, 2018 at 4:58 PM, Ivan Rakov <ivan.glu...@gmail.com> wrote:

> One more note: "kill if standalone, stop if embedded" differs from what
> you are suggesting "try graceful, then kill process regardless" only in
> case when graceful shutdown hangs.
> Do we have understanding, how often does graceful shutdown hang?
> Obviously, *grid hang* is often case, but it shouldn't be messed with
> *graceful shutdown hang*. From my experience, if something went wrong,
> users just prefer to do kill -9  because it's much more reliable and easy.
> Probably, in most of cases when kill -9 worked, graceful stop would have
> worked as well - we just don't have such statistics.
> It may be bad example, but: in our CI tests we intentionally break grid in
> many harsh ways and perform a graceful stop after the test execution, and
> it doesn't hang - otherwise we'd see many "Execution timeout" test suite
> hangs.
> Best Regards,
> Ivan Rakov
> On 14.03.2018 2:24, Dmitriy Setrakyan wrote:
>> On Tue, Mar 13, 2018 at 7:13 PM, Ivan Rakov <ivan.glu...@gmail.com>
>> wrote:
>> I just would like to add my +1 for "kill if standalone, stop if embedded"
>>> default option. My arguments:
>>> 1) Regarding "If Ignite hangs - it will likely be impossible to stop":
>>> Unfortunately, it's true that Ignite can hang during stop procedure.
>>> However, most of failures described under IEP-14 (storage IO exceptions,
>>> death of critical system worker thread, etc) normally shouldn't turn node
>>> into "impossible to stop" state. Turning into that state is a bug
>>> itself. I
>>> guess that we shouldn't choose system behavior on the basis of known
>>> bugs.
>> The whole discussion is about protecting against force-major issues,
>> including Ignite bugs. You are assuming that a user application will
>> somehow continue to function if an Ignite node is stopped. In most cases
>> it
>> will just freeze itself and cause the rest of the application to hang.
>> Again, "kill+stop" is the most deterministic and the safest default
>> behavior. Try a graceful shutdown (which will make restart easier), and
>> then kill the process regardless.
>> Note that we are arguing about the default behavior. If a user does not
>> like this default, then this user can change it to another behavior.
>> 2) User might want to handle Ignite node crash before shutting down the
>>> whole JVM - raise alert, close external resources, etc
>>> Very unlikely, but if a user is this advanced, then this user can change
>> the default behavior. Most users will not even know how to configure such
>> custom shutdown behavior and would prefer an automatic kill.
>> 3) IEP-14 document has important notes: "More than one Ignite node could
>> be
>>> started in one JVM process" and "Different nodes in one JVM process could
>>> belong to different clusters". This is possible only in embedded mode. I
>>> think, we shouldn't shock user by sudden JVM halt (possibly, along with
>>> another healthy nodes) if there's a chance of successful node stop.
>>> Has anyone actually seen a real example of that? I have not. This
>> scenario
>> is extremely unlikely and should not define the default behavior. Again,
>> if
>> a user is so advanced to come up with such a sophisticated deployment,
>> then
>> the same user should be able to set different default behaviors for
>> different clusters.

Reply via email to