Hello, Ivan. > I believe we should fix the issue instead of adapting API to temporary flaws.
Agree. Let’s fix it. > I think that clear description of active(false) impact in the documentation > is more than enough I can’t agree with this point. We shouldn’t imply the assumption that every user reads the whole documentation and completely understand the consequences of the deactivation command. This whole thread shows that even active core developers don't understand it. So my proposal is to remove --force flag only after we fix deactivation. > To sum it up, the question is whether we should reflect temporary system > design flaws in the API I can’t agree with the «temporary» design. We have neither design nor IEP or contributor who can fix current behavior. And, if I understand Alexey Goncharyuk correctly, current behavior was implemented intentionally. So, my understanding that current implementation would be here for a while. And after we fix it I totally support removing —force flag. > 24 марта 2020 г., в 12:06, Ivan Rakov <ivan.glu...@gmail.com> написал(а): > >> >> I think the only question is - Do we need —force flag in Java API or not. > > From my perspective, there's also no agreement that it should be present > in the thin clients' API. For instance, I think it shouldn't. > > As far as I know, IGNITE_REUSE_MEMORY_ON_DEACTIVATE is for *other* purpose. >> Can you provide a simple reproducer when in-memory data not cleared on >> deactivation? > > Preserving in-memory data isn't implemented so far, so I can't provide a > reproducer. My point is that we are halfway through it: we can build a > solution based on IGNITE_REUSE_MEMORY_ON_DEACTIVATE and additional logic > with reusing memory pages. > > For me, the ultimate value of Ignite into real production environment is >> user data. >> If we have some cases when data is lost - we should avoid it as hard as we >> can. >> >> So, for now, this flag required. > > Totally agree that sudden vanishing of user data is unacceptable. But I > don't see how it implies that we have to solve this issue by tangling > public API. If we see that system behaves incorrectly, I believe we should > fix the issue instead of adapting API to temporary flaws. I think that > clear description of active(false) impact in the documentation is more than > enough: on the one hand, if user didn't read documentation for the method > he calls, he can't complain about the consequences; on the other hand, if > user decided to deactivate the cluster for no matter what, -force flag will > barely stop him. > We anyway have enough time before 2.9 to implement a proper solution. > > To sum it up, the question is whether we should reflect temporary system > design flaws in the API. I think, we surely shouldn't: API certainly lives > longer and is not intended to collect workarounds for all bugs that are > already fixed or planned to be fixed. > We can collect more opinions on this. > > On Tue, Mar 24, 2020 at 10:22 AM Nikolay Izhikov <nizhi...@apache.org> > wrote: > >> Alexey. >> >> Having the way to silently vanish user data is even worse. >> So I’m strictly against removing —force flag. >> >>> 24 марта 2020 г., в 10:16, Alexei Scherbakov < >> alexey.scherbak...@gmail.com> написал(а): >>> >>> Nikolay, >>> >>> I'm on the same page with Ivan. >>> >>> Having "force" flag in public API as preposterous as having it in >>> System.exit. >>> For me it looks like badly designed API. >>> If a call to some method is dangerous it should be clearly specified in >> the >>> javadoc. >>> I'm also against some "temporary" API. >>> >>> We should: >>> >>> 1. Partially remove IGNITE-12701 except javadoc part. Note control.sh >> for a >>> long time has support for a confirmation on deactivation (interactive >> mode). >>> 2. IGNITE_REUSE_MEMORY_ON_DEACTIVATE=true already preserves memory >> content >>> after deactivation. We should start working on restoring page memory >> state >>> after subsequent reactivation. >>> 3. Describe current behavior for in-memory cache on deactivation in >> Ignite >>> documentation. >>> >>> >>> пн, 23 мар. 2020 г. в 21:22, Nikolay Izhikov <nizhi...@apache.org>: >>> >>>> Hello, Ivan. >>>> >>>>> Seems like we don't have a final agreement on whether we should add >> force >>>> flag to deactivation API. >>>> >>>> I think the only question is - Do we need —force flag in Java API or >> not. >>>> >>>> >>>>> As a final solution, I'd want to see behavior when all in-memory data >> is >>>> available after deactivation and further activation. >>>> >>>> Agree. >>>> >>>>> I believe it’s possible to don't deallocate memory >>>>> (like mentioned before, we already can achieve that with >>>> IGNITE_REUSE_MEMORY_ON_DEACTIVATE=true) and carefully reuse all loaded >> data >>>> pages on next activation and caches start. >>>> >>>> As far as I know, IGNITE_REUSE_MEMORY_ON_DEACTIVATE is for *other* >> purpose. >>>> Can you provide a simple reproducer when in-memory data not cleared on >>>> deactivation? >>>> >>>>> Considering this, do we really need to introduce force flag as a >>>> temporary precaution? >>>> >>>> My answer is yes we need it. >>>> Right now, we can’t prevent data loss on deactivation for in-memory >> caches. >>>> >>>> For me, the ultimate value of Ignite into real production environment is >>>> user data. >>>> If we have some cases when data is lost - we should avoid it as hard as >> we >>>> can. >>>> >>>> So, for now, this flag required. >>>> >>>>> I suggest to rollback [2] from AI master, stop working on [1] and focus >>>> on how to implement keeping in-memory data after the deactivation. >>>> >>>> I think we can remove —force flag only after implementation of keeping >>>> in-memory data on deactivation would be finished. >>>> If that happens before 2.9 then we can have clearer API. >>>> >>>>> 23 марта 2020 г., в 18:47, Ivan Rakov <ivan.glu...@gmail.com> >>>> написал(а): >>>>> >>>>> Folks, >>>>> >>>>> Let's revive this discussion until it's too late and all API changes >> are >>>>> merged to master [1]. >>>>> Seems like we don't have a final agreement on whether we should add >> force >>>>> flag to deactivation API. >>>>> >>>>> First of all, I think we are all on the same page that in-memory caches >>>>> data vanishing on deactivation is counter-intuitive and dangerous. As a >>>>> final solution, I'd want to see behavior when all in-memory data is >>>>> available after deactivation and further activation. I believe it's >>>>> possible to don't deallocate memory (like mentioned before, we already >>>> can >>>>> achieve that with IGNITE_REUSE_MEMORY_ON_DEACTIVATE=true) and carefully >>>>> reuse all loaded data pages on next activation and caches start. >>>>> >>>>> Also, this is a wider question, but: do we understand what cluster >>>>> deactivation is actually intended for? I can only think of two cases: >>>>> - graceful cluster shutdown: an ability to cut checkpoints and to end >>>>> transactional load consistently prior to further stop of all nodes >>>>> - blocking all API (both reads and writes) due to some maintenance >>>>> Neither of them requires forcefully clearing all in-memory data on >>>>> deactivation. If everyone agrees, from now on we should assume data >>>>> clearing as system design flaw that should be fixed, not as possible >>>>> scenario which we should support on the API level. >>>>> >>>>> Considering this, do we really need to introduce force flag as a >>>> temporary >>>>> precaution? I have at least two reasons against it: >>>>> 1) Once API was changed and released, we have to support it until the >>>> next >>>>> major release. If we all understand that data vanishing issue is >>>> fixable, I >>>>> believe we shouldn't engrave in the API flags that will become >> pointless. >>>>> 2) More personal one, but I'm against any force flags in the API. This >>>>> makes API harder to understand; more than that, the presence of such >>>> flags >>>>> just highlights that API is poorly designed. >>>>> >>>>> I suggest to rollback [2] from AI master, stop working on [1] and focus >>>> on >>>>> how to implement keeping in-memory data after the deactivation. >>>>> I think we can still require user consent for deactivation via >> control.sh >>>>> (it already requires --yes) and JMX. >>>>> >>>>> Thoughts? >>>>> >>>>> [1]: https://issues.apache.org/jira/browse/IGNITE-12614 >>>>> [2]: https://issues.apache.org/jira/browse/IGNITE-12701 >>>>> >>>>> -- >>>>> Ivan >>>>> >>>>> >>>>> On Tue, Mar 17, 2020 at 2:26 PM Vladimir Steshin <vlads...@gmail.com> >>>> wrote: >>>>> >>>>>> Nikolay, I think we should reconsider clearing at least system caches >>>>>> when deactivating. >>>>>> >>>>>> 17.03.2020 14:18, Nikolay Izhikov пишет: >>>>>>> Hello, Vladimir. >>>>>>> >>>>>>> I don’t get it. >>>>>>> >>>>>>> What is your proposal? >>>>>>> What we should do? >>>>>>> >>>>>>>> 17 марта 2020 г., в 14:11, Vladimir Steshin <vlads...@gmail.com> >>>>>> написал(а): >>>>>>>> >>>>>>>> Nikolay, hi. >>>>>>>> >>>>>>>>>>> And should be covered with the —force parameter we added. >>>>>>>> As fix for user cases - yes. My idea is to emphasize overall ability >>>> to >>>>>> lose various objects, not only data. Probably might be reconsidered in >>>>>> future. >>>>>>>> >>>>>>>> >>>>>>>> 17.03.2020 13:49, Nikolay Izhikov пишет: >>>>>>>>> Hello, Vladimir. >>>>>>>>> >>>>>>>>> If there is at lease one persistent data region then system data >>>>>> region also becomes persistent. >>>>>>>>> Your example applies only to pure in-memory clusters. >>>>>>>>> >>>>>>>>> And should be covered with the —force parameter we added. >>>>>>>>> >>>>>>>>> What do you think? >>>>>>>>> >>>>>>>>>> 17 марта 2020 г., в 13:45, Vladimir Steshin <vlads...@gmail.com> >>>>>> написал(а): >>>>>>>>>> >>>>>>>>>> Hi, all. >>>>>>>>>> >>>>>>>>>> Fixes for control.sh and the REST have been merged. Could anyone >>>> take >>>>>> a look to the previous email with an issue? Isn't this conductvery >>>> wierd? >>>>>>>>>> >>>>>> >>>> >>>> >>> >>> -- >>> >>> Best regards, >>> Alexei Scherbakov >> >>
