Pavel,
I don't understand why you are using *backwards compatibility* for completely internal things. Why you are thinking in terms of users usage when are talking about ignite-sys-cache but not thinking when refactoring, for instance, all the checkpoint classes? Take a look at the [1] issue. All the internals have been reworked and nobody even mentioned that it may affect some of the plugin developers. This is really strange, so I can't agree with you. I do not propose to rush with the ignite-sys-cache removal. I'd like to collect all the technical stuff that we depend on, fix all of them and after everything will be ready do a final removal. Slava already mentioned some crucial cases, so I hope you and Val also help with the rest of them. Let's be more precise in terms *what we need to fix*. I've made some investigations related to the ignite-sys-cache and here is my proposal: 1. Add deprecation for the 2.12 release. I hope it is still possible. 2. Apache Ignite core still have some dependencies on ignite-sys-cache that should fix for 2.13: 2.1. CLUSTER_NAME property: when it's not set the `deploymentId` of system cache is used. Let's move it to metastorage. 2.2. When the Security is enabled each compute task name (hash to be more precise) is written to the ignite-sys-cache [2]. From my point of view, we can remove it in 2.13. Can anyone check? 2.3. Slava mentioned some issues with the distributed metastorage that might be fixed. I think this is doable. 3. Remove the system cache in 2.14. [1] https://issues.apache.org/jira/browse/IGNITE-13143 [2] https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/task/GridTaskProcessor.java#L201 On Thu, 2 Dec 2021 at 12:56, Pavel Tupitsyn <ptupit...@apache.org> wrote: > > Maxim, > > > I don't think that we should wait for 3-rd party plugins to be updated > > this is an awful practice when the open-source product releases depend > > on some of the proprietary plugins > > This makes no sense, sorry. > > It is not that we depend on 3rd party plugins. > It is that *our users depend on us to preserve backwards compatibility*. > No one likes when their app breaks suddenly because of a library update. > > We know about one use case for sys-cache in GridGain, but there may be > more, no one knows. > Every breaking change should be carried out carefully and gradually. > > > On Thu, Dec 2, 2021 at 12:34 PM Maxim Muzafarov <mmu...@apache.org> wrote: > > > Slava, > > > > Thank you for the comments. > > > > > Maxim, the community must provide a reasonable time interval in order to > > notify all contributors and wait for updating all 3-rd party plugins. > > > > This is not actually true. We must notify about changes as earlier as > > possible and not only users but all the developers also. However, I > > don't think that we should wait for 3-rd party plugins to be updated > > this is an awful practice when the open-source product releases depend > > on some of the proprietary plugins. There is no dependency between > > Apache Ignite releases and proprietary plugin releases. If someone > > desires to upgrade to a new version of Apache Ignite it can update its > > plugins any time he likes. > > > > > We just need to provide a window that is enough to cover all related > > issues. > > > > Can you share all the issues you know, please? > > > > > distributed meta storage does not provide the ability to atomically > > change several properties at the same time (there are no transactions on > > this API). > > > > Can you share an example of what kind of properties we intend to > > change at the same? Will it be enough to change them through a single > > thread (e.g. the discovery thread or the exchange thread)? I agree > > here that distributed metastorage should provide such an ability, > > however, I can't find any real examples for Apache Ignite internal > > needs. Please, share the details and let's fix it. > > > > On Wed, 1 Dec 2021 at 23:19, Valentin Kulichenko > > <valentin.kuliche...@gmail.com> wrote: > > > > > > Agree with Slava. Two months is not enough time, especially considering > > > that the system cache is not functionally equivalent to the metastorage. > > I > > > suggest we do the following: > > > > > > 1. Write down the differences between the system cache and the > > metastorage, > > > and provide a transition guide for plugin developers. > > > 2. Deprecate the system cache in 2.13. > > > 3. Remove the system cache in one of the further releases. I don't think > > > it's reasonable to do this earlier than mid next year (even that is > > > potentially too early). > > > > > > Removing the system cache is the right move, but let's be more > > considerate > > > to the users. There is absolutely no need to rush. > > > > > > -Val > > > > > > On Wed, Dec 1, 2021 at 7:28 AM Вячеслав Коптилин < > > slava.kopti...@gmail.com> > > > wrote: > > > > > > > Hi Maxim, > > > > > > > > > On the other hand, I don't see any valuable reason why the change > > can't > > > > be done and we should wait for the future that never comes. > > > > Maxim, the community must provide a reasonable time interval in order > > to > > > > notify all contributors and wait for updating all 3-rd party plugins. > > > > Honestly, it does not seem to me that two, three months (the current > > > > timeline - end of March is the release date, so the end of Feb is code > > > > freeze) are quite enough. > > > > > > > > > I don't see any reasons why we should keep something in the CORE > > module > > > > that is being used at PLUGINS only. This is a lack of architecture > > design > > > > that must be fixed for sure; > > > > I agree with you. We just need to provide a window that is enough to > > cover > > > > all related issues. > > > > For example, distributed meta storage does not provide the ability to > > > > atomically change several properties at the same time (there are no > > > > transactions on this API). > > > > > > > > Perhaps, it would be nice to plan to remove the system cache on 2.14 or > > > > even 2.15. What do you think? > > > > > > > > Thanks, > > > > S. > > > > > > > > > > > > вт, 30 нояб. 2021 г. в 13:58, Maxim Muzafarov <mmu...@apache.org>: > > > > > > > > > Hello Val, > > > > > > > > > > Thank you for sharing the details. On the one hand, I agree with you > > > > > that there is no need to haste with this change and it must be > > > > > prepared carefully. On the other hand, I don't see any valuable > > reason > > > > > why the change can't be done and we should wait for the future that > > > > > never comes. > > > > > > > > > > Please, consider the following my considerations: > > > > > > > > > > - The release 2.13 is not started yet. It will take months to be > > > > > released (e.g. the end of March as a possible release date). The time > > > > > is more than enough; > > > > > - The 2.13 release has breaking changes, so it is a good opportunity > > > > > to fix some gaps; > > > > > - I don't see any reasons why we should keep something in the CORE > > > > > module that is being used at PLUGINS only. This is a lack of > > > > > architecture design that must be fixed for sure; > > > > > - The cache affects cluster stability and require additional human > > > > > resources for the code maintenance; > > > > > - As a straightforward solution plugins can create their own internal > > > > > caches and use them ever they like (or use the metastorage as I > > > > > mentioned earlier). Moving system cache to plugin doesn't look so > > > > > complicated and harmful; > > > > > > > > > > On Mon, 29 Nov 2021 at 23:07, Valentin Kulichenko > > > > > <valentin.kuliche...@gmail.com> wrote: > > > > > > > > > > > > Maxim, > > > > > > > > > > > > You're right that the system cache is still likely to be used by > > > > plugins. > > > > > > We at GridGain use it for security features, for example. As far > > as I > > > > > know, > > > > > > that's not the only case. > > > > > > > > > > > > I also agree that the metastorage should be the preferred way for > > this > > > > > kind > > > > > > of purposes, but is there any harm in keeping the system cache for > > now? > > > > > > Unlike the legacy Service Grid, this is not a public feature that > > can > > > > be > > > > > > used directly by a regular user, so I'm not sure why the rush. > > > > > > > > > > > > How about we instead deprecate the system cache, clearly document > > how > > > > to > > > > > > use the metastorage as an alternative, and then complete the > > removal > > > > > > sometime in the future? > > > > > > > > > > > > -Val > > > > > > > > > > > > On Sun, Nov 28, 2021 at 6:49 AM Maxim Muzafarov <mmu...@apache.org > > > > > > > > wrote: > > > > > > > > > > > > > Igniters, > > > > > > > > > > > > > > > > > > > > > Since the legacy service grid implementation was removed [2] I'd > > like > > > > > > > to remove the ignite-sys-cache also. It is still possible that > > some > > > > of > > > > > > > Ignite plugins (e.g. security plugin) are still using this cache, > > > > > > > however, AFAIK this is not the reason to keep the outdated system > > > > > > > cache and these plugins must be migrated to the metastorage > > instead. > > > > > > > > > > > > > > I've filed the issue [1] for removal. Any objections? > > > > > > > > > > > > > > > > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-16008 > > > > > > > [2] https://issues.apache.org/jira/browse/IGNITE-15758 > > > > > > > > > > > > > > > > > >
