Re: [DISCUSSION] Remove outdated ignite-sys-cache
Maxim, Mid next year is fine, but it makes more sense to bound it by time rather than a specific version. For example, what if we want to release 2.14 earlier for whatever reason? As for the guide, I obviously can't force anyone, but I still believe it is useful, and I hope someone will pick this up. It's also a good practice to document such changes, even for internal purposes. I would do this myself, but I'm afraid my understanding of the topic is not enough :) I think we are on the same page regarding everything else. -Val On Tue, Dec 7, 2021 at 1:46 AM Вячеслав Коптилин wrote: > Hello Maxim, > > The plan should be more precise. So at the very high level: > > > > 1. Deprecate the system cache in 2.13. > > 2. Remove the system cache in 2.14. > > > I still think that we should not be stuck to a concrete version 2.14. > The system cache should be removed when all related improvements will be > done (at least, transaction support on the distributed metastorage). > > Thanks, > S. > > пн, 6 дек. 2021 г. в 20:40, Maxim Muzafarov : > > > Val, > > > > The plan should be more precise. So at the very high level: > > > > 1. Deprecate the system cache in 2.13. > > 2. Remove the system cache in 2.14. > > > > 2.14 should happen in the mid of the next year I suppose. > > > > I don't think we should write guides for the system cache > > > metastorage migration process. This is a completely internal API and > > we can change it. Any plugin can create its own internal cache for any > > needs and if there are any hooks required for it (e.g. like > > PartitionsExchangeAware interface) it might be added prior to the > > removal. > > > > > > After we agreed on this plan we can send a notification for dev, user > > lists about these changes, so all the plugin developers will be up to > > date. But I think that everyone has already known everything, so let's > > move on :-) > > > > On Mon, 6 Dec 2021 at 20:22, Nikolay Izhikov > wrote: > > > > > > Valentin, > > > > > > > However, changes like system cache removal are much more critical, > > because a plugin might rely on it. > > > > > > I’m still don’t understand - what is the difference between system > cache > > and any other Ignite cache except the name? > > > Do we have some special guarantees for system cache? > > > > > > > Any objections to the plan I've suggested earlier? > > > > 1. Write down the differences between the system cache and the > > metastorage, and provide a transition guide for plugin developers. > > > > > > Don’t think we need any transition guide. > > > > > > > I don't think it's reasonable to do this earlier than mid next year > > > > > > This date are questionable, also. > > > > > > Other points of your plan makes sense. > > > +1 to follow it. > > > > > > > 6 дек. 2021 г., в 20:16, Valentin Kulichenko < > > valentin.kuliche...@gmail.com> написал(а): > > > > > > > > That's correct. > > > > > > > > Folks, can we agree on how we want to approach the removal of the > > system > > > > cache? Any objections to the plan I've suggested earlier? As a > > reminder, > > > > here it is: > > > > 1. Write down the differences between the system cache and the > > metastorage, > > > > and provide a transition guide for plugin developers. > > > > 2. Deprecate the system cache in 2.13. > > > > 3. Remove the system cache in one of the further releases. I don't > > think > > > > it's reasonable to do this earlier than mid next year (even that is > > > > potentially too early). > > > > > > > > As for general compatibility requirements related to plugins, I think > > they > > > > obviously should not be as strict as for the public API. It's > > > > understandable that a method signature can be updated, or another > > similar > > > > change can be made in internal APIs. However, changes like system > cache > > > > removal are much more critical, because a plugin might rely on it. > > It's our > > > > responsibility as a good friendly community to provide a reasonable > > > > alternative and a reasonable timeline for such a case. > > > > > > > > -Val > > > > > > > > On Mon, Dec 6, 2021 at 8:56 AM Вячеслав Коптилин < > > slava.kopti...@gmail.com> > > > > wrote: > > > > > > > >> Hi, > > > >> > > > >>> Plugins have access to different internal Ignite components, such > as > > > >> security processor and others, and can extend the programmatic API > of > > > >> Ignite. > > > >> > > > >>> Where docs say that we, as a community, take responsibility to keep > > > >> internals in the way plugin expect? > > > >> Nikolay, it seems to me, that quoted text clearly states about that. > > > >> Plugin has access to internal APIs and so it depends on it. If we > > want to > > > >> be a friendly community then we should discuss such changes > > > >> (removing/changing internal APIs) and provide a reasonable time in > > order to > > > >> update such dependencies. > > > >> > > > >> I think no one in this topic is against removing sys-cache. The > > question > > > >> is: is it suitable
Re: [DISCUSSION] Remove outdated ignite-sys-cache
Hello Maxim, The plan should be more precise. So at the very high level: > > 1. Deprecate the system cache in 2.13. > 2. Remove the system cache in 2.14. > I still think that we should not be stuck to a concrete version 2.14. The system cache should be removed when all related improvements will be done (at least, transaction support on the distributed metastorage). Thanks, S. пн, 6 дек. 2021 г. в 20:40, Maxim Muzafarov : > Val, > > The plan should be more precise. So at the very high level: > > 1. Deprecate the system cache in 2.13. > 2. Remove the system cache in 2.14. > > 2.14 should happen in the mid of the next year I suppose. > > I don't think we should write guides for the system cache > > metastorage migration process. This is a completely internal API and > we can change it. Any plugin can create its own internal cache for any > needs and if there are any hooks required for it (e.g. like > PartitionsExchangeAware interface) it might be added prior to the > removal. > > > After we agreed on this plan we can send a notification for dev, user > lists about these changes, so all the plugin developers will be up to > date. But I think that everyone has already known everything, so let's > move on :-) > > On Mon, 6 Dec 2021 at 20:22, Nikolay Izhikov wrote: > > > > Valentin, > > > > > However, changes like system cache removal are much more critical, > because a plugin might rely on it. > > > > I’m still don’t understand - what is the difference between system cache > and any other Ignite cache except the name? > > Do we have some special guarantees for system cache? > > > > > Any objections to the plan I've suggested earlier? > > > 1. Write down the differences between the system cache and the > metastorage, and provide a transition guide for plugin developers. > > > > Don’t think we need any transition guide. > > > > > I don't think it's reasonable to do this earlier than mid next year > > > > This date are questionable, also. > > > > Other points of your plan makes sense. > > +1 to follow it. > > > > > 6 дек. 2021 г., в 20:16, Valentin Kulichenko < > valentin.kuliche...@gmail.com> написал(а): > > > > > > That's correct. > > > > > > Folks, can we agree on how we want to approach the removal of the > system > > > cache? Any objections to the plan I've suggested earlier? As a > reminder, > > > here it is: > > > 1. Write down the differences between the system cache and the > metastorage, > > > and provide a transition guide for plugin developers. > > > 2. Deprecate the system cache in 2.13. > > > 3. Remove the system cache in one of the further releases. I don't > think > > > it's reasonable to do this earlier than mid next year (even that is > > > potentially too early). > > > > > > As for general compatibility requirements related to plugins, I think > they > > > obviously should not be as strict as for the public API. It's > > > understandable that a method signature can be updated, or another > similar > > > change can be made in internal APIs. However, changes like system cache > > > removal are much more critical, because a plugin might rely on it. > It's our > > > responsibility as a good friendly community to provide a reasonable > > > alternative and a reasonable timeline for such a case. > > > > > > -Val > > > > > > On Mon, Dec 6, 2021 at 8:56 AM Вячеслав Коптилин < > slava.kopti...@gmail.com> > > > wrote: > > > > > >> Hi, > > >> > > >>> Plugins have access to different internal Ignite components, such as > > >> security processor and others, and can extend the programmatic API of > > >> Ignite. > > >> > > >>> Where docs say that we, as a community, take responsibility to keep > > >> internals in the way plugin expect? > > >> Nikolay, it seems to me, that quoted text clearly states about that. > > >> Plugin has access to internal APIs and so it depends on it. If we > want to > > >> be a friendly community then we should discuss such changes > > >> (removing/changing internal APIs) and provide a reasonable time in > order to > > >> update such dependencies. > > >> > > >> I think no one in this topic is against removing sys-cache. The > question > > >> is: is it suitable for the community to deprecate using system cache > in > > >> 2.13 and removing it in 2.14 || 2.15? > > >> Am I missing something? Am I correct? > > >> > > >> Thanks, > > >> Slava. > > >> > > >> > > >> пн, 6 дек. 2021 г. в 15:00, Nikolay Izhikov : > > >> > > >>> Slava, I don’t get it. > > >>> > > Plugins have access to different internal Ignite components, such as > > >>> security processor and others, and can extend the programmatic API of > > >>> Ignite. > > >>> > > >>> Where docs say that we, as a community, take responsibility to keep > > >>> internals in the way plugin expect? > > >>> > > 6 дек. 2021 г., в 14:48, Вячеслав Коптилин < > slava.kopti...@gmail.com> > > >>> написал(а): > > > > Hello Nikolay, > > > > >> plugin framework allows to implement internal components and use > the > >
Re: [DISCUSSION] Remove outdated ignite-sys-cache
Val, The plan should be more precise. So at the very high level: 1. Deprecate the system cache in 2.13. 2. Remove the system cache in 2.14. 2.14 should happen in the mid of the next year I suppose. I don't think we should write guides for the system cache > metastorage migration process. This is a completely internal API and we can change it. Any plugin can create its own internal cache for any needs and if there are any hooks required for it (e.g. like PartitionsExchangeAware interface) it might be added prior to the removal. After we agreed on this plan we can send a notification for dev, user lists about these changes, so all the plugin developers will be up to date. But I think that everyone has already known everything, so let's move on :-) On Mon, 6 Dec 2021 at 20:22, Nikolay Izhikov wrote: > > Valentin, > > > However, changes like system cache removal are much more critical, because > > a plugin might rely on it. > > I’m still don’t understand - what is the difference between system cache and > any other Ignite cache except the name? > Do we have some special guarantees for system cache? > > > Any objections to the plan I've suggested earlier? > > 1. Write down the differences between the system cache and the metastorage, > > and provide a transition guide for plugin developers. > > Don’t think we need any transition guide. > > > I don't think it's reasonable to do this earlier than mid next year > > This date are questionable, also. > > Other points of your plan makes sense. > +1 to follow it. > > > 6 дек. 2021 г., в 20:16, Valentin Kulichenko > > написал(а): > > > > That's correct. > > > > Folks, can we agree on how we want to approach the removal of the system > > cache? Any objections to the plan I've suggested earlier? As a reminder, > > here it is: > > 1. Write down the differences between the system cache and the metastorage, > > and provide a transition guide for plugin developers. > > 2. Deprecate the system cache in 2.13. > > 3. Remove the system cache in one of the further releases. I don't think > > it's reasonable to do this earlier than mid next year (even that is > > potentially too early). > > > > As for general compatibility requirements related to plugins, I think they > > obviously should not be as strict as for the public API. It's > > understandable that a method signature can be updated, or another similar > > change can be made in internal APIs. However, changes like system cache > > removal are much more critical, because a plugin might rely on it. It's our > > responsibility as a good friendly community to provide a reasonable > > alternative and a reasonable timeline for such a case. > > > > -Val > > > > On Mon, Dec 6, 2021 at 8:56 AM Вячеслав Коптилин > > wrote: > > > >> Hi, > >> > >>> Plugins have access to different internal Ignite components, such as > >> security processor and others, and can extend the programmatic API of > >> Ignite. > >> > >>> Where docs say that we, as a community, take responsibility to keep > >> internals in the way plugin expect? > >> Nikolay, it seems to me, that quoted text clearly states about that. > >> Plugin has access to internal APIs and so it depends on it. If we want to > >> be a friendly community then we should discuss such changes > >> (removing/changing internal APIs) and provide a reasonable time in order to > >> update such dependencies. > >> > >> I think no one in this topic is against removing sys-cache. The question > >> is: is it suitable for the community to deprecate using system cache in > >> 2.13 and removing it in 2.14 || 2.15? > >> Am I missing something? Am I correct? > >> > >> Thanks, > >> Slava. > >> > >> > >> пн, 6 дек. 2021 г. в 15:00, Nikolay Izhikov : > >> > >>> Slava, I don’t get it. > >>> > Plugins have access to different internal Ignite components, such as > >>> security processor and others, and can extend the programmatic API of > >>> Ignite. > >>> > >>> Where docs say that we, as a community, take responsibility to keep > >>> internals in the way plugin expect? > >>> > 6 дек. 2021 г., в 14:48, Вячеслав Коптилин > >>> написал(а): > > Hello Nikolay, > > >> plugin framework allows to implement internal components and use the > internal API. > > Can you please, point out to documentation or place in Ignite code > >> that > describe this behaviour? > > Looks like it states here > >> https://ignite.apache.org/docs/latest/plugins > > > The Ignite plugin system allows you to extend the core functionality > >> of > > Ignite. Plugins have access to different internal Ignite components, > >>> such > > as security processor and others, and can extend the programmatic API > >> of > > Ignite. > > > Thanks, > S. > > > сб, 4 дек. 2021 г. в 14:54, Nikolay Izhikov : > > > Valentin > > > >> plugin framework allows to implement internal components and use the > > internal API. >
Re: [DISCUSSION] Remove outdated ignite-sys-cache
Valentin, > However, changes like system cache removal are much more critical, because a > plugin might rely on it. I’m still don’t understand - what is the difference between system cache and any other Ignite cache except the name? Do we have some special guarantees for system cache? > Any objections to the plan I've suggested earlier? > 1. Write down the differences between the system cache and the metastorage, > and provide a transition guide for plugin developers. Don’t think we need any transition guide. > I don't think it's reasonable to do this earlier than mid next year This date are questionable, also. Other points of your plan makes sense. +1 to follow it. > 6 дек. 2021 г., в 20:16, Valentin Kulichenko > написал(а): > > That's correct. > > Folks, can we agree on how we want to approach the removal of the system > cache? Any objections to the plan I've suggested earlier? As a reminder, > here it is: > 1. Write down the differences between the system cache and the metastorage, > and provide a transition guide for plugin developers. > 2. Deprecate the system cache in 2.13. > 3. Remove the system cache in one of the further releases. I don't think > it's reasonable to do this earlier than mid next year (even that is > potentially too early). > > As for general compatibility requirements related to plugins, I think they > obviously should not be as strict as for the public API. It's > understandable that a method signature can be updated, or another similar > change can be made in internal APIs. However, changes like system cache > removal are much more critical, because a plugin might rely on it. It's our > responsibility as a good friendly community to provide a reasonable > alternative and a reasonable timeline for such a case. > > -Val > > On Mon, Dec 6, 2021 at 8:56 AM Вячеслав Коптилин > wrote: > >> Hi, >> >>> Plugins have access to different internal Ignite components, such as >> security processor and others, and can extend the programmatic API of >> Ignite. >> >>> Where docs say that we, as a community, take responsibility to keep >> internals in the way plugin expect? >> Nikolay, it seems to me, that quoted text clearly states about that. >> Plugin has access to internal APIs and so it depends on it. If we want to >> be a friendly community then we should discuss such changes >> (removing/changing internal APIs) and provide a reasonable time in order to >> update such dependencies. >> >> I think no one in this topic is against removing sys-cache. The question >> is: is it suitable for the community to deprecate using system cache in >> 2.13 and removing it in 2.14 || 2.15? >> Am I missing something? Am I correct? >> >> Thanks, >> Slava. >> >> >> пн, 6 дек. 2021 г. в 15:00, Nikolay Izhikov : >> >>> Slava, I don’t get it. >>> Plugins have access to different internal Ignite components, such as >>> security processor and others, and can extend the programmatic API of >>> Ignite. >>> >>> Where docs say that we, as a community, take responsibility to keep >>> internals in the way plugin expect? >>> 6 дек. 2021 г., в 14:48, Вячеслав Коптилин >>> написал(а): Hello Nikolay, >> plugin framework allows to implement internal components and use the internal API. > Can you please, point out to documentation or place in Ignite code >> that describe this behaviour? Looks like it states here >> https://ignite.apache.org/docs/latest/plugins > The Ignite plugin system allows you to extend the core functionality >> of > Ignite. Plugins have access to different internal Ignite components, >>> such > as security processor and others, and can extend the programmatic API >> of > Ignite. Thanks, S. сб, 4 дек. 2021 г. в 14:54, Nikolay Izhikov : > Valentin > >> plugin framework allows to implement internal components and use the > internal API. > > Can you please, point out to documentation or place in Ignite code >> that > describe this behaviour? > AFAIK plugin can only use public API, internal API can be changed any >>> time > we want. > >> Folks, can anyone explain the rush? > > No rush from my side. > > Just want to understand your vision of integration points between >> Ignite > and plugins. > >> Is there any specific reason for it? > > Intention of IEP-80 is to improve Ignite codebase by removing >> abandoned > features. > >> But the fact is that there are users that can depend on the system >>> cache > via the plugin framework. > > Why this dependency can’t be changed to any regular cache? > Just replace `Ignite#cache` to `Ignite#getOrCreateCache` and that’s >> it. > > Do we have some special guarantees or logic besides system cache? > > >> 4 дек. 2021 г., в 02:14, Valentin Kulichenko < >
Re: [DISCUSSION] Remove outdated ignite-sys-cache
That's correct. Folks, can we agree on how we want to approach the removal of the system cache? Any objections to the plan I've suggested earlier? As a reminder, here it is: 1. Write down the differences between the system cache and the metastorage, and provide a transition guide for plugin developers. 2. Deprecate the system cache in 2.13. 3. Remove the system cache in one of the further releases. I don't think it's reasonable to do this earlier than mid next year (even that is potentially too early). As for general compatibility requirements related to plugins, I think they obviously should not be as strict as for the public API. It's understandable that a method signature can be updated, or another similar change can be made in internal APIs. However, changes like system cache removal are much more critical, because a plugin might rely on it. It's our responsibility as a good friendly community to provide a reasonable alternative and a reasonable timeline for such a case. -Val On Mon, Dec 6, 2021 at 8:56 AM Вячеслав Коптилин wrote: > Hi, > > > Plugins have access to different internal Ignite components, such as > security processor and others, and can extend the programmatic API of > Ignite. > > > Where docs say that we, as a community, take responsibility to keep > internals in the way plugin expect? > Nikolay, it seems to me, that quoted text clearly states about that. > Plugin has access to internal APIs and so it depends on it. If we want to > be a friendly community then we should discuss such changes > (removing/changing internal APIs) and provide a reasonable time in order to > update such dependencies. > > I think no one in this topic is against removing sys-cache. The question > is: is it suitable for the community to deprecate using system cache in > 2.13 and removing it in 2.14 || 2.15? > Am I missing something? Am I correct? > > Thanks, > Slava. > > > пн, 6 дек. 2021 г. в 15:00, Nikolay Izhikov : > > > Slava, I don’t get it. > > > > > Plugins have access to different internal Ignite components, such as > > security processor and others, and can extend the programmatic API of > > Ignite. > > > > Where docs say that we, as a community, take responsibility to keep > > internals in the way plugin expect? > > > > > 6 дек. 2021 г., в 14:48, Вячеслав Коптилин > > написал(а): > > > > > > Hello Nikolay, > > > > > >>> plugin framework allows to implement internal components and use the > > > internal API. > > >> Can you please, point out to documentation or place in Ignite code > that > > > describe this behaviour? > > > > > > Looks like it states here > https://ignite.apache.org/docs/latest/plugins > > > > > >> The Ignite plugin system allows you to extend the core functionality > of > > >> Ignite. Plugins have access to different internal Ignite components, > > such > > >> as security processor and others, and can extend the programmatic API > of > > >> Ignite. > > > > > > > > > Thanks, > > > S. > > > > > > > > > сб, 4 дек. 2021 г. в 14:54, Nikolay Izhikov : > > > > > >> Valentin > > >> > > >>> plugin framework allows to implement internal components and use the > > >> internal API. > > >> > > >> Can you please, point out to documentation or place in Ignite code > that > > >> describe this behaviour? > > >> AFAIK plugin can only use public API, internal API can be changed any > > time > > >> we want. > > >> > > >>> Folks, can anyone explain the rush? > > >> > > >> No rush from my side. > > >> > > >> Just want to understand your vision of integration points between > Ignite > > >> and plugins. > > >> > > >>> Is there any specific reason for it? > > >> > > >> Intention of IEP-80 is to improve Ignite codebase by removing > abandoned > > >> features. > > >> > > >>> But the fact is that there are users that can depend on the system > > cache > > >> via the plugin framework. > > >> > > >> Why this dependency can’t be changed to any regular cache? > > >> Just replace `Ignite#cache` to `Ignite#getOrCreateCache` and that’s > it. > > >> > > >> Do we have some special guarantees or logic besides system cache? > > >> > > >> > > >>> 4 дек. 2021 г., в 02:14, Valentin Kulichenko < > > >> valentin.kuliche...@gmail.com> написал(а): > > >>> > > >>> Nikolay, that's right, plugin framework allows to implement internal > > >>> components and use the internal API. There is a difference between a > > >> plugin > > >>> and an extension that uses only public API > > >>> > > >>> Folks, can anyone explain the rush? Is there any specific reason for > > it? > > >>> > > >>> I think we all agree that this is a good change - system cache has to > > >> go. I > > >>> guess technically it's not even a breaking change, because system > cache > > >> is > > >>> not a public API feature. But the fact is that there are users that > can > > >>> depend on the system cache via the plugin framework. Let's be > > respectful > > >> to > > >>> those users, provide a reasonable documented alternative and give > time. > > >>> > > >>> -Val
Re: [DISCUSSION] Remove outdated ignite-sys-cache
Hi, > Plugins have access to different internal Ignite components, such as security processor and others, and can extend the programmatic API of Ignite. > Where docs say that we, as a community, take responsibility to keep internals in the way plugin expect? Nikolay, it seems to me, that quoted text clearly states about that. Plugin has access to internal APIs and so it depends on it. If we want to be a friendly community then we should discuss such changes (removing/changing internal APIs) and provide a reasonable time in order to update such dependencies. I think no one in this topic is against removing sys-cache. The question is: is it suitable for the community to deprecate using system cache in 2.13 and removing it in 2.14 || 2.15? Am I missing something? Am I correct? Thanks, Slava. пн, 6 дек. 2021 г. в 15:00, Nikolay Izhikov : > Slava, I don’t get it. > > > Plugins have access to different internal Ignite components, such as > security processor and others, and can extend the programmatic API of > Ignite. > > Where docs say that we, as a community, take responsibility to keep > internals in the way plugin expect? > > > 6 дек. 2021 г., в 14:48, Вячеслав Коптилин > написал(а): > > > > Hello Nikolay, > > > >>> plugin framework allows to implement internal components and use the > > internal API. > >> Can you please, point out to documentation or place in Ignite code that > > describe this behaviour? > > > > Looks like it states here https://ignite.apache.org/docs/latest/plugins > > > >> The Ignite plugin system allows you to extend the core functionality of > >> Ignite. Plugins have access to different internal Ignite components, > such > >> as security processor and others, and can extend the programmatic API of > >> Ignite. > > > > > > Thanks, > > S. > > > > > > сб, 4 дек. 2021 г. в 14:54, Nikolay Izhikov : > > > >> Valentin > >> > >>> plugin framework allows to implement internal components and use the > >> internal API. > >> > >> Can you please, point out to documentation or place in Ignite code that > >> describe this behaviour? > >> AFAIK plugin can only use public API, internal API can be changed any > time > >> we want. > >> > >>> Folks, can anyone explain the rush? > >> > >> No rush from my side. > >> > >> Just want to understand your vision of integration points between Ignite > >> and plugins. > >> > >>> Is there any specific reason for it? > >> > >> Intention of IEP-80 is to improve Ignite codebase by removing abandoned > >> features. > >> > >>> But the fact is that there are users that can depend on the system > cache > >> via the plugin framework. > >> > >> Why this dependency can’t be changed to any regular cache? > >> Just replace `Ignite#cache` to `Ignite#getOrCreateCache` and that’s it. > >> > >> Do we have some special guarantees or logic besides system cache? > >> > >> > >>> 4 дек. 2021 г., в 02:14, Valentin Kulichenko < > >> valentin.kuliche...@gmail.com> написал(а): > >>> > >>> Nikolay, that's right, plugin framework allows to implement internal > >>> components and use the internal API. There is a difference between a > >> plugin > >>> and an extension that uses only public API > >>> > >>> Folks, can anyone explain the rush? Is there any specific reason for > it? > >>> > >>> I think we all agree that this is a good change - system cache has to > >> go. I > >>> guess technically it's not even a breaking change, because system cache > >> is > >>> not a public API feature. But the fact is that there are users that can > >>> depend on the system cache via the plugin framework. Let's be > respectful > >> to > >>> those users, provide a reasonable documented alternative and give time. > >>> > >>> -Val > >>> > >>> On Fri, Dec 3, 2021 at 8:18 AM Nikolay Izhikov > >> wrote: > >>> > I don’t understand how it possible. > > Are you talking about plugin that uses some kind of internal API? > > > 3 дек. 2021 г., в 18:50, Вячеслав Коптилин > > написал(а): > > > > Hello Nikolay, > > > > If I am not mistaken, the method you mentioned, allows to create a > >> "user" > > cache that is available through public api for the user. This does > not > > cover the case when the plugin developer wants to hide such cache and > > protect it form the end user (at least, via public api). > > > > Thanks, > > S. > > > > пт, 3 дек. 2021 г., 14:15 Nikolay Izhikov : > > > >> Vyacheslav, Val, can you please clarify - What is the issue if > third-party > >> plugins will create «ignite-sys-cache» from the code? > >> > >> Like just replacing `Ignite#cache` with the > `Ignite#getOrCreateCache`. > >> > >>> 2 дек. 2021 г., в 16:13, Вячеслав Коптилин < > slava.kopti...@gmail.com > >>> > >> написал(а): > >>> > >>> Hello Maxim, > >>> > I don't understand why you are using *backwards compatibility* for > >>> completely internal things. > Why you are thinking in terms of
Re: [DISCUSSION] Remove outdated ignite-sys-cache
Slava, I don’t get it. > Plugins have access to different internal Ignite components, such as security > processor and others, and can extend the programmatic API of Ignite. Where docs say that we, as a community, take responsibility to keep internals in the way plugin expect? > 6 дек. 2021 г., в 14:48, Вячеслав Коптилин > написал(а): > > Hello Nikolay, > >>> plugin framework allows to implement internal components and use the > internal API. >> Can you please, point out to documentation or place in Ignite code that > describe this behaviour? > > Looks like it states here https://ignite.apache.org/docs/latest/plugins > >> The Ignite plugin system allows you to extend the core functionality of >> Ignite. Plugins have access to different internal Ignite components, such >> as security processor and others, and can extend the programmatic API of >> Ignite. > > > Thanks, > S. > > > сб, 4 дек. 2021 г. в 14:54, Nikolay Izhikov : > >> Valentin >> >>> plugin framework allows to implement internal components and use the >> internal API. >> >> Can you please, point out to documentation or place in Ignite code that >> describe this behaviour? >> AFAIK plugin can only use public API, internal API can be changed any time >> we want. >> >>> Folks, can anyone explain the rush? >> >> No rush from my side. >> >> Just want to understand your vision of integration points between Ignite >> and plugins. >> >>> Is there any specific reason for it? >> >> Intention of IEP-80 is to improve Ignite codebase by removing abandoned >> features. >> >>> But the fact is that there are users that can depend on the system cache >> via the plugin framework. >> >> Why this dependency can’t be changed to any regular cache? >> Just replace `Ignite#cache` to `Ignite#getOrCreateCache` and that’s it. >> >> Do we have some special guarantees or logic besides system cache? >> >> >>> 4 дек. 2021 г., в 02:14, Valentin Kulichenko < >> valentin.kuliche...@gmail.com> написал(а): >>> >>> Nikolay, that's right, plugin framework allows to implement internal >>> components and use the internal API. There is a difference between a >> plugin >>> and an extension that uses only public API >>> >>> Folks, can anyone explain the rush? Is there any specific reason for it? >>> >>> I think we all agree that this is a good change - system cache has to >> go. I >>> guess technically it's not even a breaking change, because system cache >> is >>> not a public API feature. But the fact is that there are users that can >>> depend on the system cache via the plugin framework. Let's be respectful >> to >>> those users, provide a reasonable documented alternative and give time. >>> >>> -Val >>> >>> On Fri, Dec 3, 2021 at 8:18 AM Nikolay Izhikov >> wrote: >>> I don’t understand how it possible. Are you talking about plugin that uses some kind of internal API? > 3 дек. 2021 г., в 18:50, Вячеслав Коптилин написал(а): > > Hello Nikolay, > > If I am not mistaken, the method you mentioned, allows to create a >> "user" > cache that is available through public api for the user. This does not > cover the case when the plugin developer wants to hide such cache and > protect it form the end user (at least, via public api). > > Thanks, > S. > > пт, 3 дек. 2021 г., 14:15 Nikolay Izhikov : > >> Vyacheslav, Val, can you please clarify - What is the issue if third-party >> plugins will create «ignite-sys-cache» from the code? >> >> Like just replacing `Ignite#cache` with the `Ignite#getOrCreateCache`. >> >>> 2 дек. 2021 г., в 16:13, Вячеслав Коптилин >> >> написал(а): >>> >>> Hello Maxim, >>> I don't understand why you are using *backwards compatibility* for >>> completely internal things. Why you are thinking in terms of users usage when are talking about >>> ignite-sys-cache but not thinking when refactoring >>> First of all, we are talking about all plugin developers. It does not >>> matter it is open-source or proprietary plugins. >>> Honestly, I don't have a list of all possible plugins that have >> already >>> been developed and available under the ASF license, for instance. >>> Do you have such a list? Can you be sure that this change will not affect >>> anyone? >>> I don't understand why you are using *backwards compatibility* for >>> completely internal things. Why you are thinking in terms of users usage when are talking about >>> ignite-sys-cache but not thinking when refactoring >>> The system cache was the crucial thing in the architecture of Apache >> Ignite >>> 1.x and 2.x (at least 2.1 - 2.11?) >>> All the internals have been reworked and nobody even mentioned that >> it >>> may affect some of the plugin developers. >>> Yep, perhaps, some internal parts of Apache Ignite were reworked
Re: [DISCUSSION] Remove outdated ignite-sys-cache
Hello Nikolay, >> plugin framework allows to implement internal components and use the internal API. > Can you please, point out to documentation or place in Ignite code that describe this behaviour? Looks like it states here https://ignite.apache.org/docs/latest/plugins > The Ignite plugin system allows you to extend the core functionality of > Ignite. Plugins have access to different internal Ignite components, such > as security processor and others, and can extend the programmatic API of > Ignite. Thanks, S. сб, 4 дек. 2021 г. в 14:54, Nikolay Izhikov : > Valentin > > > plugin framework allows to implement internal components and use the > internal API. > > Can you please, point out to documentation or place in Ignite code that > describe this behaviour? > AFAIK plugin can only use public API, internal API can be changed any time > we want. > > > Folks, can anyone explain the rush? > > No rush from my side. > > Just want to understand your vision of integration points between Ignite > and plugins. > > > Is there any specific reason for it? > > Intention of IEP-80 is to improve Ignite codebase by removing abandoned > features. > > > But the fact is that there are users that can depend on the system cache > via the plugin framework. > > Why this dependency can’t be changed to any regular cache? > Just replace `Ignite#cache` to `Ignite#getOrCreateCache` and that’s it. > > Do we have some special guarantees or logic besides system cache? > > > > 4 дек. 2021 г., в 02:14, Valentin Kulichenko < > valentin.kuliche...@gmail.com> написал(а): > > > > Nikolay, that's right, plugin framework allows to implement internal > > components and use the internal API. There is a difference between a > plugin > > and an extension that uses only public API > > > > Folks, can anyone explain the rush? Is there any specific reason for it? > > > > I think we all agree that this is a good change - system cache has to > go. I > > guess technically it's not even a breaking change, because system cache > is > > not a public API feature. But the fact is that there are users that can > > depend on the system cache via the plugin framework. Let's be respectful > to > > those users, provide a reasonable documented alternative and give time. > > > > -Val > > > > On Fri, Dec 3, 2021 at 8:18 AM Nikolay Izhikov > wrote: > > > >> I don’t understand how it possible. > >> > >> Are you talking about plugin that uses some kind of internal API? > >> > >>> 3 дек. 2021 г., в 18:50, Вячеслав Коптилин > >> написал(а): > >>> > >>> Hello Nikolay, > >>> > >>> If I am not mistaken, the method you mentioned, allows to create a > "user" > >>> cache that is available through public api for the user. This does not > >>> cover the case when the plugin developer wants to hide such cache and > >>> protect it form the end user (at least, via public api). > >>> > >>> Thanks, > >>> S. > >>> > >>> пт, 3 дек. 2021 г., 14:15 Nikolay Izhikov : > >>> > Vyacheslav, Val, can you please clarify - What is the issue if > >> third-party > plugins will create «ignite-sys-cache» from the code? > > Like just replacing `Ignite#cache` with the `Ignite#getOrCreateCache`. > > > 2 дек. 2021 г., в 16:13, Вячеслав Коптилин > > написал(а): > > > > Hello Maxim, > > > >> I don't understand why you are using *backwards compatibility* for > > completely internal things. > >> Why you are thinking in terms of users usage when are talking about > > ignite-sys-cache but not thinking when refactoring > > First of all, we are talking about all plugin developers. It does not > > matter it is open-source or proprietary plugins. > > Honestly, I don't have a list of all possible plugins that have > already > > been developed and available under the ASF license, for instance. > > Do you have such a list? Can you be sure that this change will not > >> affect > > anyone? > > > >> I don't understand why you are using *backwards compatibility* for > > completely internal things. > >> Why you are thinking in terms of users usage when are talking about > > ignite-sys-cache but not thinking when refactoring > > The system cache was the crucial thing in the architecture of Apache > Ignite > > 1.x and 2.x (at least 2.1 - 2.11?) > > > >> All the internals have been reworked and nobody even mentioned that > it > > may affect some of the plugin developers. > > Yep, perhaps, some internal parts of Apache Ignite were reworked in > >> order > > to avoid using the system cache. > > However, it is still a part of Ignite and it works and can be used in > > plugins. > > Honestly, the mentioned alternative, I mean the distributed > >> metastorage, > is > > the INTERNAL thing as well. > > It means that plugin developer depends on INTERNAL entities. (it does > >> not > > matter system-cache/metastorage) > > IMHO, such questions should be
Re: [DISCUSSION] Remove outdated ignite-sys-cache
Valentin > plugin framework allows to implement internal components and use the > internal API. Can you please, point out to documentation or place in Ignite code that describe this behaviour? AFAIK plugin can only use public API, internal API can be changed any time we want. > Folks, can anyone explain the rush? No rush from my side. Just want to understand your vision of integration points between Ignite and plugins. > Is there any specific reason for it? Intention of IEP-80 is to improve Ignite codebase by removing abandoned features. > But the fact is that there are users that can depend on the system cache via > the plugin framework. Why this dependency can’t be changed to any regular cache? Just replace `Ignite#cache` to `Ignite#getOrCreateCache` and that’s it. Do we have some special guarantees or logic besides system cache? > 4 дек. 2021 г., в 02:14, Valentin Kulichenko > написал(а): > > Nikolay, that's right, plugin framework allows to implement internal > components and use the internal API. There is a difference between a plugin > and an extension that uses only public API > > Folks, can anyone explain the rush? Is there any specific reason for it? > > I think we all agree that this is a good change - system cache has to go. I > guess technically it's not even a breaking change, because system cache is > not a public API feature. But the fact is that there are users that can > depend on the system cache via the plugin framework. Let's be respectful to > those users, provide a reasonable documented alternative and give time. > > -Val > > On Fri, Dec 3, 2021 at 8:18 AM Nikolay Izhikov wrote: > >> I don’t understand how it possible. >> >> Are you talking about plugin that uses some kind of internal API? >> >>> 3 дек. 2021 г., в 18:50, Вячеслав Коптилин >> написал(а): >>> >>> Hello Nikolay, >>> >>> If I am not mistaken, the method you mentioned, allows to create a "user" >>> cache that is available through public api for the user. This does not >>> cover the case when the plugin developer wants to hide such cache and >>> protect it form the end user (at least, via public api). >>> >>> Thanks, >>> S. >>> >>> пт, 3 дек. 2021 г., 14:15 Nikolay Izhikov : >>> Vyacheslav, Val, can you please clarify - What is the issue if >> third-party plugins will create «ignite-sys-cache» from the code? Like just replacing `Ignite#cache` with the `Ignite#getOrCreateCache`. > 2 дек. 2021 г., в 16:13, Вячеслав Коптилин написал(а): > > Hello Maxim, > >> I don't understand why you are using *backwards compatibility* for > completely internal things. >> Why you are thinking in terms of users usage when are talking about > ignite-sys-cache but not thinking when refactoring > First of all, we are talking about all plugin developers. It does not > matter it is open-source or proprietary plugins. > Honestly, I don't have a list of all possible plugins that have already > been developed and available under the ASF license, for instance. > Do you have such a list? Can you be sure that this change will not >> affect > anyone? > >> I don't understand why you are using *backwards compatibility* for > completely internal things. >> Why you are thinking in terms of users usage when are talking about > ignite-sys-cache but not thinking when refactoring > The system cache was the crucial thing in the architecture of Apache Ignite > 1.x and 2.x (at least 2.1 - 2.11?) > >> All the internals have been reworked and nobody even mentioned that it > may affect some of the plugin developers. > Yep, perhaps, some internal parts of Apache Ignite were reworked in >> order > to avoid using the system cache. > However, it is still a part of Ignite and it works and can be used in > plugins. > Honestly, the mentioned alternative, I mean the distributed >> metastorage, is > the INTERNAL thing as well. > It means that plugin developer depends on INTERNAL entities. (it does >> not > matter system-cache/metastorage) > IMHO, such questions should be CAREFULLY discussed with no rush. > >> I do not propose to rush with the ignite-sys-cache removal. I'd like >> to > collect all the technical stuff that we depend on, fix all of them and > after everything will be ready do a final removal. > Good! We are on the same page! > >> 1. Add deprecation for the 2.12 release. I hope it is still possible. > If I am not mistaken, the code freeze was October 29. I think it is too > late to introduce this change. We can add deprecation for the 2.13 release. > >> Apache Ignite core still have some dependencies on ignite-sys-cache >> that > should fix for 2.13 > Ok, I agree. We need to try to find out all edge cases and add new > abilities to the metastorage in order to cover all known >
Re: [DISCUSSION] Remove outdated ignite-sys-cache
Nikolay, that's right, plugin framework allows to implement internal components and use the internal API. There is a difference between a plugin and an extension that uses only public API Folks, can anyone explain the rush? Is there any specific reason for it? I think we all agree that this is a good change - system cache has to go. I guess technically it's not even a breaking change, because system cache is not a public API feature. But the fact is that there are users that can depend on the system cache via the plugin framework. Let's be respectful to those users, provide a reasonable documented alternative and give time. -Val On Fri, Dec 3, 2021 at 8:18 AM Nikolay Izhikov wrote: > I don’t understand how it possible. > > Are you talking about plugin that uses some kind of internal API? > > > 3 дек. 2021 г., в 18:50, Вячеслав Коптилин > написал(а): > > > > Hello Nikolay, > > > > If I am not mistaken, the method you mentioned, allows to create a "user" > > cache that is available through public api for the user. This does not > > cover the case when the plugin developer wants to hide such cache and > > protect it form the end user (at least, via public api). > > > > Thanks, > > S. > > > > пт, 3 дек. 2021 г., 14:15 Nikolay Izhikov : > > > >> Vyacheslav, Val, can you please clarify - What is the issue if > third-party > >> plugins will create «ignite-sys-cache» from the code? > >> > >> Like just replacing `Ignite#cache` with the `Ignite#getOrCreateCache`. > >> > >>> 2 дек. 2021 г., в 16:13, Вячеслав Коптилин > >> написал(а): > >>> > >>> Hello Maxim, > >>> > I don't understand why you are using *backwards compatibility* for > >>> completely internal things. > Why you are thinking in terms of users usage when are talking about > >>> ignite-sys-cache but not thinking when refactoring > >>> First of all, we are talking about all plugin developers. It does not > >>> matter it is open-source or proprietary plugins. > >>> Honestly, I don't have a list of all possible plugins that have already > >>> been developed and available under the ASF license, for instance. > >>> Do you have such a list? Can you be sure that this change will not > affect > >>> anyone? > >>> > I don't understand why you are using *backwards compatibility* for > >>> completely internal things. > Why you are thinking in terms of users usage when are talking about > >>> ignite-sys-cache but not thinking when refactoring > >>> The system cache was the crucial thing in the architecture of Apache > >> Ignite > >>> 1.x and 2.x (at least 2.1 - 2.11?) > >>> > All the internals have been reworked and nobody even mentioned that it > >>> may affect some of the plugin developers. > >>> Yep, perhaps, some internal parts of Apache Ignite were reworked in > order > >>> to avoid using the system cache. > >>> However, it is still a part of Ignite and it works and can be used in > >>> plugins. > >>> Honestly, the mentioned alternative, I mean the distributed > metastorage, > >> is > >>> the INTERNAL thing as well. > >>> It means that plugin developer depends on INTERNAL entities. (it does > not > >>> matter system-cache/metastorage) > >>> IMHO, such questions should be CAREFULLY discussed with no rush. > >>> > I do not propose to rush with the ignite-sys-cache removal. I'd like > to > >>> collect all the technical stuff that we depend on, fix all of them and > >>> after everything will be ready do a final removal. > >>> Good! We are on the same page! > >>> > 1. Add deprecation for the 2.12 release. I hope it is still possible. > >>> If I am not mistaken, the code freeze was October 29. I think it is too > >>> late to introduce this change. We can add deprecation for the 2.13 > >> release. > >>> > Apache Ignite core still have some dependencies on ignite-sys-cache > that > >>> should fix for 2.13 > >>> Ok, I agree. We need to try to find out all edge cases and add new > >>> abilities to the metastorage in order to cover all known > >>> issues/requirements. > >>> > Remove the system cache in 2.14. > >>> It depends on our progress with improving the distributed metastorage. > In > >>> general, I hope it will be possible. > >>> > >>> Thanks, > >>> S. > >>> > >>> чт, 2 дек. 2021 г. в 13:36, Maxim Muzafarov : > >>> > Pavel, > > > I don't understand why you are using *backwards compatibility* for > completely internal things. Why you are thinking in terms of users > usage when are talking about ignite-sys-cache but not thinking when > refactoring, for instance, all the checkpoint classes? Take a look at > the [1] issue. All the internals have been reworked and nobody even > mentioned that it may affect some of the plugin developers. This is > really strange, so I can't agree with you. > > > I do not propose to rush with the ignite-sys-cache removal. I'd like > to collect all the technical stuff that we depend on, fix all of them > and after
Re: [DISCUSSION] Remove outdated ignite-sys-cache
I don’t understand how it possible. Are you talking about plugin that uses some kind of internal API? > 3 дек. 2021 г., в 18:50, Вячеслав Коптилин > написал(а): > > Hello Nikolay, > > If I am not mistaken, the method you mentioned, allows to create a "user" > cache that is available through public api for the user. This does not > cover the case when the plugin developer wants to hide such cache and > protect it form the end user (at least, via public api). > > Thanks, > S. > > пт, 3 дек. 2021 г., 14:15 Nikolay Izhikov : > >> Vyacheslav, Val, can you please clarify - What is the issue if third-party >> plugins will create «ignite-sys-cache» from the code? >> >> Like just replacing `Ignite#cache` with the `Ignite#getOrCreateCache`. >> >>> 2 дек. 2021 г., в 16:13, Вячеслав Коптилин >> написал(а): >>> >>> Hello Maxim, >>> I don't understand why you are using *backwards compatibility* for >>> completely internal things. Why you are thinking in terms of users usage when are talking about >>> ignite-sys-cache but not thinking when refactoring >>> First of all, we are talking about all plugin developers. It does not >>> matter it is open-source or proprietary plugins. >>> Honestly, I don't have a list of all possible plugins that have already >>> been developed and available under the ASF license, for instance. >>> Do you have such a list? Can you be sure that this change will not affect >>> anyone? >>> I don't understand why you are using *backwards compatibility* for >>> completely internal things. Why you are thinking in terms of users usage when are talking about >>> ignite-sys-cache but not thinking when refactoring >>> The system cache was the crucial thing in the architecture of Apache >> Ignite >>> 1.x and 2.x (at least 2.1 - 2.11?) >>> All the internals have been reworked and nobody even mentioned that it >>> may affect some of the plugin developers. >>> Yep, perhaps, some internal parts of Apache Ignite were reworked in order >>> to avoid using the system cache. >>> However, it is still a part of Ignite and it works and can be used in >>> plugins. >>> Honestly, the mentioned alternative, I mean the distributed metastorage, >> is >>> the INTERNAL thing as well. >>> It means that plugin developer depends on INTERNAL entities. (it does not >>> matter system-cache/metastorage) >>> IMHO, such questions should be CAREFULLY discussed with no rush. >>> I do not propose to rush with the ignite-sys-cache removal. I'd like to >>> collect all the technical stuff that we depend on, fix all of them and >>> after everything will be ready do a final removal. >>> Good! We are on the same page! >>> 1. Add deprecation for the 2.12 release. I hope it is still possible. >>> If I am not mistaken, the code freeze was October 29. I think it is too >>> late to introduce this change. We can add deprecation for the 2.13 >> release. >>> Apache Ignite core still have some dependencies on ignite-sys-cache that >>> should fix for 2.13 >>> Ok, I agree. We need to try to find out all edge cases and add new >>> abilities to the metastorage in order to cover all known >>> issues/requirements. >>> Remove the system cache in 2.14. >>> It depends on our progress with improving the distributed metastorage. In >>> general, I hope it will be possible. >>> >>> Thanks, >>> S. >>> >>> чт, 2 дек. 2021 г. в 13:36, Maxim Muzafarov : >>> Pavel, I don't understand why you are using *backwards compatibility* for completely internal things. Why you are thinking in terms of users usage when are talking about ignite-sys-cache but not thinking when refactoring, for instance, all the checkpoint classes? Take a look at the [1] issue. All the internals have been reworked and nobody even mentioned that it may affect some of the plugin developers. This is really strange, so I can't agree with you. I do not propose to rush with the ignite-sys-cache removal. I'd like to collect all the technical stuff that we depend on, fix all of them and after everything will be ready do a final removal. Slava already mentioned some crucial cases, so I hope you and Val also help with the rest of them. Let's be more precise in terms *what we need to fix*. I've made some investigations related to the ignite-sys-cache and here is my proposal: 1. Add deprecation for the 2.12 release. I hope it is still possible. 2. Apache Ignite core still have some dependencies on ignite-sys-cache that should fix for 2.13: 2.1. CLUSTER_NAME property: when it's not set the `deploymentId` of system cache is used. Let's move it to metastorage. 2.2. When the Security is enabled each compute task name (hash to be more precise) is written to the ignite-sys-cache [2]. From my point of view, we can remove it in 2.13. Can anyone check? 2.3. Slava mentioned some issues
Re: [DISCUSSION] Remove outdated ignite-sys-cache
Hello Nikolay, If I am not mistaken, the method you mentioned, allows to create a "user" cache that is available through public api for the user. This does not cover the case when the plugin developer wants to hide such cache and protect it form the end user (at least, via public api). Thanks, S. пт, 3 дек. 2021 г., 14:15 Nikolay Izhikov : > Vyacheslav, Val, can you please clarify - What is the issue if third-party > plugins will create «ignite-sys-cache» from the code? > > Like just replacing `Ignite#cache` with the `Ignite#getOrCreateCache`. > > > 2 дек. 2021 г., в 16:13, Вячеслав Коптилин > написал(а): > > > > Hello Maxim, > > > >> I don't understand why you are using *backwards compatibility* for > > completely internal things. > >> Why you are thinking in terms of users usage when are talking about > > ignite-sys-cache but not thinking when refactoring > > First of all, we are talking about all plugin developers. It does not > > matter it is open-source or proprietary plugins. > > Honestly, I don't have a list of all possible plugins that have already > > been developed and available under the ASF license, for instance. > > Do you have such a list? Can you be sure that this change will not affect > > anyone? > > > >> I don't understand why you are using *backwards compatibility* for > > completely internal things. > >> Why you are thinking in terms of users usage when are talking about > > ignite-sys-cache but not thinking when refactoring > > The system cache was the crucial thing in the architecture of Apache > Ignite > > 1.x and 2.x (at least 2.1 - 2.11?) > > > >> All the internals have been reworked and nobody even mentioned that it > > may affect some of the plugin developers. > > Yep, perhaps, some internal parts of Apache Ignite were reworked in order > > to avoid using the system cache. > > However, it is still a part of Ignite and it works and can be used in > > plugins. > > Honestly, the mentioned alternative, I mean the distributed metastorage, > is > > the INTERNAL thing as well. > > It means that plugin developer depends on INTERNAL entities. (it does not > > matter system-cache/metastorage) > > IMHO, such questions should be CAREFULLY discussed with no rush. > > > >> I do not propose to rush with the ignite-sys-cache removal. I'd like to > > collect all the technical stuff that we depend on, fix all of them and > > after everything will be ready do a final removal. > > Good! We are on the same page! > > > >> 1. Add deprecation for the 2.12 release. I hope it is still possible. > > If I am not mistaken, the code freeze was October 29. I think it is too > > late to introduce this change. We can add deprecation for the 2.13 > release. > > > >> Apache Ignite core still have some dependencies on ignite-sys-cache that > > should fix for 2.13 > > Ok, I agree. We need to try to find out all edge cases and add new > > abilities to the metastorage in order to cover all known > > issues/requirements. > > > >> Remove the system cache in 2.14. > > It depends on our progress with improving the distributed metastorage. In > > general, I hope it will be possible. > > > > Thanks, > > S. > > > > чт, 2 дек. 2021 г. в 13:36, Maxim Muzafarov : > > > >> Pavel, > >> > >> > >> I don't understand why you are using *backwards compatibility* for > >> completely internal things. Why you are thinking in terms of users > >> usage when are talking about ignite-sys-cache but not thinking when > >> refactoring, for instance, all the checkpoint classes? Take a look at > >> the [1] issue. All the internals have been reworked and nobody even > >> mentioned that it may affect some of the plugin developers. This is > >> really strange, so I can't agree with you. > >> > >> > >> I do not propose to rush with the ignite-sys-cache removal. I'd like > >> to collect all the technical stuff that we depend on, fix all of them > >> and after everything will be ready do a final removal. Slava already > >> mentioned some crucial cases, so I hope you and Val also help with the > >> rest of them. Let's be more precise in terms *what we need to fix*. > >> > >> > >> I've made some investigations related to the ignite-sys-cache and here > >> is my proposal: > >> > >> 1. Add deprecation for the 2.12 release. I hope it is still possible. > >> > >> 2. Apache Ignite core still have some dependencies on ignite-sys-cache > >> that should fix for 2.13: > >> > >> 2.1. CLUSTER_NAME property: when it's not set the `deploymentId` of > >> system cache is used. Let's move it to metastorage. > >> 2.2. When the Security is enabled each compute task name (hash to be > >> more precise) is written to the ignite-sys-cache [2]. From my point of > >> view, we can remove it in 2.13. Can anyone check? > >> 2.3. Slava mentioned some issues with the distributed metastorage that > >> might be fixed. I think this is doable. > >> > >> 3. Remove the system cache in 2.14. > >> > >> > >> > >> [1] https://issues.apache.org/jira/browse/IGNITE-13143 > >> [2] > >> >
Re: [DISCUSSION] Remove outdated ignite-sys-cache
> What is the issue if third-party plugins will create «ignite-sys-cache» from the code? Great idea! On Fri, Dec 3, 2021 at 2:15 PM Nikolay Izhikov wrote: > Vyacheslav, Val, can you please clarify - What is the issue if third-party > plugins will create «ignite-sys-cache» from the code? > > Like just replacing `Ignite#cache` with the `Ignite#getOrCreateCache`. > > > 2 дек. 2021 г., в 16:13, Вячеслав Коптилин > написал(а): > > > > Hello Maxim, > > > >> I don't understand why you are using *backwards compatibility* for > > completely internal things. > >> Why you are thinking in terms of users usage when are talking about > > ignite-sys-cache but not thinking when refactoring > > First of all, we are talking about all plugin developers. It does not > > matter it is open-source or proprietary plugins. > > Honestly, I don't have a list of all possible plugins that have already > > been developed and available under the ASF license, for instance. > > Do you have such a list? Can you be sure that this change will not affect > > anyone? > > > >> I don't understand why you are using *backwards compatibility* for > > completely internal things. > >> Why you are thinking in terms of users usage when are talking about > > ignite-sys-cache but not thinking when refactoring > > The system cache was the crucial thing in the architecture of Apache > Ignite > > 1.x and 2.x (at least 2.1 - 2.11?) > > > >> All the internals have been reworked and nobody even mentioned that it > > may affect some of the plugin developers. > > Yep, perhaps, some internal parts of Apache Ignite were reworked in order > > to avoid using the system cache. > > However, it is still a part of Ignite and it works and can be used in > > plugins. > > Honestly, the mentioned alternative, I mean the distributed metastorage, > is > > the INTERNAL thing as well. > > It means that plugin developer depends on INTERNAL entities. (it does not > > matter system-cache/metastorage) > > IMHO, such questions should be CAREFULLY discussed with no rush. > > > >> I do not propose to rush with the ignite-sys-cache removal. I'd like to > > collect all the technical stuff that we depend on, fix all of them and > > after everything will be ready do a final removal. > > Good! We are on the same page! > > > >> 1. Add deprecation for the 2.12 release. I hope it is still possible. > > If I am not mistaken, the code freeze was October 29. I think it is too > > late to introduce this change. We can add deprecation for the 2.13 > release. > > > >> Apache Ignite core still have some dependencies on ignite-sys-cache that > > should fix for 2.13 > > Ok, I agree. We need to try to find out all edge cases and add new > > abilities to the metastorage in order to cover all known > > issues/requirements. > > > >> Remove the system cache in 2.14. > > It depends on our progress with improving the distributed metastorage. In > > general, I hope it will be possible. > > > > Thanks, > > S. > > > > чт, 2 дек. 2021 г. в 13:36, Maxim Muzafarov : > > > >> Pavel, > >> > >> > >> I don't understand why you are using *backwards compatibility* for > >> completely internal things. Why you are thinking in terms of users > >> usage when are talking about ignite-sys-cache but not thinking when > >> refactoring, for instance, all the checkpoint classes? Take a look at > >> the [1] issue. All the internals have been reworked and nobody even > >> mentioned that it may affect some of the plugin developers. This is > >> really strange, so I can't agree with you. > >> > >> > >> I do not propose to rush with the ignite-sys-cache removal. I'd like > >> to collect all the technical stuff that we depend on, fix all of them > >> and after everything will be ready do a final removal. Slava already > >> mentioned some crucial cases, so I hope you and Val also help with the > >> rest of them. Let's be more precise in terms *what we need to fix*. > >> > >> > >> I've made some investigations related to the ignite-sys-cache and here > >> is my proposal: > >> > >> 1. Add deprecation for the 2.12 release. I hope it is still possible. > >> > >> 2. Apache Ignite core still have some dependencies on ignite-sys-cache > >> that should fix for 2.13: > >> > >> 2.1. CLUSTER_NAME property: when it's not set the `deploymentId` of > >> system cache is used. Let's move it to metastorage. > >> 2.2. When the Security is enabled each compute task name (hash to be > >> more precise) is written to the ignite-sys-cache [2]. From my point of > >> view, we can remove it in 2.13. Can anyone check? > >> 2.3. Slava mentioned some issues with the distributed metastorage that > >> might be fixed. I think this is doable. > >> > >> 3. Remove the system cache in 2.14. > >> > >> > >> > >> [1] https://issues.apache.org/jira/browse/IGNITE-13143 > >> [2] > >> > https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/task/GridTaskProcessor.java#L201 > >> > >> On Thu, 2 Dec 2021 at 12:56, Pavel
Re: [DISCUSSION] Remove outdated ignite-sys-cache
Vyacheslav, Val, can you please clarify - What is the issue if third-party plugins will create «ignite-sys-cache» from the code? Like just replacing `Ignite#cache` with the `Ignite#getOrCreateCache`. > 2 дек. 2021 г., в 16:13, Вячеслав Коптилин > написал(а): > > Hello Maxim, > >> I don't understand why you are using *backwards compatibility* for > completely internal things. >> Why you are thinking in terms of users usage when are talking about > ignite-sys-cache but not thinking when refactoring > First of all, we are talking about all plugin developers. It does not > matter it is open-source or proprietary plugins. > Honestly, I don't have a list of all possible plugins that have already > been developed and available under the ASF license, for instance. > Do you have such a list? Can you be sure that this change will not affect > anyone? > >> I don't understand why you are using *backwards compatibility* for > completely internal things. >> Why you are thinking in terms of users usage when are talking about > ignite-sys-cache but not thinking when refactoring > The system cache was the crucial thing in the architecture of Apache Ignite > 1.x and 2.x (at least 2.1 - 2.11?) > >> All the internals have been reworked and nobody even mentioned that it > may affect some of the plugin developers. > Yep, perhaps, some internal parts of Apache Ignite were reworked in order > to avoid using the system cache. > However, it is still a part of Ignite and it works and can be used in > plugins. > Honestly, the mentioned alternative, I mean the distributed metastorage, is > the INTERNAL thing as well. > It means that plugin developer depends on INTERNAL entities. (it does not > matter system-cache/metastorage) > IMHO, such questions should be CAREFULLY discussed with no rush. > >> I do not propose to rush with the ignite-sys-cache removal. I'd like to > collect all the technical stuff that we depend on, fix all of them and > after everything will be ready do a final removal. > Good! We are on the same page! > >> 1. Add deprecation for the 2.12 release. I hope it is still possible. > If I am not mistaken, the code freeze was October 29. I think it is too > late to introduce this change. We can add deprecation for the 2.13 release. > >> Apache Ignite core still have some dependencies on ignite-sys-cache that > should fix for 2.13 > Ok, I agree. We need to try to find out all edge cases and add new > abilities to the metastorage in order to cover all known > issues/requirements. > >> Remove the system cache in 2.14. > It depends on our progress with improving the distributed metastorage. In > general, I hope it will be possible. > > Thanks, > S. > > чт, 2 дек. 2021 г. в 13:36, Maxim Muzafarov : > >> Pavel, >> >> >> I don't understand why you are using *backwards compatibility* for >> completely internal things. Why you are thinking in terms of users >> usage when are talking about ignite-sys-cache but not thinking when >> refactoring, for instance, all the checkpoint classes? Take a look at >> the [1] issue. All the internals have been reworked and nobody even >> mentioned that it may affect some of the plugin developers. This is >> really strange, so I can't agree with you. >> >> >> I do not propose to rush with the ignite-sys-cache removal. I'd like >> to collect all the technical stuff that we depend on, fix all of them >> and after everything will be ready do a final removal. Slava already >> mentioned some crucial cases, so I hope you and Val also help with the >> rest of them. Let's be more precise in terms *what we need to fix*. >> >> >> I've made some investigations related to the ignite-sys-cache and here >> is my proposal: >> >> 1. Add deprecation for the 2.12 release. I hope it is still possible. >> >> 2. Apache Ignite core still have some dependencies on ignite-sys-cache >> that should fix for 2.13: >> >> 2.1. CLUSTER_NAME property: when it's not set the `deploymentId` of >> system cache is used. Let's move it to metastorage. >> 2.2. When the Security is enabled each compute task name (hash to be >> more precise) is written to the ignite-sys-cache [2]. From my point of >> view, we can remove it in 2.13. Can anyone check? >> 2.3. Slava mentioned some issues with the distributed metastorage that >> might be fixed. I think this is doable. >> >> 3. Remove the system cache in 2.14. >> >> >> >> [1] https://issues.apache.org/jira/browse/IGNITE-13143 >> [2] >> https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/task/GridTaskProcessor.java#L201 >> >> On Thu, 2 Dec 2021 at 12:56, Pavel Tupitsyn wrote: >>> >>> Maxim, >>> I don't think that we should wait for 3-rd party plugins to be updated this is an awful practice when the open-source product releases depend on some of the proprietary plugins >>> >>> This makes no sense, sorry. >>> >>> It is not that we depend on 3rd party plugins. >>> It is that *our users
Re: [DISCUSSION] Remove outdated ignite-sys-cache
Hello Maxim, > I don't understand why you are using *backwards compatibility* for completely internal things. > Why you are thinking in terms of users usage when are talking about ignite-sys-cache but not thinking when refactoring First of all, we are talking about all plugin developers. It does not matter it is open-source or proprietary plugins. Honestly, I don't have a list of all possible plugins that have already been developed and available under the ASF license, for instance. Do you have such a list? Can you be sure that this change will not affect anyone? > I don't understand why you are using *backwards compatibility* for completely internal things. > Why you are thinking in terms of users usage when are talking about ignite-sys-cache but not thinking when refactoring The system cache was the crucial thing in the architecture of Apache Ignite 1.x and 2.x (at least 2.1 - 2.11?) > All the internals have been reworked and nobody even mentioned that it may affect some of the plugin developers. Yep, perhaps, some internal parts of Apache Ignite were reworked in order to avoid using the system cache. However, it is still a part of Ignite and it works and can be used in plugins. Honestly, the mentioned alternative, I mean the distributed metastorage, is the INTERNAL thing as well. It means that plugin developer depends on INTERNAL entities. (it does not matter system-cache/metastorage) IMHO, such questions should be CAREFULLY discussed with no rush. > I do not propose to rush with the ignite-sys-cache removal. I'd like to collect all the technical stuff that we depend on, fix all of them and after everything will be ready do a final removal. Good! We are on the same page! > 1. Add deprecation for the 2.12 release. I hope it is still possible. If I am not mistaken, the code freeze was October 29. I think it is too late to introduce this change. We can add deprecation for the 2.13 release. > Apache Ignite core still have some dependencies on ignite-sys-cache that should fix for 2.13 Ok, I agree. We need to try to find out all edge cases and add new abilities to the metastorage in order to cover all known issues/requirements. > Remove the system cache in 2.14. It depends on our progress with improving the distributed metastorage. In general, I hope it will be possible. Thanks, S. чт, 2 дек. 2021 г. в 13:36, Maxim Muzafarov : > Pavel, > > > I don't understand why you are using *backwards compatibility* for > completely internal things. Why you are thinking in terms of users > usage when are talking about ignite-sys-cache but not thinking when > refactoring, for instance, all the checkpoint classes? Take a look at > the [1] issue. All the internals have been reworked and nobody even > mentioned that it may affect some of the plugin developers. This is > really strange, so I can't agree with you. > > > I do not propose to rush with the ignite-sys-cache removal. I'd like > to collect all the technical stuff that we depend on, fix all of them > and after everything will be ready do a final removal. Slava already > mentioned some crucial cases, so I hope you and Val also help with the > rest of them. Let's be more precise in terms *what we need to fix*. > > > I've made some investigations related to the ignite-sys-cache and here > is my proposal: > > 1. Add deprecation for the 2.12 release. I hope it is still possible. > > 2. Apache Ignite core still have some dependencies on ignite-sys-cache > that should fix for 2.13: > > 2.1. CLUSTER_NAME property: when it's not set the `deploymentId` of > system cache is used. Let's move it to metastorage. > 2.2. When the Security is enabled each compute task name (hash to be > more precise) is written to the ignite-sys-cache [2]. From my point of > view, we can remove it in 2.13. Can anyone check? > 2.3. Slava mentioned some issues with the distributed metastorage that > might be fixed. I think this is doable. > > 3. Remove the system cache in 2.14. > > > > [1] https://issues.apache.org/jira/browse/IGNITE-13143 > [2] > https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/task/GridTaskProcessor.java#L201 > > On Thu, 2 Dec 2021 at 12:56, Pavel Tupitsyn wrote: > > > > Maxim, > > > > > I don't think that we should wait for 3-rd party plugins to be updated > > > this is an awful practice when the open-source product releases depend > > > on some of the proprietary plugins > > > > This makes no sense, sorry. > > > > It is not that we depend on 3rd party plugins. > > It is that *our users depend on us to preserve backwards compatibility*. > > No one likes when their app breaks suddenly because of a library update. > > > > We know about one use case for sys-cache in GridGain, but there may be > > more, no one knows. > > Every breaking change should be carried out carefully and gradually. > > > > > > On Thu, Dec 2, 2021 at 12:34 PM Maxim Muzafarov > wrote: > > > > > Slava, > > > > > > Thank you for the comments. >
Re: [DISCUSSION] Remove outdated ignite-sys-cache
Pavel, I don't understand why you are using *backwards compatibility* for completely internal things. Why you are thinking in terms of users usage when are talking about ignite-sys-cache but not thinking when refactoring, for instance, all the checkpoint classes? Take a look at the [1] issue. All the internals have been reworked and nobody even mentioned that it may affect some of the plugin developers. This is really strange, so I can't agree with you. I do not propose to rush with the ignite-sys-cache removal. I'd like to collect all the technical stuff that we depend on, fix all of them and after everything will be ready do a final removal. Slava already mentioned some crucial cases, so I hope you and Val also help with the rest of them. Let's be more precise in terms *what we need to fix*. I've made some investigations related to the ignite-sys-cache and here is my proposal: 1. Add deprecation for the 2.12 release. I hope it is still possible. 2. Apache Ignite core still have some dependencies on ignite-sys-cache that should fix for 2.13: 2.1. CLUSTER_NAME property: when it's not set the `deploymentId` of system cache is used. Let's move it to metastorage. 2.2. When the Security is enabled each compute task name (hash to be more precise) is written to the ignite-sys-cache [2]. From my point of view, we can remove it in 2.13. Can anyone check? 2.3. Slava mentioned some issues with the distributed metastorage that might be fixed. I think this is doable. 3. Remove the system cache in 2.14. [1] https://issues.apache.org/jira/browse/IGNITE-13143 [2] https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/task/GridTaskProcessor.java#L201 On Thu, 2 Dec 2021 at 12:56, Pavel Tupitsyn wrote: > > Maxim, > > > I don't think that we should wait for 3-rd party plugins to be updated > > this is an awful practice when the open-source product releases depend > > on some of the proprietary plugins > > This makes no sense, sorry. > > It is not that we depend on 3rd party plugins. > It is that *our users depend on us to preserve backwards compatibility*. > No one likes when their app breaks suddenly because of a library update. > > We know about one use case for sys-cache in GridGain, but there may be > more, no one knows. > Every breaking change should be carried out carefully and gradually. > > > On Thu, Dec 2, 2021 at 12:34 PM Maxim Muzafarov wrote: > > > Slava, > > > > Thank you for the comments. > > > > > Maxim, the community must provide a reasonable time interval in order to > > notify all contributors and wait for updating all 3-rd party plugins. > > > > This is not actually true. We must notify about changes as earlier as > > possible and not only users but all the developers also. However, I > > don't think that we should wait for 3-rd party plugins to be updated > > this is an awful practice when the open-source product releases depend > > on some of the proprietary plugins. There is no dependency between > > Apache Ignite releases and proprietary plugin releases. If someone > > desires to upgrade to a new version of Apache Ignite it can update its > > plugins any time he likes. > > > > > We just need to provide a window that is enough to cover all related > > issues. > > > > Can you share all the issues you know, please? > > > > > distributed meta storage does not provide the ability to atomically > > change several properties at the same time (there are no transactions on > > this API). > > > > Can you share an example of what kind of properties we intend to > > change at the same? Will it be enough to change them through a single > > thread (e.g. the discovery thread or the exchange thread)? I agree > > here that distributed metastorage should provide such an ability, > > however, I can't find any real examples for Apache Ignite internal > > needs. Please, share the details and let's fix it. > > > > On Wed, 1 Dec 2021 at 23:19, Valentin Kulichenko > > wrote: > > > > > > Agree with Slava. Two months is not enough time, especially considering > > > that the system cache is not functionally equivalent to the metastorage. > > I > > > suggest we do the following: > > > > > > 1. Write down the differences between the system cache and the > > metastorage, > > > and provide a transition guide for plugin developers. > > > 2. Deprecate the system cache in 2.13. > > > 3. Remove the system cache in one of the further releases. I don't think > > > it's reasonable to do this earlier than mid next year (even that is > > > potentially too early). > > > > > > Removing the system cache is the right move, but let's be more > > considerate > > > to the users. There is absolutely no need to rush. > > > > > > -Val > > > > > > On Wed, Dec 1, 2021 at 7:28 AM Вячеслав Коптилин < > > slava.kopti...@gmail.com> > > > wrote: > > > > > > > Hi Maxim, > > > > > > > > > On the other hand, I don't see any valuable reason why the change > > can't > > > > be done and we
Re: [DISCUSSION] Remove outdated ignite-sys-cache
Maxim, > I don't think that we should wait for 3-rd party plugins to be updated > this is an awful practice when the open-source product releases depend > on some of the proprietary plugins This makes no sense, sorry. It is not that we depend on 3rd party plugins. It is that *our users depend on us to preserve backwards compatibility*. No one likes when their app breaks suddenly because of a library update. We know about one use case for sys-cache in GridGain, but there may be more, no one knows. Every breaking change should be carried out carefully and gradually. On Thu, Dec 2, 2021 at 12:34 PM Maxim Muzafarov wrote: > Slava, > > Thank you for the comments. > > > Maxim, the community must provide a reasonable time interval in order to > notify all contributors and wait for updating all 3-rd party plugins. > > This is not actually true. We must notify about changes as earlier as > possible and not only users but all the developers also. However, I > don't think that we should wait for 3-rd party plugins to be updated > this is an awful practice when the open-source product releases depend > on some of the proprietary plugins. There is no dependency between > Apache Ignite releases and proprietary plugin releases. If someone > desires to upgrade to a new version of Apache Ignite it can update its > plugins any time he likes. > > > We just need to provide a window that is enough to cover all related > issues. > > Can you share all the issues you know, please? > > > distributed meta storage does not provide the ability to atomically > change several properties at the same time (there are no transactions on > this API). > > Can you share an example of what kind of properties we intend to > change at the same? Will it be enough to change them through a single > thread (e.g. the discovery thread or the exchange thread)? I agree > here that distributed metastorage should provide such an ability, > however, I can't find any real examples for Apache Ignite internal > needs. Please, share the details and let's fix it. > > On Wed, 1 Dec 2021 at 23:19, Valentin Kulichenko > wrote: > > > > Agree with Slava. Two months is not enough time, especially considering > > that the system cache is not functionally equivalent to the metastorage. > I > > suggest we do the following: > > > > 1. Write down the differences between the system cache and the > metastorage, > > and provide a transition guide for plugin developers. > > 2. Deprecate the system cache in 2.13. > > 3. Remove the system cache in one of the further releases. I don't think > > it's reasonable to do this earlier than mid next year (even that is > > potentially too early). > > > > Removing the system cache is the right move, but let's be more > considerate > > to the users. There is absolutely no need to rush. > > > > -Val > > > > On Wed, Dec 1, 2021 at 7:28 AM Вячеслав Коптилин < > slava.kopti...@gmail.com> > > wrote: > > > > > Hi Maxim, > > > > > > > On the other hand, I don't see any valuable reason why the change > can't > > > be done and we should wait for the future that never comes. > > > Maxim, the community must provide a reasonable time interval in order > to > > > notify all contributors and wait for updating all 3-rd party plugins. > > > Honestly, it does not seem to me that two, three months (the current > > > timeline - end of March is the release date, so the end of Feb is code > > > freeze) are quite enough. > > > > > > > I don't see any reasons why we should keep something in the CORE > module > > > that is being used at PLUGINS only. This is a lack of architecture > design > > > that must be fixed for sure; > > > I agree with you. We just need to provide a window that is enough to > cover > > > all related issues. > > > For example, distributed meta storage does not provide the ability to > > > atomically change several properties at the same time (there are no > > > transactions on this API). > > > > > > Perhaps, it would be nice to plan to remove the system cache on 2.14 or > > > even 2.15. What do you think? > > > > > > Thanks, > > > S. > > > > > > > > > вт, 30 нояб. 2021 г. в 13:58, Maxim Muzafarov : > > > > > > > Hello Val, > > > > > > > > Thank you for sharing the details. On the one hand, I agree with you > > > > that there is no need to haste with this change and it must be > > > > prepared carefully. On the other hand, I don't see any valuable > reason > > > > why the change can't be done and we should wait for the future that > > > > never comes. > > > > > > > > Please, consider the following my considerations: > > > > > > > > - The release 2.13 is not started yet. It will take months to be > > > > released (e.g. the end of March as a possible release date). The time > > > > is more than enough; > > > > - The 2.13 release has breaking changes, so it is a good opportunity > > > > to fix some gaps; > > > > - I don't see any reasons why we should keep something in the CORE > > > > module that is being used at PLUGINS only. This
Re: [DISCUSSION] Remove outdated ignite-sys-cache
Slava, Thank you for the comments. > Maxim, the community must provide a reasonable time interval in order to > notify all contributors and wait for updating all 3-rd party plugins. This is not actually true. We must notify about changes as earlier as possible and not only users but all the developers also. However, I don't think that we should wait for 3-rd party plugins to be updated this is an awful practice when the open-source product releases depend on some of the proprietary plugins. There is no dependency between Apache Ignite releases and proprietary plugin releases. If someone desires to upgrade to a new version of Apache Ignite it can update its plugins any time he likes. > We just need to provide a window that is enough to cover all related issues. Can you share all the issues you know, please? > distributed meta storage does not provide the ability to atomically change > several properties at the same time (there are no transactions on this API). Can you share an example of what kind of properties we intend to change at the same? Will it be enough to change them through a single thread (e.g. the discovery thread or the exchange thread)? I agree here that distributed metastorage should provide such an ability, however, I can't find any real examples for Apache Ignite internal needs. Please, share the details and let's fix it. On Wed, 1 Dec 2021 at 23:19, Valentin Kulichenko wrote: > > Agree with Slava. Two months is not enough time, especially considering > that the system cache is not functionally equivalent to the metastorage. I > suggest we do the following: > > 1. Write down the differences between the system cache and the metastorage, > and provide a transition guide for plugin developers. > 2. Deprecate the system cache in 2.13. > 3. Remove the system cache in one of the further releases. I don't think > it's reasonable to do this earlier than mid next year (even that is > potentially too early). > > Removing the system cache is the right move, but let's be more considerate > to the users. There is absolutely no need to rush. > > -Val > > On Wed, Dec 1, 2021 at 7:28 AM Вячеслав Коптилин > wrote: > > > Hi Maxim, > > > > > On the other hand, I don't see any valuable reason why the change can't > > be done and we should wait for the future that never comes. > > Maxim, the community must provide a reasonable time interval in order to > > notify all contributors and wait for updating all 3-rd party plugins. > > Honestly, it does not seem to me that two, three months (the current > > timeline - end of March is the release date, so the end of Feb is code > > freeze) are quite enough. > > > > > I don't see any reasons why we should keep something in the CORE module > > that is being used at PLUGINS only. This is a lack of architecture design > > that must be fixed for sure; > > I agree with you. We just need to provide a window that is enough to cover > > all related issues. > > For example, distributed meta storage does not provide the ability to > > atomically change several properties at the same time (there are no > > transactions on this API). > > > > Perhaps, it would be nice to plan to remove the system cache on 2.14 or > > even 2.15. What do you think? > > > > Thanks, > > S. > > > > > > вт, 30 нояб. 2021 г. в 13:58, Maxim Muzafarov : > > > > > Hello Val, > > > > > > Thank you for sharing the details. On the one hand, I agree with you > > > that there is no need to haste with this change and it must be > > > prepared carefully. On the other hand, I don't see any valuable reason > > > why the change can't be done and we should wait for the future that > > > never comes. > > > > > > Please, consider the following my considerations: > > > > > > - The release 2.13 is not started yet. It will take months to be > > > released (e.g. the end of March as a possible release date). The time > > > is more than enough; > > > - The 2.13 release has breaking changes, so it is a good opportunity > > > to fix some gaps; > > > - I don't see any reasons why we should keep something in the CORE > > > module that is being used at PLUGINS only. This is a lack of > > > architecture design that must be fixed for sure; > > > - The cache affects cluster stability and require additional human > > > resources for the code maintenance; > > > - As a straightforward solution plugins can create their own internal > > > caches and use them ever they like (or use the metastorage as I > > > mentioned earlier). Moving system cache to plugin doesn't look so > > > complicated and harmful; > > > > > > On Mon, 29 Nov 2021 at 23:07, Valentin Kulichenko > > > wrote: > > > > > > > > Maxim, > > > > > > > > You're right that the system cache is still likely to be used by > > plugins. > > > > We at GridGain use it for security features, for example. As far as I > > > know, > > > > that's not the only case. > > > > > > > > I also agree that the metastorage should be the preferred way for this > > > kind > > > > of
Re: [DISCUSSION] Remove outdated ignite-sys-cache
Agree with Slava. Two months is not enough time, especially considering that the system cache is not functionally equivalent to the metastorage. I suggest we do the following: 1. Write down the differences between the system cache and the metastorage, and provide a transition guide for plugin developers. 2. Deprecate the system cache in 2.13. 3. Remove the system cache in one of the further releases. I don't think it's reasonable to do this earlier than mid next year (even that is potentially too early). Removing the system cache is the right move, but let's be more considerate to the users. There is absolutely no need to rush. -Val On Wed, Dec 1, 2021 at 7:28 AM Вячеслав Коптилин wrote: > Hi Maxim, > > > On the other hand, I don't see any valuable reason why the change can't > be done and we should wait for the future that never comes. > Maxim, the community must provide a reasonable time interval in order to > notify all contributors and wait for updating all 3-rd party plugins. > Honestly, it does not seem to me that two, three months (the current > timeline - end of March is the release date, so the end of Feb is code > freeze) are quite enough. > > > I don't see any reasons why we should keep something in the CORE module > that is being used at PLUGINS only. This is a lack of architecture design > that must be fixed for sure; > I agree with you. We just need to provide a window that is enough to cover > all related issues. > For example, distributed meta storage does not provide the ability to > atomically change several properties at the same time (there are no > transactions on this API). > > Perhaps, it would be nice to plan to remove the system cache on 2.14 or > even 2.15. What do you think? > > Thanks, > S. > > > вт, 30 нояб. 2021 г. в 13:58, Maxim Muzafarov : > > > Hello Val, > > > > Thank you for sharing the details. On the one hand, I agree with you > > that there is no need to haste with this change and it must be > > prepared carefully. On the other hand, I don't see any valuable reason > > why the change can't be done and we should wait for the future that > > never comes. > > > > Please, consider the following my considerations: > > > > - The release 2.13 is not started yet. It will take months to be > > released (e.g. the end of March as a possible release date). The time > > is more than enough; > > - The 2.13 release has breaking changes, so it is a good opportunity > > to fix some gaps; > > - I don't see any reasons why we should keep something in the CORE > > module that is being used at PLUGINS only. This is a lack of > > architecture design that must be fixed for sure; > > - The cache affects cluster stability and require additional human > > resources for the code maintenance; > > - As a straightforward solution plugins can create their own internal > > caches and use them ever they like (or use the metastorage as I > > mentioned earlier). Moving system cache to plugin doesn't look so > > complicated and harmful; > > > > On Mon, 29 Nov 2021 at 23:07, Valentin Kulichenko > > wrote: > > > > > > Maxim, > > > > > > You're right that the system cache is still likely to be used by > plugins. > > > We at GridGain use it for security features, for example. As far as I > > know, > > > that's not the only case. > > > > > > I also agree that the metastorage should be the preferred way for this > > kind > > > of purposes, but is there any harm in keeping the system cache for now? > > > Unlike the legacy Service Grid, this is not a public feature that can > be > > > used directly by a regular user, so I'm not sure why the rush. > > > > > > How about we instead deprecate the system cache, clearly document how > to > > > use the metastorage as an alternative, and then complete the removal > > > sometime in the future? > > > > > > -Val > > > > > > On Sun, Nov 28, 2021 at 6:49 AM Maxim Muzafarov > > wrote: > > > > > > > Igniters, > > > > > > > > > > > > Since the legacy service grid implementation was removed [2] I'd like > > > > to remove the ignite-sys-cache also. It is still possible that some > of > > > > Ignite plugins (e.g. security plugin) are still using this cache, > > > > however, AFAIK this is not the reason to keep the outdated system > > > > cache and these plugins must be migrated to the metastorage instead. > > > > > > > > I've filed the issue [1] for removal. Any objections? > > > > > > > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-16008 > > > > [2] https://issues.apache.org/jira/browse/IGNITE-15758 > > > > > > >
Re: [DISCUSSION] Remove outdated ignite-sys-cache
Hi Maxim, > On the other hand, I don't see any valuable reason why the change can't be done and we should wait for the future that never comes. Maxim, the community must provide a reasonable time interval in order to notify all contributors and wait for updating all 3-rd party plugins. Honestly, it does not seem to me that two, three months (the current timeline - end of March is the release date, so the end of Feb is code freeze) are quite enough. > I don't see any reasons why we should keep something in the CORE module that is being used at PLUGINS only. This is a lack of architecture design that must be fixed for sure; I agree with you. We just need to provide a window that is enough to cover all related issues. For example, distributed meta storage does not provide the ability to atomically change several properties at the same time (there are no transactions on this API). Perhaps, it would be nice to plan to remove the system cache on 2.14 or even 2.15. What do you think? Thanks, S. вт, 30 нояб. 2021 г. в 13:58, Maxim Muzafarov : > Hello Val, > > Thank you for sharing the details. On the one hand, I agree with you > that there is no need to haste with this change and it must be > prepared carefully. On the other hand, I don't see any valuable reason > why the change can't be done and we should wait for the future that > never comes. > > Please, consider the following my considerations: > > - The release 2.13 is not started yet. It will take months to be > released (e.g. the end of March as a possible release date). The time > is more than enough; > - The 2.13 release has breaking changes, so it is a good opportunity > to fix some gaps; > - I don't see any reasons why we should keep something in the CORE > module that is being used at PLUGINS only. This is a lack of > architecture design that must be fixed for sure; > - The cache affects cluster stability and require additional human > resources for the code maintenance; > - As a straightforward solution plugins can create their own internal > caches and use them ever they like (or use the metastorage as I > mentioned earlier). Moving system cache to plugin doesn't look so > complicated and harmful; > > On Mon, 29 Nov 2021 at 23:07, Valentin Kulichenko > wrote: > > > > Maxim, > > > > You're right that the system cache is still likely to be used by plugins. > > We at GridGain use it for security features, for example. As far as I > know, > > that's not the only case. > > > > I also agree that the metastorage should be the preferred way for this > kind > > of purposes, but is there any harm in keeping the system cache for now? > > Unlike the legacy Service Grid, this is not a public feature that can be > > used directly by a regular user, so I'm not sure why the rush. > > > > How about we instead deprecate the system cache, clearly document how to > > use the metastorage as an alternative, and then complete the removal > > sometime in the future? > > > > -Val > > > > On Sun, Nov 28, 2021 at 6:49 AM Maxim Muzafarov > wrote: > > > > > Igniters, > > > > > > > > > Since the legacy service grid implementation was removed [2] I'd like > > > to remove the ignite-sys-cache also. It is still possible that some of > > > Ignite plugins (e.g. security plugin) are still using this cache, > > > however, AFAIK this is not the reason to keep the outdated system > > > cache and these plugins must be migrated to the metastorage instead. > > > > > > I've filed the issue [1] for removal. Any objections? > > > > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-16008 > > > [2] https://issues.apache.org/jira/browse/IGNITE-15758 > > > >
Re: [DISCUSSION] Remove outdated ignite-sys-cache
Hello Val, Thank you for sharing the details. On the one hand, I agree with you that there is no need to haste with this change and it must be prepared carefully. On the other hand, I don't see any valuable reason why the change can't be done and we should wait for the future that never comes. Please, consider the following my considerations: - The release 2.13 is not started yet. It will take months to be released (e.g. the end of March as a possible release date). The time is more than enough; - The 2.13 release has breaking changes, so it is a good opportunity to fix some gaps; - I don't see any reasons why we should keep something in the CORE module that is being used at PLUGINS only. This is a lack of architecture design that must be fixed for sure; - The cache affects cluster stability and require additional human resources for the code maintenance; - As a straightforward solution plugins can create their own internal caches and use them ever they like (or use the metastorage as I mentioned earlier). Moving system cache to plugin doesn't look so complicated and harmful; On Mon, 29 Nov 2021 at 23:07, Valentin Kulichenko wrote: > > Maxim, > > You're right that the system cache is still likely to be used by plugins. > We at GridGain use it for security features, for example. As far as I know, > that's not the only case. > > I also agree that the metastorage should be the preferred way for this kind > of purposes, but is there any harm in keeping the system cache for now? > Unlike the legacy Service Grid, this is not a public feature that can be > used directly by a regular user, so I'm not sure why the rush. > > How about we instead deprecate the system cache, clearly document how to > use the metastorage as an alternative, and then complete the removal > sometime in the future? > > -Val > > On Sun, Nov 28, 2021 at 6:49 AM Maxim Muzafarov wrote: > > > Igniters, > > > > > > Since the legacy service grid implementation was removed [2] I'd like > > to remove the ignite-sys-cache also. It is still possible that some of > > Ignite plugins (e.g. security plugin) are still using this cache, > > however, AFAIK this is not the reason to keep the outdated system > > cache and these plugins must be migrated to the metastorage instead. > > > > I've filed the issue [1] for removal. Any objections? > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-16008 > > [2] https://issues.apache.org/jira/browse/IGNITE-15758 > >
Re: [DISCUSSION] Remove outdated ignite-sys-cache
Maxim, You're right that the system cache is still likely to be used by plugins. We at GridGain use it for security features, for example. As far as I know, that's not the only case. I also agree that the metastorage should be the preferred way for this kind of purposes, but is there any harm in keeping the system cache for now? Unlike the legacy Service Grid, this is not a public feature that can be used directly by a regular user, so I'm not sure why the rush. How about we instead deprecate the system cache, clearly document how to use the metastorage as an alternative, and then complete the removal sometime in the future? -Val On Sun, Nov 28, 2021 at 6:49 AM Maxim Muzafarov wrote: > Igniters, > > > Since the legacy service grid implementation was removed [2] I'd like > to remove the ignite-sys-cache also. It is still possible that some of > Ignite plugins (e.g. security plugin) are still using this cache, > however, AFAIK this is not the reason to keep the outdated system > cache and these plugins must be migrated to the metastorage instead. > > I've filed the issue [1] for removal. Any objections? > > > [1] https://issues.apache.org/jira/browse/IGNITE-16008 > [2] https://issues.apache.org/jira/browse/IGNITE-15758 >
