Re: Ignite 3.0 Ignition API, node startup, and thin client startup
Ivan, I've seen the link, but I still don't understand what exactly you propose to change in the current API, and what is your concern. Could you please clarify? How you think Ignite API should look like? -Val On Thu, Jul 8, 2021 at 2:18 PM Ivan Daschinsky wrote: > Val, I have already gave examples -- lettuce, a very performant and modern > redis java client > > I can duplicate links again > https://lettuce.io/core/release/api/io/lettuce/core/RedisClient.html > > https://lettuce.io/core/release/api/io/lettuce/core/api/StatefulRedisConnection.html > https://www.baeldung.com/java-redis-lettuce > > чт, 8 июл. 2021 г., 23:47 Valentin Kulichenko < > valentin.kuliche...@gmail.com > >: > > > Ivan, > > > > Can you please clarify what you mean by "separate creation of client and > > connection"? Can you give an example? > > > > -Val > > > > On Thu, Jul 8, 2021 at 12:53 PM Ivan Daschinsky > > wrote: > > > > > I'm sorry, but why we didn't consider to separate creation of Client > and > > > connection? Why not to make async variant of connection? See for > example > > > [1] > > > [1] --- https://lettuce.io/core/release/api/index.html > > > > > > > > > чт, 8 июл. 2021 г., 09:50 Pavel Tupitsyn : > > > > > > > Val, > > > > > > > > So the plan is: > > > > > > > > - Remove Ignition#start from the public API > > > > - Make Ignition a class, not an interface > > > > - Add static Ignition#startClient > > > > > > > > Sounds good? > > > > > > > > On Thu, Jul 8, 2021 at 6:13 AM Valentin Kulichenko < > > > > valentin.kuliche...@gmail.com> wrote: > > > > > > > > > Hi Ivan, > > > > > > > > > > Ignition IS the entry point to Ignite, so I'm not sure I got your > > point > > > > :) > > > > > Where is the contradiction? > > > > > > > > > > Either way, please feel free to give your suggestions for an > > > alternative > > > > > name if you have any. > > > > > > > > > > -Val > > > > > > > > > > On Wed, Jul 7, 2021 at 7:56 PM Ivan Pavlukhina < > vololo...@gmail.com> > > > > > wrote: > > > > > > > > > > > A side note. Actually “Ignition” naming always confused me. I > think > > > > about > > > > > > it as some fancy named API entry point for Ignite. Perhaps it is > a > > > good > > > > > > moment to revisit naming. > > > > > > > > > > > > > On 8 Jul 2021, at 07:09, Valentin Kulichenko < > > > > > > valentin.kuliche...@gmail.com> wrote: > > > > > > > > > > > > > > Hi Pavel, > > > > > > > > > > > > > > I don't think we will need the pure embedded mode, but we still > > > need > > > > to > > > > > > be > > > > > > > able to access the API from compute and services. That said, > > there > > > > are > > > > > > two > > > > > > > usages of the 'Ignite' API: > > > > > > > > > > > > > > 1. Remote, via the binary protocol. > > > > > > > 2. Local - needed for compute and services. (This is how it > > works > > > > > now.) > > > > > > > > > > > > > > I believe that the API should be the same, and there should be > a > > > > > unified > > > > > > > access point. Ignition seems to be a good candidate for this. > > > > > > > > > > > > > > Ignition#start should eventually be removed from the public > API. > > It > > > > is > > > > > > > currently there only because we don't have the thin client yet. > > > > > > > > > > > > > > -Val > > > > > > > > > > > > > >> On Wed, Jul 7, 2021 at 5:47 AM Pavel Tupitsyn < > > > ptupit...@apache.org > > > > > > > > > > > wrote: > > > > > > >> > > > > > > >> Igniters, > > > > > > >> > > > > > > >> I have a few questions regarding server node startup and thin > > > > clients. > > > > > > >> > > > > > > >> State of things: > > > > > > >> - Server nodes will be started with 'ignite run' from CLI [1] > > > > > > >> - ignite-api module represents our public API > > > > > > >> - ignite-api has Ignition interface to start server nodes > > > > > > >> > > > > > > >> Questions: > > > > > > >> - What's the idea behind Ignition interface in the public API? > > Are > > > > we > > > > > > going > > > > > > >> to have an "embedded mode" where servers can be started from > > > code? I > > > > > > >> thought this was not planned. > > > > > > >> - How are users supposed to retrieve an instance of the > Ignition > > > > > > interface? > > > > > > >> - Are there any plans to start thin clients from Ignition > > > interface, > > > > > or > > > > > > >> should we have a separate way of doing this? > > > > > > >> > > > > > > >> > > > > > > >> [1] > > > > > > >> > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=158873958 > > > > > > >> > > > > > > > > > > > > > > > > > > > > >
Re: Ignite 3.0 Ignition API, node startup, and thin client startup
Val, I have already gave examples -- lettuce, a very performant and modern redis java client I can duplicate links again https://lettuce.io/core/release/api/io/lettuce/core/RedisClient.html https://lettuce.io/core/release/api/io/lettuce/core/api/StatefulRedisConnection.html https://www.baeldung.com/java-redis-lettuce чт, 8 июл. 2021 г., 23:47 Valentin Kulichenko : > Ivan, > > Can you please clarify what you mean by "separate creation of client and > connection"? Can you give an example? > > -Val > > On Thu, Jul 8, 2021 at 12:53 PM Ivan Daschinsky > wrote: > > > I'm sorry, but why we didn't consider to separate creation of Client and > > connection? Why not to make async variant of connection? See for example > > [1] > > [1] --- https://lettuce.io/core/release/api/index.html > > > > > > чт, 8 июл. 2021 г., 09:50 Pavel Tupitsyn : > > > > > Val, > > > > > > So the plan is: > > > > > > - Remove Ignition#start from the public API > > > - Make Ignition a class, not an interface > > > - Add static Ignition#startClient > > > > > > Sounds good? > > > > > > On Thu, Jul 8, 2021 at 6:13 AM Valentin Kulichenko < > > > valentin.kuliche...@gmail.com> wrote: > > > > > > > Hi Ivan, > > > > > > > > Ignition IS the entry point to Ignite, so I'm not sure I got your > point > > > :) > > > > Where is the contradiction? > > > > > > > > Either way, please feel free to give your suggestions for an > > alternative > > > > name if you have any. > > > > > > > > -Val > > > > > > > > On Wed, Jul 7, 2021 at 7:56 PM Ivan Pavlukhina > > > > wrote: > > > > > > > > > A side note. Actually “Ignition” naming always confused me. I think > > > about > > > > > it as some fancy named API entry point for Ignite. Perhaps it is a > > good > > > > > moment to revisit naming. > > > > > > > > > > > On 8 Jul 2021, at 07:09, Valentin Kulichenko < > > > > > valentin.kuliche...@gmail.com> wrote: > > > > > > > > > > > > Hi Pavel, > > > > > > > > > > > > I don't think we will need the pure embedded mode, but we still > > need > > > to > > > > > be > > > > > > able to access the API from compute and services. That said, > there > > > are > > > > > two > > > > > > usages of the 'Ignite' API: > > > > > > > > > > > > 1. Remote, via the binary protocol. > > > > > > 2. Local - needed for compute and services. (This is how it > works > > > > now.) > > > > > > > > > > > > I believe that the API should be the same, and there should be a > > > > unified > > > > > > access point. Ignition seems to be a good candidate for this. > > > > > > > > > > > > Ignition#start should eventually be removed from the public API. > It > > > is > > > > > > currently there only because we don't have the thin client yet. > > > > > > > > > > > > -Val > > > > > > > > > > > >> On Wed, Jul 7, 2021 at 5:47 AM Pavel Tupitsyn < > > ptupit...@apache.org > > > > > > > > > wrote: > > > > > >> > > > > > >> Igniters, > > > > > >> > > > > > >> I have a few questions regarding server node startup and thin > > > clients. > > > > > >> > > > > > >> State of things: > > > > > >> - Server nodes will be started with 'ignite run' from CLI [1] > > > > > >> - ignite-api module represents our public API > > > > > >> - ignite-api has Ignition interface to start server nodes > > > > > >> > > > > > >> Questions: > > > > > >> - What's the idea behind Ignition interface in the public API? > Are > > > we > > > > > going > > > > > >> to have an "embedded mode" where servers can be started from > > code? I > > > > > >> thought this was not planned. > > > > > >> - How are users supposed to retrieve an instance of the Ignition > > > > > interface? > > > > > >> - Are there any plans to start thin clients from Ignition > > interface, > > > > or > > > > > >> should we have a separate way of doing this? > > > > > >> > > > > > >> > > > > > >> [1] > > > > > >> > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=158873958 > > > > > >> > > > > > > > > > > > > > > >
Re: Ignite 3.0 Ignition API, node startup, and thin client startup
Ivan, Can you please clarify what you mean by "separate creation of client and connection"? Can you give an example? -Val On Thu, Jul 8, 2021 at 12:53 PM Ivan Daschinsky wrote: > I'm sorry, but why we didn't consider to separate creation of Client and > connection? Why not to make async variant of connection? See for example > [1] > [1] --- https://lettuce.io/core/release/api/index.html > > > чт, 8 июл. 2021 г., 09:50 Pavel Tupitsyn : > > > Val, > > > > So the plan is: > > > > - Remove Ignition#start from the public API > > - Make Ignition a class, not an interface > > - Add static Ignition#startClient > > > > Sounds good? > > > > On Thu, Jul 8, 2021 at 6:13 AM Valentin Kulichenko < > > valentin.kuliche...@gmail.com> wrote: > > > > > Hi Ivan, > > > > > > Ignition IS the entry point to Ignite, so I'm not sure I got your point > > :) > > > Where is the contradiction? > > > > > > Either way, please feel free to give your suggestions for an > alternative > > > name if you have any. > > > > > > -Val > > > > > > On Wed, Jul 7, 2021 at 7:56 PM Ivan Pavlukhina > > > wrote: > > > > > > > A side note. Actually “Ignition” naming always confused me. I think > > about > > > > it as some fancy named API entry point for Ignite. Perhaps it is a > good > > > > moment to revisit naming. > > > > > > > > > On 8 Jul 2021, at 07:09, Valentin Kulichenko < > > > > valentin.kuliche...@gmail.com> wrote: > > > > > > > > > > Hi Pavel, > > > > > > > > > > I don't think we will need the pure embedded mode, but we still > need > > to > > > > be > > > > > able to access the API from compute and services. That said, there > > are > > > > two > > > > > usages of the 'Ignite' API: > > > > > > > > > > 1. Remote, via the binary protocol. > > > > > 2. Local - needed for compute and services. (This is how it works > > > now.) > > > > > > > > > > I believe that the API should be the same, and there should be a > > > unified > > > > > access point. Ignition seems to be a good candidate for this. > > > > > > > > > > Ignition#start should eventually be removed from the public API. It > > is > > > > > currently there only because we don't have the thin client yet. > > > > > > > > > > -Val > > > > > > > > > >> On Wed, Jul 7, 2021 at 5:47 AM Pavel Tupitsyn < > ptupit...@apache.org > > > > > > > wrote: > > > > >> > > > > >> Igniters, > > > > >> > > > > >> I have a few questions regarding server node startup and thin > > clients. > > > > >> > > > > >> State of things: > > > > >> - Server nodes will be started with 'ignite run' from CLI [1] > > > > >> - ignite-api module represents our public API > > > > >> - ignite-api has Ignition interface to start server nodes > > > > >> > > > > >> Questions: > > > > >> - What's the idea behind Ignition interface in the public API? Are > > we > > > > going > > > > >> to have an "embedded mode" where servers can be started from > code? I > > > > >> thought this was not planned. > > > > >> - How are users supposed to retrieve an instance of the Ignition > > > > interface? > > > > >> - Are there any plans to start thin clients from Ignition > interface, > > > or > > > > >> should we have a separate way of doing this? > > > > >> > > > > >> > > > > >> [1] > > > > >> > > > > > > > > > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=158873958 > > > > >> > > > > > > > > > >
Re: Ignite 3.0 Ignition API, node startup, and thin client startup
Pavel, Yes, makes sense. -Val On Wed, Jul 7, 2021 at 11:50 PM Pavel Tupitsyn wrote: > Val, > > So the plan is: > > - Remove Ignition#start from the public API > - Make Ignition a class, not an interface > - Add static Ignition#startClient > > Sounds good? > > On Thu, Jul 8, 2021 at 6:13 AM Valentin Kulichenko < > valentin.kuliche...@gmail.com> wrote: > > > Hi Ivan, > > > > Ignition IS the entry point to Ignite, so I'm not sure I got your point > :) > > Where is the contradiction? > > > > Either way, please feel free to give your suggestions for an alternative > > name if you have any. > > > > -Val > > > > On Wed, Jul 7, 2021 at 7:56 PM Ivan Pavlukhina > > wrote: > > > > > A side note. Actually “Ignition” naming always confused me. I think > about > > > it as some fancy named API entry point for Ignite. Perhaps it is a good > > > moment to revisit naming. > > > > > > > On 8 Jul 2021, at 07:09, Valentin Kulichenko < > > > valentin.kuliche...@gmail.com> wrote: > > > > > > > > Hi Pavel, > > > > > > > > I don't think we will need the pure embedded mode, but we still need > to > > > be > > > > able to access the API from compute and services. That said, there > are > > > two > > > > usages of the 'Ignite' API: > > > > > > > > 1. Remote, via the binary protocol. > > > > 2. Local - needed for compute and services. (This is how it works > > now.) > > > > > > > > I believe that the API should be the same, and there should be a > > unified > > > > access point. Ignition seems to be a good candidate for this. > > > > > > > > Ignition#start should eventually be removed from the public API. It > is > > > > currently there only because we don't have the thin client yet. > > > > > > > > -Val > > > > > > > >> On Wed, Jul 7, 2021 at 5:47 AM Pavel Tupitsyn > > > > wrote: > > > >> > > > >> Igniters, > > > >> > > > >> I have a few questions regarding server node startup and thin > clients. > > > >> > > > >> State of things: > > > >> - Server nodes will be started with 'ignite run' from CLI [1] > > > >> - ignite-api module represents our public API > > > >> - ignite-api has Ignition interface to start server nodes > > > >> > > > >> Questions: > > > >> - What's the idea behind Ignition interface in the public API? Are > we > > > going > > > >> to have an "embedded mode" where servers can be started from code? I > > > >> thought this was not planned. > > > >> - How are users supposed to retrieve an instance of the Ignition > > > interface? > > > >> - Are there any plans to start thin clients from Ignition interface, > > or > > > >> should we have a separate way of doing this? > > > >> > > > >> > > > >> [1] > > > >> > > > > > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=158873958 > > > >> > > > > > >
Re: Ignite 3.0 Ignition API, node startup, and thin client startup
I'm sorry, but why we didn't consider to separate creation of Client and connection? Why not to make async variant of connection? See for example [1] [1] --- https://lettuce.io/core/release/api/index.html чт, 8 июл. 2021 г., 09:50 Pavel Tupitsyn : > Val, > > So the plan is: > > - Remove Ignition#start from the public API > - Make Ignition a class, not an interface > - Add static Ignition#startClient > > Sounds good? > > On Thu, Jul 8, 2021 at 6:13 AM Valentin Kulichenko < > valentin.kuliche...@gmail.com> wrote: > > > Hi Ivan, > > > > Ignition IS the entry point to Ignite, so I'm not sure I got your point > :) > > Where is the contradiction? > > > > Either way, please feel free to give your suggestions for an alternative > > name if you have any. > > > > -Val > > > > On Wed, Jul 7, 2021 at 7:56 PM Ivan Pavlukhina > > wrote: > > > > > A side note. Actually “Ignition” naming always confused me. I think > about > > > it as some fancy named API entry point for Ignite. Perhaps it is a good > > > moment to revisit naming. > > > > > > > On 8 Jul 2021, at 07:09, Valentin Kulichenko < > > > valentin.kuliche...@gmail.com> wrote: > > > > > > > > Hi Pavel, > > > > > > > > I don't think we will need the pure embedded mode, but we still need > to > > > be > > > > able to access the API from compute and services. That said, there > are > > > two > > > > usages of the 'Ignite' API: > > > > > > > > 1. Remote, via the binary protocol. > > > > 2. Local - needed for compute and services. (This is how it works > > now.) > > > > > > > > I believe that the API should be the same, and there should be a > > unified > > > > access point. Ignition seems to be a good candidate for this. > > > > > > > > Ignition#start should eventually be removed from the public API. It > is > > > > currently there only because we don't have the thin client yet. > > > > > > > > -Val > > > > > > > >> On Wed, Jul 7, 2021 at 5:47 AM Pavel Tupitsyn > > > > wrote: > > > >> > > > >> Igniters, > > > >> > > > >> I have a few questions regarding server node startup and thin > clients. > > > >> > > > >> State of things: > > > >> - Server nodes will be started with 'ignite run' from CLI [1] > > > >> - ignite-api module represents our public API > > > >> - ignite-api has Ignition interface to start server nodes > > > >> > > > >> Questions: > > > >> - What's the idea behind Ignition interface in the public API? Are > we > > > going > > > >> to have an "embedded mode" where servers can be started from code? I > > > >> thought this was not planned. > > > >> - How are users supposed to retrieve an instance of the Ignition > > > interface? > > > >> - Are there any plans to start thin clients from Ignition interface, > > or > > > >> should we have a separate way of doing this? > > > >> > > > >> > > > >> [1] > > > >> > > > > > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=158873958 > > > >> > > > > > >
[MTCGA]: new failures in builds [6077453] needs to be handled
Hi Igniters, I've detected some new issue on TeamCity to be handled. You are more than welcomed to help. *New test failure in master-nightly HistoricalRebalanceTwoPartsInDifferentCheckpointsTest.testOlderPartitionUpdateFirst https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=434349873165174858=%3Cdefault%3E=testDetails No changes in the build - Here's a reminder of what contributors were agreed to do https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute - Should you have any questions please contact dev@ignite.apache.org Best Regards, Apache Ignite TeamCity Bot https://github.com/apache/ignite-teamcity-bot Notification generated at 21:14:28 08-07-2021
Re: [DOCUMENTATION] Create of Ignite-style images
Hi Nikolay, Check the “svg” folders which contain most of the images we use across the website and docs. You can build new pics out of those. — Denis > On Jul 8, 2021, at 11:20 PM, Nikolay Izhikov wrote: > > Hello, Igniters. > > We have many beautiful images [1] in documentation. > I want to create images that have similar style with other to keep docs > consistent. > Do we have some guides or styles I can use? > > [1] https://ignite.apache.org/docs/2.10.0/images/ignite_clustering.png
[DOCUMENTATION] Create of Ignite-style images
Hello, Igniters. We have many beautiful images [1] in documentation. I want to create images that have similar style with other to keep docs consistent. Do we have some guides or styles I can use? [1] https://ignite.apache.org/docs/2.10.0/images/ignite_clustering.png
Re: Problem with dropping the index
I assume DurableBackgroundTask will no longer use references to root ids. The proposed solution looks good to me. чт, 8 июл. 2021 г. в 10:28, Ivan Bessonov : > Hi guys, > > I see that the original message and its clarification is hard to consume. > The problem is that indexes are identified by their names. So, the > situation > that Kirill described is a valid one: > - you have index X for columns (a, b); > - you drop it; > - you create index X for columns (a, c). > > This is a realistic scenario. If all of that happened during a single > checkpoint > interval and node failed. > > Real issue here is the fact that we restore metastorage separately from > everything else. > This means that DurableBackgroundTask will start deleting X (a, c) if we're > not careful. > > It all depends on the implementation. I'd suggest Alexeys solution with a > few tweaks: > - I'm not convinced that we need to log indexes creation, this is > excessive; > - how do you drop index for the metatree: > -- pick a unique name; > -- acquire checkpoint read lock; > -- create DurableBackgroundTask that will delete index with that unique > name; > -- create logical WAL record "drop index X"; > -- "rename" link to the index in meta tree; > -- release checkpoint read lock; > -- start DurableBackgroundTask. > > Recovery for "drop index X" should also create another > DurableBackgroundTask, > this way I think we'll be able to avoid any possible issues. If > DurableBackgroundTask > has a name of an unexisting index then that's fine, task will be completed > as > successful, that's a normal flow. > > I hope everything's clear here, I'm ready to clarify more details about my > idea. Thank you! > > > ср, 7 июл. 2021 г. в 17:20, ткаленко кирилл : > > > I'll try to explain: > > > > We create indexes through InlineIndexFactory#createIndex, where at the > > beginning we create the root page using QueryIndexDefinition#treeName, so > > if we create the same index 2 times, then we will refer to the same root. > > > > In order to make the deletion of the index independent, I propose to > > replace the name of the root pages with an arbitrary name (other than > > QueryIndexDefinition#treeName, for example UUID.toString()), in order to > > avoid the problems of a node crash, I propose to fix this in a logical > > record. > > > > After we change the name of the root page to delete the index, we can > > easily delete it and rebuild it (if the user does it) in parallel. > > > > If we consider your proposal, then the creation and deletion of the index > > will go in parallel and they will look at the same roots. > > > > You can see how the renaming will take place here: > > https://github.com/apache/ignite/pull/9223 > > > > 07.07.2021, 10:28, "Alexei Scherbakov" : > > > вт, 6 июл. 2021 г. в 15:57, ткаленко кирилл : > > > > > >> >> Can you clarify what it means to rename root index trees ? > > >> Replacing > > >> > > > org.apache.ignite.internal.processors.cache.persistence.IndexStorageImpl.IndexItem, > > >> changing IndexItem#idxName, but keeping fIndexItem#pageId. > > > > > > Changing to what ? Some temporary name ? Can you give a detailed step > by > > > step description of the algorithm ? > > > > > >> Suggested solution is not suitable for the situation: add index -> > drop > > >> index -> add an index. We can start deleting the last added index. > > > > > > How can we do that, give me an example ? > > > > > > From my understanding, the suggested solution should work ok for any > > number > > > of create/drop sequences. > > > > > >> 06.07.2021, 14:00, "Alexei Scherbakov" >: > > >> > Can you clarify what it means to rename root index trees ? > > >> > > > >> > The simple solution which immediately comes to me is > > >> > > > >> > 1) write logical record on index creation - on reading it create an > > index > > >> > during logical recovery > > >> > 2) write logical record on index deletion - on reading it delete an > > index > > >> > during logical recovery and start background clearing task with > real > > root > > >> > pages. > > >> > > > >> > Will it work for you ? > > >> > > > >> > вт, 6 июл. 2021 г. в 12:27, ткаленко кирилл >: > > >> > > > >> >> Hello everyone! > > >> >> > > >> >> Currently, dropping indexes consists of the following steps (based > > on > > >> >> SchemaAbstractDiscoveryMessage's): > > >> >> > > >> >> Step 1: Removing the index from the SQL engine and starting > > >> >> DurableBackgroundCleanupIndexTreeTask, which removes the index > trees > > >> in the > > >> >> background; > > >> >> Step 1.1: DurableBackgroundCleanupIndexTreeTask is added to the > > >> >> metaStorage and removed after successful completion at the next > > >> checkpoint. > > >> >> > > >> >> Step 2: Removing the index from the cache configuration and > persist > > it. > > >> >> > > >> >> Problems: > > >> >> > > >> >> 1)We add and immediately delete the index, a checkpoint does not > > happen > > >> >> and the node crashes,
Re: Problem with dropping the index
Hi guys, I see that the original message and its clarification is hard to consume. The problem is that indexes are identified by their names. So, the situation that Kirill described is a valid one: - you have index X for columns (a, b); - you drop it; - you create index X for columns (a, c). This is a realistic scenario. If all of that happened during a single checkpoint interval and node failed. Real issue here is the fact that we restore metastorage separately from everything else. This means that DurableBackgroundTask will start deleting X (a, c) if we're not careful. It all depends on the implementation. I'd suggest Alexeys solution with a few tweaks: - I'm not convinced that we need to log indexes creation, this is excessive; - how do you drop index for the metatree: -- pick a unique name; -- acquire checkpoint read lock; -- create DurableBackgroundTask that will delete index with that unique name; -- create logical WAL record "drop index X"; -- "rename" link to the index in meta tree; -- release checkpoint read lock; -- start DurableBackgroundTask. Recovery for "drop index X" should also create another DurableBackgroundTask, this way I think we'll be able to avoid any possible issues. If DurableBackgroundTask has a name of an unexisting index then that's fine, task will be completed as successful, that's a normal flow. I hope everything's clear here, I'm ready to clarify more details about my idea. Thank you! ср, 7 июл. 2021 г. в 17:20, ткаленко кирилл : > I'll try to explain: > > We create indexes through InlineIndexFactory#createIndex, where at the > beginning we create the root page using QueryIndexDefinition#treeName, so > if we create the same index 2 times, then we will refer to the same root. > > In order to make the deletion of the index independent, I propose to > replace the name of the root pages with an arbitrary name (other than > QueryIndexDefinition#treeName, for example UUID.toString()), in order to > avoid the problems of a node crash, I propose to fix this in a logical > record. > > After we change the name of the root page to delete the index, we can > easily delete it and rebuild it (if the user does it) in parallel. > > If we consider your proposal, then the creation and deletion of the index > will go in parallel and they will look at the same roots. > > You can see how the renaming will take place here: > https://github.com/apache/ignite/pull/9223 > > 07.07.2021, 10:28, "Alexei Scherbakov" : > > вт, 6 июл. 2021 г. в 15:57, ткаленко кирилл : > > > >> >> Can you clarify what it means to rename root index trees ? > >> Replacing > >> > > org.apache.ignite.internal.processors.cache.persistence.IndexStorageImpl.IndexItem, > >> changing IndexItem#idxName, but keeping fIndexItem#pageId. > > > > Changing to what ? Some temporary name ? Can you give a detailed step by > > step description of the algorithm ? > > > >> Suggested solution is not suitable for the situation: add index -> drop > >> index -> add an index. We can start deleting the last added index. > > > > How can we do that, give me an example ? > > > > From my understanding, the suggested solution should work ok for any > number > > of create/drop sequences. > > > >> 06.07.2021, 14:00, "Alexei Scherbakov" : > >> > Can you clarify what it means to rename root index trees ? > >> > > >> > The simple solution which immediately comes to me is > >> > > >> > 1) write logical record on index creation - on reading it create an > index > >> > during logical recovery > >> > 2) write logical record on index deletion - on reading it delete an > index > >> > during logical recovery and start background clearing task with real > root > >> > pages. > >> > > >> > Will it work for you ? > >> > > >> > вт, 6 июл. 2021 г. в 12:27, ткаленко кирилл : > >> > > >> >> Hello everyone! > >> >> > >> >> Currently, dropping indexes consists of the following steps (based > on > >> >> SchemaAbstractDiscoveryMessage's): > >> >> > >> >> Step 1: Removing the index from the SQL engine and starting > >> >> DurableBackgroundCleanupIndexTreeTask, which removes the index trees > >> in the > >> >> background; > >> >> Step 1.1: DurableBackgroundCleanupIndexTreeTask is added to the > >> >> metaStorage and removed after successful completion at the next > >> checkpoint. > >> >> > >> >> Step 2: Removing the index from the cache configuration and persist > it. > >> >> > >> >> Problems: > >> >> > >> >> 1)We add and immediately delete the index, a checkpoint does not > happen > >> >> and the node crashes, after restarting > >> >> DurableBackgroundCleanupIndexTreeTask will not be able to complete > and > >> will > >> >> periodically restart due to the fact that it saves > >> >> DurableBackgroundCleanupIndexTreeTask#rootPages (root pages of index > >> trees) > >> >> that have not appeared; > >> >> > >> >> 2)After adding a DurableBackgroundCleanupIndexTreeTask node crashes, > >> after > >> >> restarting the node, the
Re: Ignite 3.0 Ignition API, node startup, and thin client startup
Val, So the plan is: - Remove Ignition#start from the public API - Make Ignition a class, not an interface - Add static Ignition#startClient Sounds good? On Thu, Jul 8, 2021 at 6:13 AM Valentin Kulichenko < valentin.kuliche...@gmail.com> wrote: > Hi Ivan, > > Ignition IS the entry point to Ignite, so I'm not sure I got your point :) > Where is the contradiction? > > Either way, please feel free to give your suggestions for an alternative > name if you have any. > > -Val > > On Wed, Jul 7, 2021 at 7:56 PM Ivan Pavlukhina > wrote: > > > A side note. Actually “Ignition” naming always confused me. I think about > > it as some fancy named API entry point for Ignite. Perhaps it is a good > > moment to revisit naming. > > > > > On 8 Jul 2021, at 07:09, Valentin Kulichenko < > > valentin.kuliche...@gmail.com> wrote: > > > > > > Hi Pavel, > > > > > > I don't think we will need the pure embedded mode, but we still need to > > be > > > able to access the API from compute and services. That said, there are > > two > > > usages of the 'Ignite' API: > > > > > > 1. Remote, via the binary protocol. > > > 2. Local - needed for compute and services. (This is how it works > now.) > > > > > > I believe that the API should be the same, and there should be a > unified > > > access point. Ignition seems to be a good candidate for this. > > > > > > Ignition#start should eventually be removed from the public API. It is > > > currently there only because we don't have the thin client yet. > > > > > > -Val > > > > > >> On Wed, Jul 7, 2021 at 5:47 AM Pavel Tupitsyn > > wrote: > > >> > > >> Igniters, > > >> > > >> I have a few questions regarding server node startup and thin clients. > > >> > > >> State of things: > > >> - Server nodes will be started with 'ignite run' from CLI [1] > > >> - ignite-api module represents our public API > > >> - ignite-api has Ignition interface to start server nodes > > >> > > >> Questions: > > >> - What's the idea behind Ignition interface in the public API? Are we > > going > > >> to have an "embedded mode" where servers can be started from code? I > > >> thought this was not planned. > > >> - How are users supposed to retrieve an instance of the Ignition > > interface? > > >> - Are there any plans to start thin clients from Ignition interface, > or > > >> should we have a separate way of doing this? > > >> > > >> > > >> [1] > > >> > > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=158873958 > > >> > > >
