[jira] [Updated] (IGNITE-21213) Coordination of mechanisms of determination for primary on replicaside
[ https://issues.apache.org/jira/browse/IGNITE-21213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladislav Pyatkov updated IGNITE-21213: --- Description: h3. Motivation In the replica listener, we have unconsidered mechanisms between each other to determine primary rteplica. The first one is based on the placement driver API (it is used in _PartitionReplicaListener#ensureReplicaIsPrimary_) and the other one is based on the placement driver events (the events are hadeled by two methods: _ReplicaManager#onPrimaryReplicaElected_, _ReplicaManager#onPrimaryReplicaExpired_). Because the replica messages and events are handled in different threads, any variety of processing is possible. For example, the replica can release all transaction locks (by PRIMARY_REPLICA_EXPIRED event) and then handle a message for this transaction (because ensureReplicaIsPrimary was done before), assuming that all the locks are holding. h3. Definition of done The two mechanisms work in coordination. was: h3. Motivation In the replica listener, we have unconsidered mechanisms between each other to determine primary rteplica. The first one is based on the placement driver API (it is used in PartitionReplicaListener#ensureReplicaIsPrimary) and the other one is based on the placement driver events (the events are hadeled by two methods: ReplicaManager#onPrimaryReplicaElected, ReplicaManager#onPrimaryReplicaExpired). Because the replica messages and events are handled in different threads, any variety of processing is possible. For example, the replica can release all transaction locks (by PRIMARY_REPLICA_EXPIRED event) and then handle a message for this transaction (because ensureReplicaIsPrimary was done before), assuming that all the locks are holding. h3. Definition of done The two mechanisms work in coordination. > Coordination of mechanisms of determination for primary on replicaside > -- > > Key: IGNITE-21213 > URL: https://issues.apache.org/jira/browse/IGNITE-21213 > Project: Ignite > Issue Type: Bug >Reporter: Vladislav Pyatkov >Priority: Major > Labels: ignite-3 > > h3. Motivation > In the replica listener, we have unconsidered mechanisms between each other > to determine primary rteplica. The first one is based on the placement driver > API (it is used in _PartitionReplicaListener#ensureReplicaIsPrimary_) and the > other one is based on the placement driver events (the events are hadeled by > two methods: _ReplicaManager#onPrimaryReplicaElected_, > _ReplicaManager#onPrimaryReplicaExpired_). > Because the replica messages and events are handled in different threads, any > variety of processing is possible. For example, the replica can release all > transaction locks (by > PRIMARY_REPLICA_EXPIRED event) and then handle a message for this transaction > (because ensureReplicaIsPrimary was done before), assuming that all the locks > are holding. > h3. Definition of done > The two mechanisms work in coordination. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-21213) Coordination of mechanisms of determination for primary on replicaside
[ https://issues.apache.org/jira/browse/IGNITE-21213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladislav Pyatkov updated IGNITE-21213: --- Description: h3. Motivation In the replica listener, we have unconsidered mechanisms between each other to determine primary rteplica. The first one is based on the placement driver API (it is used in PartitionReplicaListener#ensureReplicaIsPrimary) and the other one is based on the placement driver events (the events are hadeled by two methods: ReplicaManager#onPrimaryReplicaElected, ReplicaManager#onPrimaryReplicaExpired). Because the replica messages and events are handled in different threads, any variety of processing is possible. For example, the replica can release all transaction locks (by PRIMARY_REPLICA_EXPIRED event) and then handle a message for this transaction (because ensureReplicaIsPrimary was done before), assuming that all the locks are holding. h3. Definition of done The two mechanisms work in coordination. was: h3. Motivation In the replica listener, we have unconsidered mechanisms between each other to determine primary rteplica. The first one is based on the placement driver API (it is used in PartitionReplicaListener#ensureReplicaIsPrimary) and the other one is based on the placement driver events (the events are hadeled by two methods: ReplicaManager#onPrimaryReplicaElected, ReplicaManager#onPrimaryReplicaExpired). Because the replica messages and events are handled in different threads, any variety of processing is possible. For example, the replica can release all transaction locks (by PRIMARY_REPLICA_EXPIRED event) and then handle a message for this transaction (because ensureReplicaIsPrimary was done before), assuming that all the locks are holding. > Coordination of mechanisms of determination for primary on replicaside > -- > > Key: IGNITE-21213 > URL: https://issues.apache.org/jira/browse/IGNITE-21213 > Project: Ignite > Issue Type: Bug >Reporter: Vladislav Pyatkov >Priority: Major > Labels: ignite-3 > > h3. Motivation > In the replica listener, we have unconsidered mechanisms between each other > to determine primary rteplica. The first one is based on the placement driver > API (it is used in PartitionReplicaListener#ensureReplicaIsPrimary) and the > other one is based on the placement driver events (the events are hadeled by > two methods: ReplicaManager#onPrimaryReplicaElected, > ReplicaManager#onPrimaryReplicaExpired). > Because the replica messages and events are handled in different threads, any > variety of processing is possible. For example, the replica can release all > transaction locks (by > PRIMARY_REPLICA_EXPIRED event) and then handle a message for this transaction > (because ensureReplicaIsPrimary was done before), assuming that all the locks > are holding. > h3. Definition of done > The two mechanisms work in coordination. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-16947) Resolve message send future when message is acknowledged
[ https://issues.apache.org/jira/browse/IGNITE-16947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Puchkovskiy updated IGNITE-16947: --- Reviewer: Aleksandr Polovtcev (was: Ivan Bessonov) > Resolve message send future when message is acknowledged > > > Key: IGNITE-16947 > URL: https://issues.apache.org/jira/browse/IGNITE-16947 > Project: Ignite > Issue Type: Improvement > Components: networking >Reporter: Semyon Danilov >Assignee: Roman Puchkovskiy >Priority: Major > Labels: ignite-3 > Time Spent: 10m > Remaining Estimate: 0h > > At this moment, NettySender#send returns a future that is resolved when > message is written into the network buffer. We should resolve this future > when we receive an acknowledgment from the remote node. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-21213) Coordination of mechanisms of determination for primary on replicaside
[ https://issues.apache.org/jira/browse/IGNITE-21213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladislav Pyatkov updated IGNITE-21213: --- Labels: ignite-3 (was: ) > Coordination of mechanisms of determination for primary on replicaside > -- > > Key: IGNITE-21213 > URL: https://issues.apache.org/jira/browse/IGNITE-21213 > Project: Ignite > Issue Type: Bug >Reporter: Vladislav Pyatkov >Priority: Major > Labels: ignite-3 > > h3. Motivation > In the replica listener, we have unconsidered mechanisms between each other > to determine primary rteplica. The first one is based on the placement driver > API (it is used in PartitionReplicaListener#ensureReplicaIsPrimary) and the > other one is based on the placement driver events (the events are hadeled by > two methods: ReplicaManager#onPrimaryReplicaElected, > ReplicaManager#onPrimaryReplicaExpired). > Because the replica messages and events are handled in different threads, any > variety of processing is possible. For example, the replica can release all > transaction locks (by > PRIMARY_REPLICA_EXPIRED event) and then handle a message for this transaction > (because ensureReplicaIsPrimary was done before), assuming that all the locks > are holding. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-21213) Coordination of mechanisms of determination for primary on replicaside
[ https://issues.apache.org/jira/browse/IGNITE-21213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladislav Pyatkov updated IGNITE-21213: --- Description: h3. Motivation In the replica listener, we have unconsidered mechanisms between each other to determine primary rteplica. The first one is based on the placement driver API (it is used in PartitionReplicaListener#ensureReplicaIsPrimary) and the other one is based on the placement driver events (the events are hadeled by two methods: ReplicaManager#onPrimaryReplicaElected, ReplicaManager#onPrimaryReplicaExpired). Because the replica messages and events are handled in different threads, any variety of processing is possible. For example, the replica can release all transaction locks (by PRIMARY_REPLICA_EXPIRED event) and then handle a message for this transaction (because ensureReplicaIsPrimary was done before), assuming that all the locks are holding. was: h3. Motivation In the replica listener, we have unconsidered mechanisms between each other to determine primary rteplica. The first one is based on the placement driver API (it is used in PartitionReplicaListener#ensureReplicaIsPrimary) and the other one is based on the placement driver events (the events are hadeled by two methods: ReplicaManager#onPrimaryReplicaElected, ReplicaManager#onPrimaryReplicaExpired). Because the replica messages and events are handled in different threads, any variety of processing is possible. The replica can release all transaction locks (by PRIMARY_REPLICA_EXPIRED event) and then handle a message for this transaction (because ensureReplicaIsPrimary was done before), assuming that all the locks are holding. > Coordination of mechanisms of determination for primary on replicaside > -- > > Key: IGNITE-21213 > URL: https://issues.apache.org/jira/browse/IGNITE-21213 > Project: Ignite > Issue Type: Bug >Reporter: Vladislav Pyatkov >Priority: Major > > h3. Motivation > In the replica listener, we have unconsidered mechanisms between each other > to determine primary rteplica. The first one is based on the placement driver > API (it is used in PartitionReplicaListener#ensureReplicaIsPrimary) and the > other one is based on the placement driver events (the events are hadeled by > two methods: ReplicaManager#onPrimaryReplicaElected, > ReplicaManager#onPrimaryReplicaExpired). > Because the replica messages and events are handled in different threads, any > variety of processing is possible. For example, the replica can release all > transaction locks (by > PRIMARY_REPLICA_EXPIRED event) and then handle a message for this transaction > (because ensureReplicaIsPrimary was done before), assuming that all the locks > are holding. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-21213) Coordination of mechanisms of determination for primary on replicaside
Vladislav Pyatkov created IGNITE-21213: -- Summary: Coordination of mechanisms of determination for primary on replicaside Key: IGNITE-21213 URL: https://issues.apache.org/jira/browse/IGNITE-21213 Project: Ignite Issue Type: Bug Reporter: Vladislav Pyatkov h3. Motivation In the replica listener, we have unconsidered mechanisms between each other to determine primary rteplica. The first one is based on the placement driver API (it is used in PartitionReplicaListener#ensureReplicaIsPrimary) and the other one is based on the placement driver events (the events are hadeled by two methods: ReplicaManager#onPrimaryReplicaElected, ReplicaManager#onPrimaryReplicaExpired). Because the replica messages and events are handled in different threads, any variety of processing is possible. The replica can release all transaction locks (by PRIMARY_REPLICA_EXPIRED event) and then handle a message for this transaction (because ensureReplicaIsPrimary was done before), assuming that all the locks are holding. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20477) Async component start
[
https://issues.apache.org/jira/browse/IGNITE-20477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mirza Aliev updated IGNITE-20477:
-
Reviewer: Aleksandr Polovtcev
> Async component start
> -
>
> Key: IGNITE-20477
> URL: https://issues.apache.org/jira/browse/IGNITE-20477
> Project: Ignite
> Issue Type: Task
>Reporter: Aleksandr Polovtcev
>Assignee: Mirza Aliev
>Priority: Major
> Labels: ignite-3
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Currently all Ignite components start synchronously (see
> {{IgniteComponent#start}}). This is inconvenient, because some components can
> complete their startup only when Meta Storage has initialized all Version
> Values (see {{IgniteImpl#recoverComponentsStateOnStart}}). Because of this,
> some components employ a hack which consists of having a "special" Versioned
> Value, which is injected with a future that gets resolved only after the
> enclosing component completes its startup (see {{startVv}} in
> {{TableManager}} or {{IndexManager}}). This blocks the Watch Processor inside
> Meta Storage, preventing it from processing further updates.
> This problem with this approach that it is quite cryptic and hard to
> understand. Instead I propose to do the following:
> # Change the signature of {{IgniteComponent#start}} to
> {{CompletableFuture start()}}.
> # All actions in the components startup will be divided into two categories:
> sync actions, that can be executed synchronously in order for the component
> to be usable by other components during their startup, and async actions,
> which need to wait for any of the Versioned Values to be initialized by the
> Meta Storage. Such async actions should be wrapped in a {{CompletableFuture}}
> and returned from the {{start}} method.
> # {{IgniteImpl}} startup procedure should be updated to collect the futures
> from all components and wait for their completion inside
> {{recoverComponentsStateOnStart}}, after
> {{metaStorageMgr.notifyRevisionUpdateListenerOnStart()}} has been called, but
> before Watches are deployed ({{metaStorageMgr.deployWatches()}})
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-21209) Do not try handle PrimaryReplicaMissException on transaction operation
[ https://issues.apache.org/jira/browse/IGNITE-21209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladislav Pyatkov updated IGNITE-21209: --- Reviewer: Denis Chudov > Do not try handle PrimaryReplicaMissException on transaction operation > -- > > Key: IGNITE-21209 > URL: https://issues.apache.org/jira/browse/IGNITE-21209 > Project: Ignite > Issue Type: Improvement >Reporter: Vladislav Pyatkov >Assignee: Vladislav Pyatkov >Priority: Major > Labels: ignite-3 > Time Spent: 10m > Remaining Estimate: 0h > > h3. Motivation > A transaction enlists a primary replica before making an operation request. > But the PrimaryReplicaMissException can be thrown after the request tries to > be sent. This means that any ties to remap the request cannot enlist a new > primary replica, and all this is meaningless. The code, which handles the > PrimaryReplicaMissException, just wastes time. > h3. Definition of done > Do not handle the PrimaryReplicaMissException on any transaction operation > after the primary replica is already enlisted. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IGNITE-21208) Sql. DEFAULT expression with NULL value processed unexpectedly
[
https://issues.apache.org/jira/browse/IGNITE-21208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Evgeny Stanilovsky reassigned IGNITE-21208:
---
Assignee: Evgeny Stanilovsky
> Sql. DEFAULT expression with NULL value processed unexpectedly
> --
>
> Key: IGNITE-21208
> URL: https://issues.apache.org/jira/browse/IGNITE-21208
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Affects Versions: 3.0.0-beta1
>Reporter: Evgeny Stanilovsky
>Assignee: Evgeny Stanilovsky
>Priority: Major
> Labels: ignite-3
>
> Correct processing for NULL consistent expressions require additional
> overriding of IgniteSqlToRelConvertor#convertValues, seems we can avoid it.
> Expressions need to be passed without such overriding method:
> {noformat}
> create table t (id int, c int DEFAULT null);
> insert into t(id, c) values(1, DEFAULT);
> insert into t(id) values(2);
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-21212) Increment and Decrement REST API documentation lists incorrect parameter
Igor Belyakov created IGNITE-21212: -- Summary: Increment and Decrement REST API documentation lists incorrect parameter Key: IGNITE-21212 URL: https://issues.apache.org/jira/browse/IGNITE-21212 Project: Ignite Issue Type: Bug Components: documentation Affects Versions: 2.16 Reporter: Igor Belyakov The documentation for the Increment/Decrement REST API endpoints (https://ignite.apache.org/docs/latest/restapi#increment) contains information regarding the "cacheName" parameter, which in fact this endpoint doesn't have. (see: [https://github.com/apache/ignite/blob/ignite-2.16/modules/core/src/main/java/org/apache/ignite/internal/processors/rest/request/DataStructuresRequest.java]) This redundant parameter could be confusing since with it the endpoint looks like an API to increment/decrement the value stored in the cache, while in fact it is related to the IgniteAtomicLong data structure. (see [https://ignite.apache.org/docs/latest/data-structures/atomic-types]). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (IGNITE-21022) Replicate PK along with RowId for row removal
[ https://issues.apache.org/jira/browse/IGNITE-21022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey N. Gura resolved IGNITE-21022. - Resolution: Won't Fix There is no need to replicate PK in order to remove a row from a storage. > Replicate PK along with RowId for row removal > - > > Key: IGNITE-21022 > URL: https://issues.apache.org/jira/browse/IGNITE-21022 > Project: Ignite > Issue Type: Improvement >Reporter: Ivan Bessonov >Priority: Major > Labels: ignite-3 > > Right now, we replicate only RowId in UpdateCommand/UpdateAllCommand during > the remove. This may not be a desired solution for the future. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-21206) Lease grant messages should be retried after the failure of the placement driver active actor
[ https://issues.apache.org/jira/browse/IGNITE-21206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17804355#comment-17804355 ] Vladislav Pyatkov commented on IGNITE-21206: Merged 8d3b926e0b8a3ad39591f46a4903d13f4efdb966 > Lease grant messages should be retried after the failure of the placement > driver active actor > - > > Key: IGNITE-21206 > URL: https://issues.apache.org/jira/browse/IGNITE-21206 > Project: Ignite > Issue Type: Improvement >Reporter: Denis Chudov >Assignee: Denis Chudov >Priority: Major > Labels: ignite-3 > Time Spent: 20m > Remaining Estimate: 0h > > If the lease grant message response is sent to the node which is no longer > active, the corresponding lease (which expires after the long time interval > intended for lease acceptance) will not be negotiated again by a new active > actor until it expires. So we will not have the primary replica for the lease > acceptance interval, which is currently 2 minutes. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20380) Try to deal with TableManager#assignmentsChangeListeners
[ https://issues.apache.org/jira/browse/IGNITE-20380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Puchkovskiy updated IGNITE-20380: --- Description: *org.apache.ignite.internal.table.distributed.TableManager#assignmentsChangeListeners* that are used in *org.apache.ignite.client.handler.ClientInboundMessageHandler* have been discovered and I would like to see if it would be possible to change these listeners to catalog listeners for example or something like that. And also, most likely, the call of listeners should be made after writing to the metastore. was: *org.apache.ignite.internal.table.distributed.TableManager#assignmentsChangeListeners* that are used in *org.apache.ignite.client.handler.ClientInboundMessageHandler* have been discovered and I would like to see if it would be possible to change these listeners to catalog listeners for example or something like that. And also, most likely, the call of listeners should be made after writing to the metastore. > Try to deal with TableManager#assignmentsChangeListeners > > > Key: IGNITE-20380 > URL: https://issues.apache.org/jira/browse/IGNITE-20380 > Project: Ignite > Issue Type: Improvement >Reporter: Kirill Tkalenko >Priority: Major > Labels: ignite-3 > > *org.apache.ignite.internal.table.distributed.TableManager#assignmentsChangeListeners* > that are used in > *org.apache.ignite.client.handler.ClientInboundMessageHandler* have been > discovered and I would like to see if it would be possible to change these > listeners to catalog listeners for example or something like that. > And also, most likely, the call of listeners should be made after writing to > the metastore. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20046) Improve the Catalog interfaces
[ https://issues.apache.org/jira/browse/IGNITE-20046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Puchkovskiy updated IGNITE-20046: --- Environment: (was: Catalog service Part 3) > Improve the Catalog interfaces > -- > > Key: IGNITE-20046 > URL: https://issues.apache.org/jira/browse/IGNITE-20046 > Project: Ignite > Issue Type: Improvement >Reporter: Kirill Tkalenko >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > Catalog related interfaces and classes are missing javadocs, need to fix this. > CatalogService getters, which accepts ID and timestamp, looks useless and, > probably, could be removed. > Let's also change timestamp type `long->HybridTimestamp` -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20263) Get rid of DataStorageConfigurationSchema
[
https://issues.apache.org/jira/browse/IGNITE-20263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Puchkovskiy updated IGNITE-20263:
---
Description: We need to get rid of
{*}org.apache.ignite.internal.schema.configuration.storage.DataStorageConfigurationSchema{*},
its descendants and the code associated with it. (was: We need to get rid of
*org.apache.ignite.internal.schema.configuration.storage.DataStorageConfigurationSchema*,
its descendants and the code associated with it.)
> Get rid of DataStorageConfigurationSchema
> -
>
> Key: IGNITE-20263
> URL: https://issues.apache.org/jira/browse/IGNITE-20263
> Project: Ignite
> Issue Type: Improvement
>Reporter: Kirill Tkalenko
>Priority: Major
> Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>
> We need to get rid of
> {*}org.apache.ignite.internal.schema.configuration.storage.DataStorageConfigurationSchema{*},
> its descendants and the code associated with it.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-19790) Read range from metastore at start UpdateLogImpl
[
https://issues.apache.org/jira/browse/IGNITE-19790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Puchkovskiy updated IGNITE-19790:
---
Description: Now, in
{*}org.apache.ignite.internal.catalog.storage.UpdateLogImpl#restoreStateFromVault{*},
versions are read from the volt, starting from 1, which is not correct because
the log can be cut off. We need to read from the metastore by range. (was:
Now, in
*org.apache.ignite.internal.catalog.storage.UpdateLogImpl#restoreStateFromVault*,
versions are read from the volt, starting from 1, which is not correct because
the log can be cut off. We need to read from the metastore by range.)
> Read range from metastore at start UpdateLogImpl
>
>
> Key: IGNITE-19790
> URL: https://issues.apache.org/jira/browse/IGNITE-19790
> Project: Ignite
> Issue Type: Improvement
>Reporter: Kirill Tkalenko
>Priority: Major
> Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>
> Now, in
> {*}org.apache.ignite.internal.catalog.storage.UpdateLogImpl#restoreStateFromVault{*},
> versions are read from the volt, starting from 1, which is not correct
> because the log can be cut off. We need to read from the metastore by range.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-19670) Improve CatalogService test coverage.
[ https://issues.apache.org/jira/browse/IGNITE-19670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Puchkovskiy updated IGNITE-19670: --- Ignite Flags: (was: Docs Required,Release Notes Required) > Improve CatalogService test coverage. > - > > Key: IGNITE-19670 > URL: https://issues.apache.org/jira/browse/IGNITE-19670 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Andrey Mashenkov >Priority: Major > Labels: ignite-3, tech-debt-test > > 1. CatalogServiceSelftTest.testCreateTable (+testDropTable) looks a bit > complicated. It checks creation of more than one table. Let's simplify the > test by reverting last changes > 2. We use shared counter to generate unique identifier for schema objects. > Some tests checks schema object id, and some doesn't. Let's move > schema-object's id check into separate test, to verify which command > increments the counter, and which doesn't. > 3. Let's add a test that will check ABA problem. E.g. create-drop-create > table (or index) with same name and check the object can be resolved > correctly by name and by id (regarding object versioning in Catalog, of > course). > 4. Move Catalog operations tests to separate class. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19643) Catalog validation code refactoring.
[ https://issues.apache.org/jira/browse/IGNITE-19643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Puchkovskiy updated IGNITE-19643: --- Description: Let's implement some validation rules that each Catalog command should check before the execution. Then replace current boilerplate validation code with applying these rules using Visitor pattern. (was: Let's implement some validation rules that each Catalog command should check before the execution. Then replace current boilerplate validation code with applying these rules using Visitor pattern.) > Catalog validation code refactoring. > > > Key: IGNITE-19643 > URL: https://issues.apache.org/jira/browse/IGNITE-19643 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Andrey Mashenkov >Priority: Minor > Labels: ignite-3, tech-debt > > Let's implement some validation rules that each Catalog command should check > before the execution. Then replace current boilerplate validation code with > applying these rules using Visitor pattern. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19719) Make CatalogDataStorageDescriptor support for each storage engine
[ https://issues.apache.org/jira/browse/IGNITE-19719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Puchkovskiy updated IGNITE-19719: --- Description: At the moment, we do not have the ability to implement *org.apache.ignite.internal.catalog.descriptors.CatalogDataStorageDescriptor* for each storage engine, we need to come up with a mechanism for how to do this. (was: At the moment, we do not have the ability to implement *org.apache.ignite.internal.catalog.descriptors.CatalogDataStorageDescriptor* for each storage engine, we need to come up with a mechanism for how to do this.) > Make CatalogDataStorageDescriptor support for each storage engine > - > > Key: IGNITE-19719 > URL: https://issues.apache.org/jira/browse/IGNITE-19719 > Project: Ignite > Issue Type: Improvement > Environment: Catalog service Part 3 >Reporter: Kirill Tkalenko >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > At the moment, we do not have the ability to implement > *org.apache.ignite.internal.catalog.descriptors.CatalogDataStorageDescriptor* > for each storage engine, we need to come up with a mechanism for how to do > this. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19550) Reuse IDs of dropped Catalog objects for new objects
[ https://issues.apache.org/jira/browse/IGNITE-19550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Puchkovskiy updated IGNITE-19550: --- Description: In IGNITE-19524, we are switching from UUIDs to ints as IDs of the Catalog objects (tables, indices, views, zones). Some users have use-cases where they constantly create, drop and recreate same tables. In such cases, it might make sense to reuse object IDs to slow down the growth of the global counter. A design is required. was: In IGNITE-19524, we are switching from UUIDs to ints as IDs of the Catalog objects (tables, indices, views, zones). Some users have use-cases where they constantly create, drop and recreate same tables. In such cases, it might make sense to reuse object IDs to slow down the growth of the global counter. A design is required. > Reuse IDs of dropped Catalog objects for new objects > > > Key: IGNITE-19550 > URL: https://issues.apache.org/jira/browse/IGNITE-19550 > Project: Ignite > Issue Type: Improvement >Reporter: Roman Puchkovskiy >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > In IGNITE-19524, we are switching from UUIDs to ints as IDs of the Catalog > objects (tables, indices, views, zones). Some users have use-cases where they > constantly create, drop and recreate same tables. In such cases, it might > make sense to reuse object IDs to slow down the growth of the global counter. > A design is required. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19082) Describe Catalog operation flow in README and cleanup dead code.
[ https://issues.apache.org/jira/browse/IGNITE-19082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Puchkovskiy updated IGNITE-19082: --- Description: Let's * ensure Catalog is used by default as a single schema management point by TableManager, IndexManager, SqlSchemaManager, SchemaRegistry. * Describe CatalogService operations in README.md * drop schema related code from configuration. * drop outdated code from TableManager, IndexManager, SqlSchemaManager, SchemaRegistry. * make a PR for merging feature branch to main (if applicable). * ensure there are end-to-end tests for the cases (if applicable) described in CatalogServiceSelfTest. Also drop InternalSchemaTest. was: Let's * ensure Catalog is used by default as a single schema management point by TableManager, IndexManager, SqlSchemaManager, SchemaRegistry. * Describe CatalogService operations in README.md * drop schema related code from configuration. * drop outdated code from TableManager, IndexManager, SqlSchemaManager, SchemaRegistry. * make a PR for merging feature branch to main (if applicable). * ensure there are end-to-end tests for the cases (if applicable) described in CatalogServiceSelfTest. Also drop InternalSchemaTest. > Describe Catalog operation flow in README and cleanup dead code. > > > Key: IGNITE-19082 > URL: https://issues.apache.org/jira/browse/IGNITE-19082 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Andrey Mashenkov >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > Let's > * ensure Catalog is used by default as a single schema management point by > TableManager, IndexManager, SqlSchemaManager, SchemaRegistry. > * Describe CatalogService operations in README.md > * drop schema related code from configuration. > * drop outdated code from TableManager, IndexManager, SqlSchemaManager, > SchemaRegistry. > * make a PR for merging feature branch to main (if applicable). > * ensure there are end-to-end tests for the cases (if applicable) described > in CatalogServiceSelfTest. Also drop InternalSchemaTest. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19082) Describe Catalog operation flow in README and cleanup dead code.
[ https://issues.apache.org/jira/browse/IGNITE-19082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Puchkovskiy updated IGNITE-19082: --- Description: Let's * ensure Catalog is used by default as a single schema management point by TableManager, IndexManager, SqlSchemaManager, SchemaRegistry. * Describe CatalogService operations in README.md * drop schema related code from configuration. * drop outdated code from TableManager, IndexManager, SqlSchemaManager, SchemaRegistry. * make a PR for merging feature branch to main (if applicable). * ensure there are end-to-end tests for the cases (if applicable) described in CatalogServiceSelfTest. Also drop InternalSchemaTest. was: Let's * ensure Catalog is used by default as a single schema management point by TableManager, IndexManager, SqlSchemaManager, SchemaRegistry. * Describe CatalogService operations in README.md * drop schema related code from configuration. * drop outdated code from TableManager, IndexManager, SqlSchemaManager, SchemaRegistry. * make a PR for merging feature branch to main (if applicable). * ensure there are end-to-end tests for the cases (if applicable) described in CatalogServiceSelfTest. Also drop InternalSchemaTest. > Describe Catalog operation flow in README and cleanup dead code. > > > Key: IGNITE-19082 > URL: https://issues.apache.org/jira/browse/IGNITE-19082 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Andrey Mashenkov >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > Let's > * ensure Catalog is used by default as a single schema management point by > TableManager, IndexManager, SqlSchemaManager, SchemaRegistry. > * Describe CatalogService operations in README.md > * drop schema related code from configuration. > * drop outdated code from TableManager, IndexManager, SqlSchemaManager, > SchemaRegistry. > * make a PR for merging feature branch to main (if applicable). > * ensure there are end-to-end tests for the cases (if applicable) described > in CatalogServiceSelfTest. Also drop InternalSchemaTest. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-21149) Structure for waiting for the completion of RW operations before starting to build an index for partitions
[ https://issues.apache.org/jira/browse/IGNITE-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17804323#comment-17804323 ] Roman Puchkovskiy commented on IGNITE-21149: The patch looks good to me > Structure for waiting for the completion of RW operations before starting to > build an index for partitions > -- > > Key: IGNITE-21149 > URL: https://issues.apache.org/jira/browse/IGNITE-21149 > Project: Ignite > Issue Type: Improvement >Reporter: Kirill Tkalenko >Assignee: Kirill Tkalenko >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > Time Spent: 4h 50m > Remaining Estimate: 0h > > To implement IGNITE-2, we need a structure that will allow us to wait for > the completion all in-flight operations of RW transactions before the index > appears in order to begin building indexes for partition. > The basic idea of this structure is described in IGNITE-2. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19790) Read range from metastore at start UpdateLogImpl
[ https://issues.apache.org/jira/browse/IGNITE-19790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Puchkovskiy updated IGNITE-19790: --- Epic Link: IGNITE-21211 (was: IGNITE-20473) > Read range from metastore at start UpdateLogImpl > > > Key: IGNITE-19790 > URL: https://issues.apache.org/jira/browse/IGNITE-19790 > Project: Ignite > Issue Type: Improvement >Reporter: Kirill Tkalenko >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > Now, in > *org.apache.ignite.internal.catalog.storage.UpdateLogImpl#restoreStateFromVault*, > versions are read from the volt, starting from 1, which is not correct > because the log can be cut off. We need to read from the metastore by range. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19082) Describe Catalog operation flow in README and cleanup dead code.
[ https://issues.apache.org/jira/browse/IGNITE-19082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Puchkovskiy updated IGNITE-19082: --- Epic Link: IGNITE-21211 (was: IGNITE-20473) > Describe Catalog operation flow in README and cleanup dead code. > > > Key: IGNITE-19082 > URL: https://issues.apache.org/jira/browse/IGNITE-19082 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Andrey Mashenkov >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > Let's > * ensure Catalog is used by default as a single schema management point by > TableManager, IndexManager, SqlSchemaManager, SchemaRegistry. > * Describe CatalogService operations in README.md > * drop schema related code from configuration. > * drop outdated code from TableManager, IndexManager, SqlSchemaManager, > SchemaRegistry. > * make a PR for merging feature branch to main (if applicable). > * ensure there are end-to-end tests for the cases (if applicable) described > in CatalogServiceSelfTest. Also drop InternalSchemaTest. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20380) Try to deal with TableManager#assignmentsChangeListeners
[ https://issues.apache.org/jira/browse/IGNITE-20380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Puchkovskiy updated IGNITE-20380: --- Epic Link: IGNITE-21211 (was: IGNITE-20473) > Try to deal with TableManager#assignmentsChangeListeners > > > Key: IGNITE-20380 > URL: https://issues.apache.org/jira/browse/IGNITE-20380 > Project: Ignite > Issue Type: Improvement >Reporter: Kirill Tkalenko >Priority: Major > Labels: ignite-3 > > *org.apache.ignite.internal.table.distributed.TableManager#assignmentsChangeListeners* > that are used in > *org.apache.ignite.client.handler.ClientInboundMessageHandler* have been > discovered and I would like to see if it would be possible to change these > listeners to catalog listeners for example or something like that. > And also, most likely, the call of listeners should be made after writing to > the metastore. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20046) Improve the Catalog interfaces
[ https://issues.apache.org/jira/browse/IGNITE-20046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Puchkovskiy updated IGNITE-20046: --- Epic Link: IGNITE-21211 (was: IGNITE-20473) Environment: Catalog service Part 3 > Improve the Catalog interfaces > -- > > Key: IGNITE-20046 > URL: https://issues.apache.org/jira/browse/IGNITE-20046 > Project: Ignite > Issue Type: Improvement > Environment: Catalog service Part 3 >Reporter: Kirill Tkalenko >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > Catalog related interfaces and classes are missing javadocs, need to fix this. > CatalogService getters, which accepts ID and timestamp, looks useless and, > probably, could be removed. > Let's also change timestamp type `long->HybridTimestamp` -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20263) Get rid of DataStorageConfigurationSchema
[ https://issues.apache.org/jira/browse/IGNITE-20263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Puchkovskiy updated IGNITE-20263: --- Epic Link: IGNITE-21211 (was: IGNITE-20473) > Get rid of DataStorageConfigurationSchema > - > > Key: IGNITE-20263 > URL: https://issues.apache.org/jira/browse/IGNITE-20263 > Project: Ignite > Issue Type: Improvement >Reporter: Kirill Tkalenko >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > We need to get rid of > *org.apache.ignite.internal.schema.configuration.storage.DataStorageConfigurationSchema*, > its descendants and the code associated with it. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19719) Make CatalogDataStorageDescriptor support for each storage engine
[ https://issues.apache.org/jira/browse/IGNITE-19719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Puchkovskiy updated IGNITE-19719: --- Epic Link: IGNITE-21211 (was: IGNITE-20473) Environment: Catalog service Part 3 > Make CatalogDataStorageDescriptor support for each storage engine > - > > Key: IGNITE-19719 > URL: https://issues.apache.org/jira/browse/IGNITE-19719 > Project: Ignite > Issue Type: Improvement > Environment: Catalog service Part 3 >Reporter: Kirill Tkalenko >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > At the moment, we do not have the ability to implement > *org.apache.ignite.internal.catalog.descriptors.CatalogDataStorageDescriptor* > for each storage engine, we need to come up with a mechanism for how to do > this. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19670) Improve CatalogService test coverage.
[ https://issues.apache.org/jira/browse/IGNITE-19670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Puchkovskiy updated IGNITE-19670: --- Epic Link: IGNITE-21211 (was: IGNITE-20473) > Improve CatalogService test coverage. > - > > Key: IGNITE-19670 > URL: https://issues.apache.org/jira/browse/IGNITE-19670 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Andrey Mashenkov >Priority: Major > Labels: ignite-3, tech-debt-test > > 1. CatalogServiceSelftTest.testCreateTable (+testDropTable) looks a bit > complicated. It checks creation of more than one table. Let's simplify the > test by reverting last changes > 2. We use shared counter to generate unique identifier for schema objects. > Some tests checks schema object id, and some doesn't. Let's move > schema-object's id check into separate test, to verify which command > increments the counter, and which doesn't. > 3. Let's add a test that will check ABA problem. E.g. create-drop-create > table (or index) with same name and check the object can be resolved > correctly by name and by id (regarding object versioning in Catalog, of > course). > 4. Move Catalog operations tests to separate class. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19550) Reuse IDs of dropped Catalog objects for new objects
[ https://issues.apache.org/jira/browse/IGNITE-19550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Puchkovskiy updated IGNITE-19550: --- Epic Link: IGNITE-21211 (was: IGNITE-20473) > Reuse IDs of dropped Catalog objects for new objects > > > Key: IGNITE-19550 > URL: https://issues.apache.org/jira/browse/IGNITE-19550 > Project: Ignite > Issue Type: Improvement >Reporter: Roman Puchkovskiy >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > In IGNITE-19524, we are switching from UUIDs to ints as IDs of the Catalog > objects (tables, indices, views, zones). Some users have use-cases where they > constantly create, drop and recreate same tables. In such cases, it might > make sense to reuse object IDs to slow down the growth of the global counter. > A design is required. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19643) Catalog validation code refactoring.
[ https://issues.apache.org/jira/browse/IGNITE-19643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Puchkovskiy updated IGNITE-19643: --- Epic Link: IGNITE-21211 (was: IGNITE-20473) > Catalog validation code refactoring. > > > Key: IGNITE-19643 > URL: https://issues.apache.org/jira/browse/IGNITE-19643 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Andrey Mashenkov >Priority: Minor > Labels: ignite-3, tech-debt > > Let's implement some validation rules that each Catalog command should check > before the execution. Then replace current boilerplate validation code with > applying these rules using Visitor pattern. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-21211) Catalog service advanced improvements
Roman Puchkovskiy created IGNITE-21211: -- Summary: Catalog service advanced improvements Key: IGNITE-21211 URL: https://issues.apache.org/jira/browse/IGNITE-21211 Project: Ignite Issue Type: Epic Reporter: Roman Puchkovskiy -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-21206) Lease grant messages should be retried after the failure of the placement driver active actor
[ https://issues.apache.org/jira/browse/IGNITE-21206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17804240#comment-17804240 ] Denis Chudov commented on IGNITE-21206: --- This should fix the flakiness of ItTableRaftSnapshotsTest.leaderFeedsFollowerWithSnapshot https://ci.ignite.apache.org/test/3905104914336486846?currentProjectId=ApacheIgnite3xGradle_Test_IntegrationTests=true > Lease grant messages should be retried after the failure of the placement > driver active actor > - > > Key: IGNITE-21206 > URL: https://issues.apache.org/jira/browse/IGNITE-21206 > Project: Ignite > Issue Type: Improvement >Reporter: Denis Chudov >Assignee: Denis Chudov >Priority: Major > Labels: ignite-3 > Time Spent: 10m > Remaining Estimate: 0h > > If the lease grant message response is sent to the node which is no longer > active, the corresponding lease (which expires after the long time interval > intended for lease acceptance) will not be negotiated again by a new active > actor until it expires. So we will not have the primary replica for the lease > acceptance interval, which is currently 2 minutes. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IGNITE-21206) Lease grant messages should be retried after the failure of the placement driver active actor
[ https://issues.apache.org/jira/browse/IGNITE-21206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denis Chudov reassigned IGNITE-21206: - Assignee: Denis Chudov > Lease grant messages should be retried after the failure of the placement > driver active actor > - > > Key: IGNITE-21206 > URL: https://issues.apache.org/jira/browse/IGNITE-21206 > Project: Ignite > Issue Type: Improvement >Reporter: Denis Chudov >Assignee: Denis Chudov >Priority: Major > Labels: ignite-3 > > If the lease grant message response is sent to the node which is no longer > active, the corresponding lease (which expires after the long time interval > intended for lease acceptance) will not be negotiated again by a new active > actor until it expires. So we will not have the primary replica for the lease > acceptance interval, which is currently 2 minutes. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-21210) Sql. Incorrect precision derivation for negative numeric types
Evgeny Stanilovsky created IGNITE-21210:
---
Summary: Sql. Incorrect precision derivation for negative numeric
types
Key: IGNITE-21210
URL: https://issues.apache.org/jira/browse/IGNITE-21210
Project: Ignite
Issue Type: Bug
Components: sql
Affects Versions: 3.0.0-beta1
Reporter: Evgeny Stanilovsky
Seems precision for negative numeric types derived erroneously.
{noformat}
@Test
public void testLiteralTypeMatch() throws Exception {
String query = "SELECT -1.1, DECIMAL '-1.1'";
IgniteRel rel = physicalPlan(query, new IgniteSchema(DEFAULT_SCHEMA, 1,
List.of()));
RelDataType numericLitType =
rel.getRowType().getFieldList().get(0).getType();
RelDataType decimalLitType =
rel.getRowType().getFieldList().get(1).getType();
assertEquals(numericLitType, decimalLitType);
}
{noformat}
throws on comparison:
{noformat}
Expected :DECIMAL(2, 1)
Actual :DECIMAL(3, 1)
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (IGNITE-21209) Do not try handle PrimaryReplicaMissException on transaction operation
[ https://issues.apache.org/jira/browse/IGNITE-21209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladislav Pyatkov reassigned IGNITE-21209: -- Assignee: Vladislav Pyatkov > Do not try handle PrimaryReplicaMissException on transaction operation > -- > > Key: IGNITE-21209 > URL: https://issues.apache.org/jira/browse/IGNITE-21209 > Project: Ignite > Issue Type: Improvement >Reporter: Vladislav Pyatkov >Assignee: Vladislav Pyatkov >Priority: Major > Labels: ignite-3 > > h3. Motivation > A transaction enlists a primary replica before making an operation request. > But the PrimaryReplicaMissException can be thrown after the request tries to > be sent. This means that any ties to remap the request cannot enlist a new > primary replica, and all this is meaningless. The code, which handles the > PrimaryReplicaMissException, just wastes time. > h3. Definition of done > Do not handle the PrimaryReplicaMissException on any transaction operation > after the primary replica is already enlisted. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-21209) Do not try handle PrimaryReplicaMissException on transaction operation
[ https://issues.apache.org/jira/browse/IGNITE-21209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladislav Pyatkov updated IGNITE-21209: --- Labels: ignite-3 (was: ) > Do not try handle PrimaryReplicaMissException on transaction operation > -- > > Key: IGNITE-21209 > URL: https://issues.apache.org/jira/browse/IGNITE-21209 > Project: Ignite > Issue Type: Improvement >Reporter: Vladislav Pyatkov >Priority: Major > Labels: ignite-3 > > h3. Motivation > A transaction enlists a primary replica before making an operation request. > But the PrimaryReplicaMissException can be thrown after the request tries to > be sent. This means that any ties to remap the request cannot enlist a new > primary replica, and all this is meaningless. The code, which handles the > PrimaryReplicaMissException, just wastes time. > h3. Definition of done > Do not handle the PrimaryReplicaMissException on any transaction operation > after the primary replica is already enlisted. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-21209) Do not try handle PrimaryReplicaMissException on transaction operation
Vladislav Pyatkov created IGNITE-21209: -- Summary: Do not try handle PrimaryReplicaMissException on transaction operation Key: IGNITE-21209 URL: https://issues.apache.org/jira/browse/IGNITE-21209 Project: Ignite Issue Type: Improvement Reporter: Vladislav Pyatkov h3. Motivation A transaction enlists a primary replica before making an operation request. But the PrimaryReplicaMissException can be thrown after the request tries to be sent. This means that any ties to remap the request cannot enlist a new primary replica, and all this is meaningless. The code, which handles the PrimaryReplicaMissException, just wastes time. h3. Definition of done Do not handle the PrimaryReplicaMissException on any transaction operation after the primary replica is already enlisted. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-20883) ItDataSchemaSyncTest.checkSchemasCorrectlyRestore() is flaky with The query was cancelled while executing
[
https://issues.apache.org/jira/browse/IGNITE-20883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17804234#comment-17804234
]
Vladislav Pyatkov commented on IGNITE-20883:
Merged e06d6142d32da1684c777a1e9b38b4d92db9ac91
> ItDataSchemaSyncTest.checkSchemasCorrectlyRestore() is flaky with The query
> was cancelled while executing
> -
>
> Key: IGNITE-20883
> URL: https://issues.apache.org/jira/browse/IGNITE-20883
> Project: Ignite
> Issue Type: Bug
>Reporter: Alexander Lapin
>Assignee: Vladislav Pyatkov
>Priority: Major
> Labels: ignite-3
> Fix For: 3.0.0-beta2
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
>
> {code:java}
> org.apache.ignite.sql.SqlException: IGN-SQL-8
> TraceId:bdc067e0-18f5-4c17-a3c7-9777e02a9abd The query was cancelled while
> executing. at
> java.base@11.0.17/java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:710)
> at
> app//org.apache.ignite.internal.util.ExceptionUtils$1.copy(ExceptionUtils.java:765)
> at
> app//org.apache.ignite.internal.util.ExceptionUtils$ExceptionFactory.createCopy(ExceptionUtils.java:699)
> at
> app//org.apache.ignite.internal.util.ExceptionUtils.copyExceptionWithCause(ExceptionUtils.java:536)
> at
> app//org.apache.ignite.internal.util.ExceptionUtils.copyExceptionWithCauseInternal(ExceptionUtils.java:634)
> at
> app//org.apache.ignite.internal.util.ExceptionUtils.copyExceptionWithCause(ExceptionUtils.java:487)
> at
> app//org.apache.ignite.internal.sql.AbstractSession.execute(AbstractSession.java:63)
> at
> app//org.apache.ignite.internal.runner.app.ItDataSchemaSyncTest.checkSchemasCorrectlyRestore(ItDataSchemaSyncTest.java:268)
> at
> java.base@11.0.17/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method) at
> java.base@11.0.17/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base@11.0.17/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base@11.0.17/java.lang.reflect.Method.invoke(Method.java:566) at
> app//org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:727)
> at
> app//org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
> at
> app//org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
> at
> app//org.junit.jupiter.engine.extension.SameThreadTimeoutInvocation.proceed(SameThreadTimeoutInvocation.java:45)
> at
> app//org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156)
> at
> app//org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:147)
> at
> app//org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:86)
> at
> app//org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103)
> at
> app//org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93)
> at
> app//org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
> at
> app//org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
> at
> app//org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
> at
> app//org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
> at
> app//org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:92)
> at
> app//org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:86)
> at
> app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:217)
> at
> app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
> at
> app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:213)
> at
> app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:138)
> at
> app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:68)
> at
> app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151)
> at
>
[jira] [Updated] (IGNITE-21199) Introduce a mechanism to manage and schedule Ignite services tasks
[ https://issues.apache.org/jira/browse/IGNITE-21199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Scherbakov updated IGNITE-21199: --- Description: We have service tasks such as GC, and we anticipate more, which consume resources shared with user tasks, such as queries. These tasks should not monopolize resources when a node is engaged in user queries but should utilize all resources during idle times. We need to develop a common mechanism to manage resource allocation between service tasks and user tasks, ensuring this component is reusable across different service task-creating components. The idea is to assign a weight to each user/system operation. The weight can be the approx number of dirty pages, produced by an operation. Before attempting an operation, a corresponding weight must be "reserved" in the total weight pool (configurable by a user) was: We have service tasks such as GC, and we anticipate more, which consume resources shared with user tasks, such as queries. These tasks should not monopolize resources when a node is engaged in user queries but should utilize all resources during idle times. We need to develop a common mechanism to manage resource allocation between service tasks and user tasks, ensuring this component is reusable across different service task-creating components. > Introduce a mechanism to manage and schedule Ignite services tasks > -- > > Key: IGNITE-21199 > URL: https://issues.apache.org/jira/browse/IGNITE-21199 > Project: Ignite > Issue Type: Improvement >Reporter: Ivan Gagarkin >Priority: Major > Labels: ignite-3 > > We have service tasks such as GC, and we anticipate more, which consume > resources shared with user tasks, such as queries. These tasks should not > monopolize resources when a node is engaged in user queries but should > utilize all resources during idle times. > We need to develop a common mechanism to manage resource allocation between > service tasks and user tasks, ensuring this component is reusable across > different service task-creating components. > The idea is to assign a weight to each user/system operation. The weight can > be the approx number of dirty pages, produced by an operation. Before > attempting an operation, a corresponding weight must be "reserved" in the > total weight pool (configurable by a user) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-21205) Don't store table versions in meta-storage in SchemaManager
[ https://issues.apache.org/jira/browse/IGNITE-21205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Bessonov updated IGNITE-21205: --- Description: Current implementation blocks meta-storage watch thread (doesn't allow new watch events to be processed, to be precise) until new schema version is written into the meta-storage. This is an expensive IO operation, and it might introduce unexpected turbulence. Some scenarios greatly suffer from it. For example, we can't process lease updates while we're writing into meta-storage, which shouldn't be the case. was: Current implementation blocks meta-storage watch thread until new schema version is written into the meta-storage. This is an expensive IO operation, and it might introduce unexpected turbulence. Some scenarios greatly suffer from it. For example, we can't process lease updates while we're writing into meta-storage, which shouldn't be the case. > Don't store table versions in meta-storage in SchemaManager > --- > > Key: IGNITE-21205 > URL: https://issues.apache.org/jira/browse/IGNITE-21205 > Project: Ignite > Issue Type: Improvement >Reporter: Ivan Bessonov >Assignee: Ivan Bessonov >Priority: Major > Labels: ignite-3 > Time Spent: 20m > Remaining Estimate: 0h > > Current implementation blocks meta-storage watch thread (doesn't allow new > watch events to be processed, to be precise) until new schema version is > written into the meta-storage. This is an expensive IO operation, and it > might introduce unexpected turbulence. > Some scenarios greatly suffer from it. For example, we can't process lease > updates while we're writing into meta-storage, which shouldn't be the case. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-21204) Use shared rocksdb instance for all TX state storages
[ https://issues.apache.org/jira/browse/IGNITE-21204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17804171#comment-17804171 ] Kirill Tkalenko commented on IGNITE-21204: -- Looks good. > Use shared rocksdb instance for all TX state storages > - > > Key: IGNITE-21204 > URL: https://issues.apache.org/jira/browse/IGNITE-21204 > Project: Ignite > Issue Type: Improvement >Reporter: Ivan Bessonov >Assignee: Ivan Bessonov >Priority: Major > Labels: ignite-3 > Time Spent: 1h 40m > Remaining Estimate: 0h > > Current implementation uses too many resources if you create multiple tables. > Table creation time suffers too. > We need to use the same approach as "rocksdb" storage engine uses. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-21208) Sql. DEFAULT expression with NULL value processed unexpectedly
Evgeny Stanilovsky created IGNITE-21208:
---
Summary: Sql. DEFAULT expression with NULL value processed
unexpectedly
Key: IGNITE-21208
URL: https://issues.apache.org/jira/browse/IGNITE-21208
Project: Ignite
Issue Type: Bug
Components: sql
Affects Versions: 3.0.0-beta1
Reporter: Evgeny Stanilovsky
Correct processing for NULL consistent expressions require additional
overriding of IgniteSqlToRelConvertor#convertValues, seems we can avoid it.
Expressions need to be passed without such overriding method:
{noformat}
create table t (id int, c int DEFAULT null);
insert into t(id, c) values(1, DEFAULT);
insert into t(id) values(2);
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
