[jira] [Updated] (IGNITE-21113) Tests involving Storage functionality fail on RocksDB Storage Engine
[
https://issues.apache.org/jira/browse/IGNITE-21113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Chugunov updated IGNITE-21113:
-
Attachment: _Integration_Tests_Module_SQL_Engine_424.log.zip
Description:
When IGNITE-21048 was implemented two more PRs (one for [Volatile
Storage|https://github.com/apache/ignite-3/pull/2953], one for
[RocksDB-based|https://github.com/apache/ignite-3/pull/2952]) were open to run
existing tests against different Storage Engines.
Both of them hung.
In the second run for RocksDB Storage several tests fail because of lack of
configuration for RocksDB Storage Engine:
* ItRebalanceDistributedTest
* ItSecondaryIndexTest.
The list above may be incomplete, there could be tests that reveal actual
problems with implementation rather than issues with tests themselves.
Specifically, some tests from ItSecondaryIndexTest fail with the following
exception in logs:
{code:java}
java.nio.BufferUnderflowException
at java.base/java.nio.Buffer.nextGetIndex(Buffer.java:651)
at java.base/java.nio.DirectByteBuffer.getShort(DirectByteBuffer.java:613)
at
org.apache.ignite.internal.storage.rocksdb.index.RocksDbBinaryTupleComparator.compare(RocksDbBinaryTupleComparator.java:67)
at
org.rocksdb.AbstractComparatorJniBridge.compareInternal(AbstractComparatorJniBridge.java:51)
at org.rocksdb.RocksDB.iteratorCF(Native Method)
at org.rocksdb.RocksDB.newIterator(RocksDB.java:2914)
at
org.apache.ignite.internal.rocksdb.ColumnFamily.newIterator(ColumnFamily.java:187)
at
org.apache.ignite.internal.storage.rocksdb.index.AbstractRocksDbIndexStorage$UpToDatePeekCursor.(AbstractRocksDbIndexStorage.java:246)
at
org.apache.ignite.internal.storage.rocksdb.index.RocksDbSortedIndexStorage$1.(RocksDbSortedIndexStorage.java:187)
at
org.apache.ignite.internal.storage.rocksdb.index.RocksDbSortedIndexStorage.scan(RocksDbSortedIndexStorage.java:187)
at
org.apache.ignite.internal.storage.rocksdb.index.RocksDbSortedIndexStorage.lambda$scan$3(RocksDbSortedIndexStorage.java:150)
at
org.apache.ignite.internal.storage.rocksdb.index.AbstractRocksDbIndexStorage.busy(AbstractRocksDbIndexStorage.java:201)
at
org.apache.ignite.internal.storage.rocksdb.index.RocksDbSortedIndexStorage.scan(RocksDbSortedIndexStorage.java:144)
at
org.apache.ignite.internal.table.distributed.replicator.PartitionReplicaListener.lambda$scanSortedIndex$59(PartitionReplicaListener.java:1246)
at
java.base/java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1461)
at
org.apache.ignite.internal.table.distributed.replicator.PartitionReplicaListener.scanSortedIndex(PartitionReplicaListener.java:1245)
at
org.apache.ignite.internal.table.distributed.replicator.PartitionReplicaListener.lambda$processReadOnlyScanRetrieveBatchAction$32(PartitionReplicaListener.java:794)
...
at
org.apache.ignite.internal.table.distributed.replicator.PartitionReplicaListener.invoke(PartitionReplicaListener.java:465)
at
org.apache.ignite.internal.replicator.Replica.processRequest(Replica.java:139)
at
org.apache.ignite.internal.replicator.ReplicaManager.onReplicaMessageReceived(ReplicaManager.java:293)
at
org.apache.ignite.network.DefaultMessagingService.sendToSelf(DefaultMessagingService.java:319)
...
at
org.apache.ignite.internal.replicator.ReplicaService.sendToReplica(ReplicaService.java:92)
at
org.apache.ignite.internal.replicator.ReplicaService.invoke(ReplicaService.java:195)
at
org.apache.ignite.internal.table.distributed.storage.InternalTableImpl.lambda$scan$54(InternalTableImpl.java:1324)
at
org.apache.ignite.internal.table.distributed.storage.InternalTableImpl$PartitionScanPublisher$PartitionScanSubscription.scanBatch(InternalTableImpl.java:1856)
...
at
org.apache.ignite.internal.util.subscription.ConcatenatedPublisher.subscribe(ConcatenatedPublisher.java:53)
at
org.apache.ignite.internal.sql.engine.exec.rel.StorageScanNode.requestNextBatch(StorageScanNode.java:186)
at
org.apache.ignite.internal.sql.engine.exec.rel.StorageScanNode.push(StorageScanNode.java:158)
at
org.apache.ignite.internal.sql.engine.exec.ExecutionContext.lambda$execute$0(ExecutionContext.java:309)
at
org.apache.ignite.internal.sql.engine.exec.QueryTaskExecutorImpl.lambda$execute$0(QueryTaskExecutorImpl.java:81)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834) {code}
Full logs are attached to the ticket.
was:
When IGNITE-21048 was implemented two more PRs (one for [Volatile
Storage|https://github.com/apache/ignite-3/pull/2953], one for
[RocksDB-based|https://github.com/apache/ignite-3/pull/2952]) were open to run
existing tests against
[jira] [Created] (IGNITE-21113) Tests involving Storage functionality fail on RocksDB Storage Engine
Sergey Chugunov created IGNITE-21113: Summary: Tests involving Storage functionality fail on RocksDB Storage Engine Key: IGNITE-21113 URL: https://issues.apache.org/jira/browse/IGNITE-21113 Project: Ignite Issue Type: Bug Components: persistence Reporter: Sergey Chugunov Fix For: 3.0.0-beta2 When IGNITE-21048 was implemented two more PRs (one for [Volatile Storage|https://github.com/apache/ignite-3/pull/2953], one for [RocksDB-based|https://github.com/apache/ignite-3/pull/2952]) were open to run existing tests against different Storage Engines. Both of them hung. In the second run for RocksDB Storage several tests fail because of lack of configuration for RocksDB Storage Engine: * ItRebalanceDistributedTest * ItSecondaryIndexTest. The list above may be incomplete, there could be tests that reveal actual problems with implementation rather than issues with tests themselves. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-21098) Increase additional wait after DDL by MaxClockSkew
[ https://issues.apache.org/jira/browse/IGNITE-21098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17798489#comment-17798489 ] Sergey Chugunov commented on IGNITE-21098: -- [~rpuch] , patch looks good to me, I merged it to main branch. Thank you for contribution! > Increase additional wait after DDL by MaxClockSkew > -- > > Key: IGNITE-21098 > URL: https://issues.apache.org/jira/browse/IGNITE-21098 > Project: Ignite > Issue Type: Improvement >Reporter: Roman Puchkovskiy >Assignee: Roman Puchkovskiy >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > Time Spent: 10m > Remaining Estimate: 0h > > When executing an RO transaction, readTimestamp has to be determined. If > MaxObservableTimestamp is not available (which is currently true after a > DDL), then TxManagerImpl takes readTimestamp as HLC.now() - > idleSafeTimePropagationPeriod - MaxClockSkew. > In IGNITE-20668, an additional wait after a DDL operation was added to make > sure that a readTimestamp calculated in this way does not look before the DDL > (otherwise, the RO would see the previous schema). But in IGNITE-20668, only > idleSafeTimePropagationPeriod was added after a DDL, and MaxClockSkew > component was forgotten. > We must add MaxClockSkew to the wait. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-21104) Tests involving Storage functionality fail on Volatile Storage Engine
Sergey Chugunov created IGNITE-21104:
Summary: Tests involving Storage functionality fail on Volatile
Storage Engine
Key: IGNITE-21104
URL: https://issues.apache.org/jira/browse/IGNITE-21104
Project: Ignite
Issue Type: Bug
Components: persistence
Reporter: Sergey Chugunov
Fix For: 3.0.0-beta2
When IGNITE-21048 was implemented two more PRs (one for [Volatile
Storage|https://github.com/apache/ignite-3/pull/2953], one for
[RocksDB-based|https://github.com/apache/ignite-3/pull/2952]) were open to run
existing tests against different Storage Engines.
Both of them hung.
In the first run for Volatile Storage at least two tests were identified that
hang with AssertionError:
* ItBuildIndexTest;
*
ItInternalTableTest.
Both tests have this assertion in logs:
{code:java}
Caused by: java.lang.AssertionError
at
org.apache.ignite.internal.pagememory.tree.BplusTree$InitRoot.run(BplusTree.java:916)
~[main/:?]
at
org.apache.ignite.internal.pagememory.tree.BplusTree$InitRoot.run(BplusTree.java:896)
~[main/:?]
at
org.apache.ignite.internal.pagememory.util.PageHandler.writePage(PageHandler.java:298)
~[main/:?]
at
org.apache.ignite.internal.pagememory.datastructure.DataStructure.write(DataStructure.java:369)
~[main/:?]
at
org.apache.ignite.internal.pagememory.tree.BplusTree.initTree(BplusTree.java:1045)
~[main/:?]
at
org.apache.ignite.internal.storage.pagememory.mv.VersionChainTree.(VersionChainTree.java:76)
~[main/:?]
at
org.apache.ignite.internal.storage.pagememory.VolatilePageMemoryTableStorage.createVersionChainTree(VolatilePageMemoryTableStorage.java:156)
~[main/:?]
at
org.apache.ignite.internal.storage.pagememory.VolatilePageMemoryTableStorage.createMvPartitionStorage(VolatilePageMemoryTableStorage.java:72)
~[main/:?]
at
org.apache.ignite.internal.storage.pagememory.VolatilePageMemoryTableStorage.createMvPartitionStorage(VolatilePageMemoryTableStorage.java:40)
~[main/:?]
at
org.apache.ignite.internal.storage.pagememory.AbstractPageMemoryTableStorage.lambda$createMvPartition$4(AbstractPageMemoryTableStorage.java:164)
~[main/:?]
at
org.apache.ignite.internal.storage.util.MvPartitionStorages.lambda$create$1(MvPartitionStorages.java:121)
~[main/:?]
at
java.util.concurrent.CompletableFuture.uniApplyNow(CompletableFuture.java:680)
~[?:?]
... 39 more {code}
This behavior can be reproduced locally with 100% fail rate.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-21048) A way to change default storage engine in tests
Sergey Chugunov created IGNITE-21048: Summary: A way to change default storage engine in tests Key: IGNITE-21048 URL: https://issues.apache.org/jira/browse/IGNITE-21048 Project: Ignite Issue Type: Improvement Reporter: Sergey Chugunov Assignee: Sergey Chugunov Fix For: 3.0.0-beta2 With the current implementation of CatalogManager default Storage Engine is hardcoded to AIPERSIST (PageMemory implementation with persistence) and there is no way to configure it to something else. There is an idea to integrate specification of Default Distribution Zone configuration into init script, but it is not implemented yet. So the only way to specify a different engine and test against it is to introduce a system property that could change defaults. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-20852) Opposite connection attempts may cause connection failure
[
https://issues.apache.org/jira/browse/IGNITE-20852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17789145#comment-17789145
]
Sergey Chugunov commented on IGNITE-20852:
--
[~rpuch] , I reviewed your patch, it looks good to me so I merged it to main
branch in commit {*}29dedf63f6699bf9705874de7b9e990b4b0ecddc{*}.
Thank you for contribution!
> Opposite connection attempts may cause connection failure
> -
>
> Key: IGNITE-20852
> URL: https://issues.apache.org/jira/browse/IGNITE-20852
> Project: Ignite
> Issue Type: Bug
>Reporter: Roman Puchkovskiy
>Assignee: Roman Puchkovskiy
>Priority: Major
> Labels: ignite-3
> Fix For: 3.0.0-beta2
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Sometimes 2 nodes (A and B) try to establish logical connections to each
> other at the same time. Only one of the connections can remain, so another
> one is aborted. When such an abortion happens, the client that initiated the
> aborted physical connection must be transparently switched to the connection
> opened 'from the other side'.
> A subsituation where each of the competitors has already acquired a lock (at
> different sides of the connection) is called a clinch. In other cases, one
> node manages to take both locks.
> In most of such situations, MessagingService gets an exception on one of the
> sides. We must avoid this.
> Another problem is that the clinch resulution protocol sometimes kills an
> 'almost established connection' (or even a fully estabished connection) [that
> has managed to aquire recovery descriptors on both sides] just because
> another connection (that has just failed to aquire first recovery descriptor)
> is preferred by the tie-breaker. The tie-breaker must only be used in true
> clinch situations (each of the competitors has 1 descriptor aquired).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-20908) Describe network handshake protocol in the README
[ https://issues.apache.org/jira/browse/IGNITE-20908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17789106#comment-17789106 ] Sergey Chugunov commented on IGNITE-20908: -- The other place to improve our dev documentation is javadoc for RecoveryClientHandshakeManager and RecoveryServerHandshakeManager. Right now it is not clear from the javadocs what their duties are and what "Client" and "Server" parts of their names even mean. > Describe network handshake protocol in the README > - > > Key: IGNITE-20908 > URL: https://issues.apache.org/jira/browse/IGNITE-20908 > Project: Ignite > Issue Type: Improvement >Reporter: Roman Puchkovskiy >Priority: Major > Labels: ignite-3 > > We need to describe the network handshake protocol (implemented in > ConnectionManager) in a README file to make it easier to understand the code. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20132) Do not allow an indexed column to be dropped
[ https://issues.apache.org/jira/browse/IGNITE-20132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-20132: - Epic Link: IGNITE-20782 (was: IGNITE-17766) > Do not allow an indexed column to be dropped > > > Key: IGNITE-20132 > URL: https://issues.apache.org/jira/browse/IGNITE-20132 > Project: Ignite > Issue Type: Improvement >Reporter: Roman Puchkovskiy >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > If a column is included in some index which state is STARTING/READY, an > attempt to drop such a column must be rejected -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20525) Fix ItBuildIndexTest#testChangePrimaryReplicaOnMiddleBuildIndex
[ https://issues.apache.org/jira/browse/IGNITE-20525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-20525: - Epic Link: IGNITE-20782 (was: IGNITE-17766) > Fix ItBuildIndexTest#testChangePrimaryReplicaOnMiddleBuildIndex > --- > > Key: IGNITE-20525 > URL: https://issues.apache.org/jira/browse/IGNITE-20525 > Project: Ignite > Issue Type: Improvement >Reporter: Kirill Tkalenko >Priority: Major > Labels: ignite-3 > > At the moment, > *org.apache.ignite.internal.sql.engine.ItBuildIndexTest#testChangePrimaryReplicaOnMiddleBuildIndex* > is not easy to fix, since we do not have the ability to change the > leaseholder in the middle of building the index, such an opportunity should > appear IGNITE-20365. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-18595) Implement index build process during the full state transfer
[
https://issues.apache.org/jira/browse/IGNITE-18595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Chugunov updated IGNITE-18595:
-
Epic Link: IGNITE-20782 (was: IGNITE-17766)
> Implement index build process during the full state transfer
>
>
> Key: IGNITE-18595
> URL: https://issues.apache.org/jira/browse/IGNITE-18595
> Project: Ignite
> Issue Type: Improvement
>Reporter: Ivan Bessonov
>Priority: Major
> Labels: ignite-3
>
> Here there is no source of information for schema versions, associated with
> individual inserts. The core idea of the full rebalance is that all versions
> of all rows will be sent, while indexes will be rebuilt locally on the
> consumer. This is unfortunate. Why, you may ask.
> Imagine the following situation:
> * time T1: table A with index X is created
> * time T2: user uploads the data
> * time T3: user drops index X
> * time T4: “clean” node N enters topology and downloads data via full
> rebalance procedure
> * time T5: N becomes a leader and receives (already running) RO transactions
> with timestamp T2 Ideally, index X should be available for timestamp T. If the index is already
> available, it can’t suddenly become unavailable without an explicit rebuild
> request from the user (I guess).
> The LATEST schema version at the moment of rebalance must be known. That’s
> unavoidable and makes total sense. First idea that comes to mind is updating
> all Registered and Available indexes. Situation, when an index has more
> indexed rows than it requires, is correct. Scan queries only return indexed
> rows that match corresponding value in the partition MV store. The real
> problem would be having less data than required.
> The way that the approach is described in paragraph above is not quite
> correct. Let’s consider that there is a BinaryRow version. It defines a set
> of columns in the table at the moment of update. Not all row versions are
> compatible with all indexes. For example, you cannot put data into an index
> if a column has been deleted. On the other hand, you can put data in the
> index if a column has not yet been created (assuming it has a default value).
> In both cases the column is missing from the row version, but the outcome is
> very different.
> This fact has some implications. A set of indexes to be updated depends on
> the row version for every particular row. I propose calculating it as a set
> of all indexes from a {_}maximal continuous range of db schemas{_}, that (if
> not empty) starts with the earliest known schema and _all schemas in the
> range have all indexed columns_ existing in the table.
> For example, there’s a table T:
> |DB schema version|Table columns|
> |1|PK, A|
> |2|PK, A, B|
> |3 (LATEST)|PK, B|
>
> In such configuration, ranges would be:
> |Index columns|Schemas range|
> |A|[1 ... 2]|
> |B|[1 ... 3]|
> |A, B|[1 ... 2]|
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-20117) Implement index backfill process
[ https://issues.apache.org/jira/browse/IGNITE-20117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-20117: - Epic Link: IGNITE-20782 (was: IGNITE-17766) > Implement index backfill process > > > Key: IGNITE-20117 > URL: https://issues.apache.org/jira/browse/IGNITE-20117 > Project: Ignite > Issue Type: Improvement >Reporter: Roman Puchkovskiy >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > Currently, we have backfill process for an index (aka 'index build'). It > needs to be tuned to satisfy the following requirements: > # When starting the backfill process, we must first wait till > safeTime(partition)>=’STARTING state activation timestamp’ to avoid a race > between starting the backfill process and executing writes that are before > the index creation (as these writes should not write to the index). > # If for a row found during the backfill process, there are row versions with > commitTs <= indexCreationActivationTs, then the most recent of them is > written to the index > # If for a row found during the backfill process, there are row versions with > commitTs > indexCreationActivationTs, then the oldest of them is added to the > index; otherwise, if there are no such row versions, but there is a write > intent (and the transaction to which it belongs started before > indexCreationActivationTs), it is added to the index > # When the backfill process is finished on all partitions, another schema > update is installed that declares that the index is in the READY state. This > installation should be conditional. That is, if the index is still STARTING, > it should succeed; otherwise (if the index was removed by installing a > concurrent ‘delete from the Catalog’ schema update due to a DROP command), > nothing should be done here > # The backfill process stops early as soon as it detects that the index moved > to ‘deleted from the Catalog’ state. Each step of the process might be > supplied with a timestamp (from the same clock that moves the partition’s > SafeTime ahead) and that timestamp could be used to check the index > existence; this will allow to avoid a race between index destruction and the > backfill process. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20125) Write to write-compatible indices when writing to partition
[ https://issues.apache.org/jira/browse/IGNITE-20125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-20125: - Epic Link: IGNITE-20782 (was: IGNITE-17766) > Write to write-compatible indices when writing to partition > --- > > Key: IGNITE-20125 > URL: https://issues.apache.org/jira/browse/IGNITE-20125 > Project: Ignite > Issue Type: Improvement >Reporter: Roman Puchkovskiy >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > For each operation that is writing, the operation’s timestamp (which moves > partition’s SafeTime forward) T~op~ is used to get the schema corresponding > to the operation. When it’s obtained, all writable (STARTING, READY, > STOPPING) indices that are write-compatible at T~op~ are taken, and the > current operation writes to them all. > An index is write-compatible at timestamp T~op~ if for each column of the > index the following holds: the column was not dropped at all, or it was > dropped strictly after T~op~. > If an index does not exist anymore at T~op~, the write to it is just ignored, > the transaction is not aborted. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20133) Compute hashes for integral/decimal columns in a stable way
[
https://issues.apache.org/jira/browse/IGNITE-20133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Chugunov updated IGNITE-20133:
-
Epic Link: IGNITE-20782 (was: IGNITE-17766)
> Compute hashes for integral/decimal columns in a stable way
> ---
>
> Key: IGNITE-20133
> URL: https://issues.apache.org/jira/browse/IGNITE-20133
> Project: Ignite
> Issue Type: Improvement
>Reporter: Roman Puchkovskiy
>Priority: Minor
> Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>
> The idea is to make hash computation for integral and decimal types satisfy
> the following property: if a column type is changed from an integral to a
> decimal type, the hashes for values that are already stored remain the same.
> This will allow us to permit chaning type (integral -> decimal and decimal ->
> longer decimal) of a column that is included in a HASH index.
> A hash that has this property is the following function:
> hash(val.toString(TRIM_TRAILING_ZEROS)). For instance, for 1 it will be
> hash("1"), for 1.000 it will also be hash("1"), but for 1.23 it will give
> hash("1.23").
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-19150) Implement proper destruction of indexes
[ https://issues.apache.org/jira/browse/IGNITE-19150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-19150: - Epic Link: IGNITE-20782 (was: IGNITE-17766) > Implement proper destruction of indexes > --- > > Key: IGNITE-19150 > URL: https://issues.apache.org/jira/browse/IGNITE-19150 > Project: Ignite > Issue Type: Improvement >Reporter: Kirill Tkalenko >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > At the moment, we do not honestly destroy indexes anywhere in the core code, > we simply remove them from the collection of indexes when inserting data. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20126) Check that the index exists before reading from it
[ https://issues.apache.org/jira/browse/IGNITE-20126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-20126: - Epic Link: IGNITE-20782 (was: IGNITE-17766) > Check that the index exists before reading from it > -- > > Key: IGNITE-20126 > URL: https://issues.apache.org/jira/browse/IGNITE-20126 > Project: Ignite > Issue Type: Improvement >Reporter: Roman Puchkovskiy >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > Before each read-from-index operation (like opening a scan cursor or getting > next batch of such a cursor), we should check whether the index exists (i.e. > not removed from the Catalog) at moment opTs. If not, the operation must be > failed and the current RW tx must be aborted. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20131) Consider all index-related schema changes as compatible
[ https://issues.apache.org/jira/browse/IGNITE-20131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-20131: - Epic Link: IGNITE-20782 (was: IGNITE-17766) > Consider all index-related schema changes as compatible > --- > > Key: IGNITE-20131 > URL: https://issues.apache.org/jira/browse/IGNITE-20131 > Project: Ignite > Issue Type: Improvement >Reporter: Roman Puchkovskiy >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > All allowed index state transfers are compatible: > * Creation (as STARTING) > * STARTING -> READY > * READY -> STOPPING > * STOPPING -> [not-existent] > * STARTING -> [not-existent] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19202) Creating index storage for partitions
[ https://issues.apache.org/jira/browse/IGNITE-19202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-19202: - Epic Link: IGNITE-20782 (was: IGNITE-17766) > Creating index storage for partitions > - > > Key: IGNITE-19202 > URL: https://issues.apache.org/jira/browse/IGNITE-19202 > Project: Ignite > Issue Type: Improvement >Reporter: Kirill Tkalenko >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > At the moment, index storages are created by factories that are added when > listening to add an index to the configuration. > What is not entirely correct, we need to directly create index storages for > partitions. > Of course, we need to take into account the node assignments for the > partition. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20134) Only allow changing type of indexed column when indexed values representation remains the same
[ https://issues.apache.org/jira/browse/IGNITE-20134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-20134: - Epic Link: IGNITE-20782 (was: IGNITE-17766) > Only allow changing type of indexed column when indexed values representation > remains the same > -- > > Key: IGNITE-20134 > URL: https://issues.apache.org/jira/browse/IGNITE-20134 > Project: Ignite > Issue Type: Improvement >Reporter: Roman Puchkovskiy >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > When an attempt to change type of a column that is included in an index is > made, this should only be permitted if the representation of the column > values in the index will remain unchanged (and, hence, index rebuild will not > be needed). > The following changes are acceptable: > * integral->integral (as integral types are represented as varints) > * integral->decimal (with enough precision) and float->double for SORTED > indices where the ordering remains the same > * integral->decimal and decimal->decimal (with enough precision) for HASH > indices (requires IGNITE-20133) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-20782) Ignite 3.0: Dynamically created indexes integration and improvements
Sergey Chugunov created IGNITE-20782: Summary: Ignite 3.0: Dynamically created indexes integration and improvements Key: IGNITE-20782 URL: https://issues.apache.org/jira/browse/IGNITE-20782 Project: Ignite Issue Type: Epic Components: persistence Reporter: Sergey Chugunov -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17766) Ignite 3.0: Dynamically created SQL indexes lifecycle and management, basic cases
[ https://issues.apache.org/jira/browse/IGNITE-17766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-17766: - Summary: Ignite 3.0: Dynamically created SQL indexes lifecycle and management, basic cases (was: Ignite 3.0: Dynamically created SQL indexes lifecycle and management) > Ignite 3.0: Dynamically created SQL indexes lifecycle and management, basic > cases > - > > Key: IGNITE-17766 > URL: https://issues.apache.org/jira/browse/IGNITE-17766 > Project: Ignite > Issue Type: Epic > Components: persistence >Reporter: Sergey Chugunov >Priority: Major > Labels: ignite-3 > > Executing dynamic index management with CREATE/DROP INDEX commands requires > specific management of index lifecycle. > It includes at least: > # Triggering necessary operation on affinity nodes (nodes hosting data of a > table with new index). > # Adding information about new index to configuration. > # Monitoring process of index creation and updating index status. > # Providing notification mechanism to SQL engine so index will be used if and > only if it is fully built and ready to be used. > # Cancelling operation and cleaning stuff up in case user drops index. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20122) When a STARTING index is dropped, it should be removed right away
[ https://issues.apache.org/jira/browse/IGNITE-20122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-20122: - Epic Link: IGNITE-20782 (was: IGNITE-17766) > When a STARTING index is dropped, it should be removed right away > - > > Key: IGNITE-20122 > URL: https://issues.apache.org/jira/browse/IGNITE-20122 > Project: Ignite > Issue Type: Improvement >Reporter: Roman Puchkovskiy >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > When a DROP INDEX is executed for a an index that is STARTING, it should be > removed right away (skipping the READY and STOPPING states) and start > destruction. This should be done using a conditional schema update > (IGNITE-20115) to avoid a race with switching to the READY state. > If the conditional schema update fails (because the index has been switched > to the READY state), we should fallback to the usual procedure (IGNITE-20119). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20121) Start index destruction when it is removed from the Catalog
[ https://issues.apache.org/jira/browse/IGNITE-20121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-20121: - Epic Link: IGNITE-20782 (was: IGNITE-17766) > Start index destruction when it is removed from the Catalog > --- > > Key: IGNITE-20121 > URL: https://issues.apache.org/jira/browse/IGNITE-20121 > Project: Ignite > Issue Type: Improvement >Reporter: Roman Puchkovskiy >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > Index destruction, before starting, must wait until the partition’s SafeTime > becomes >= ‘Activation moment of index removal’ (aka ‘end time of STOPPING > state’ for this index). This is to avoid a race between operations on the > index (including writes and reads) and its destruction. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20120) Remove index from Catalog when activation of its STOPPING state is below GC LWM
[ https://issues.apache.org/jira/browse/IGNITE-20120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-20120: - Epic Link: IGNITE-20782 (was: IGNITE-17766) > Remove index from Catalog when activation of its STOPPING state is below GC > LWM > --- > > Key: IGNITE-20120 > URL: https://issues.apache.org/jira/browse/IGNITE-20120 > Project: Ignite > Issue Type: Improvement >Reporter: Roman Puchkovskiy >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > When activation moment of the STOPPING state of an index is below GC LWM > (meaning that no new transaction can read from this index), the index should > be removed from the Catalog. A conditional schema update (IGNITE-20115) might > be used. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20119) Switch index to STOPPING state as a reaction to DROP INDEX
[ https://issues.apache.org/jira/browse/IGNITE-20119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-20119: - Epic Link: IGNITE-20782 (was: IGNITE-17766) > Switch index to STOPPING state as a reaction to DROP INDEX > -- > > Key: IGNITE-20119 > URL: https://issues.apache.org/jira/browse/IGNITE-20119 > Project: Ignite > Issue Type: Improvement >Reporter: Roman Puchkovskiy >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20118) Create an index in STARTING state
[ https://issues.apache.org/jira/browse/IGNITE-20118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-20118: - Epic Link: IGNITE-20782 (was: IGNITE-17766) > Create an index in STARTING state > - > > Key: IGNITE-20118 > URL: https://issues.apache.org/jira/browse/IGNITE-20118 > Project: Ignite > Issue Type: Improvement >Reporter: Roman Puchkovskiy >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > The 'state' field should be added (probably, instead of 'writeOnly' flag). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19513) Must wait for indexes to be created before rebalancing
[ https://issues.apache.org/jira/browse/IGNITE-19513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-19513: - Epic Link: IGNITE-20782 (was: IGNITE-17766) > Must wait for indexes to be created before rebalancing > -- > > Key: IGNITE-19513 > URL: https://issues.apache.org/jira/browse/IGNITE-19513 > Project: Ignite > Issue Type: Improvement >Reporter: Kirill Tkalenko >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > Now, to prevent a race between the creation of indexes and the start of the > rebalance, an explicit (crutch) wait for the creation of indexes has been > added, which looks terrible. > We must organize the code so that at the time the rebalance starts, all > previous indexes are created and no new indexes appear during the rebalance. > See TODO. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20115) Support conditional schema update install
[ https://issues.apache.org/jira/browse/IGNITE-20115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-20115: - Epic Link: IGNITE-20782 (was: IGNITE-17766) > Support conditional schema update install > - > > Key: IGNITE-20115 > URL: https://issues.apache.org/jira/browse/IGNITE-20115 > Project: Ignite > Issue Type: Improvement >Reporter: Roman Puchkovskiy >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > Currently, the Catalog allows to install a schema update, but it's > unconditional. > We'll need a way to install schema updates conditionally, like this: 'if the > latest index state is X, install a schema update chaning it to Y, otherwise > do nothing and return false'. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19717) Get rid of creating indexes at the start of a MvTableStorage
[ https://issues.apache.org/jira/browse/IGNITE-19717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-19717: - Epic Link: IGNITE-20782 (was: IGNITE-17766) > Get rid of creating indexes at the start of a MvTableStorage > > > Key: IGNITE-19717 > URL: https://issues.apache.org/jira/browse/IGNITE-19717 > Project: Ignite > Issue Type: Improvement >Reporter: Kirill Tkalenko >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > Now, at the start of the *MvTableStorage*, we create indexes, which is not > correct, this should be done by external code that should know about all > indexes. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19712) Handle rebalance wrt indexes
[ https://issues.apache.org/jira/browse/IGNITE-19712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-19712: - Epic Link: IGNITE-20782 (was: IGNITE-17766) > Handle rebalance wrt indexes > > > Key: IGNITE-19712 > URL: https://issues.apache.org/jira/browse/IGNITE-19712 > Project: Ignite > Issue Type: Bug >Reporter: Semyon Danilov >Priority: Major > Labels: ignite-3 > > After IGNITE-19363, index storages are no longer lazily instantiated. Need to > listen to assignment changes and start new storages -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19498) Start building indexes only once
[ https://issues.apache.org/jira/browse/IGNITE-19498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-19498: - Epic Link: IGNITE-20782 (was: IGNITE-17766) > Start building indexes only once > > > Key: IGNITE-19498 > URL: https://issues.apache.org/jira/browse/IGNITE-19498 > Project: Ignite > Issue Type: Improvement >Reporter: Kirill Tkalenko >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > As part of IGNITE-19177, the building of indexes moves from the > *IndexManager* to the replica, because of this, a race may occur when the > replica becomes the leader, and we need to start listening to the creation of > indexes in the configuration and reading previous indexes from the > configuration. > Therefore, we need to figure out how we can avoid the double start of index > building, maybe this is not a problem and later we will just close the ticket. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19112) Get rid of MvTableStorage#getOrCreateIndex
[ https://issues.apache.org/jira/browse/IGNITE-19112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-19112: - Epic Link: IGNITE-20782 (was: IGNITE-17766) > Get rid of MvTableStorage#getOrCreateIndex > -- > > Key: IGNITE-19112 > URL: https://issues.apache.org/jira/browse/IGNITE-19112 > Project: Ignite > Issue Type: Improvement >Reporter: Kirill Tkalenko >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > Get rid of methods: > * org.apache.ignite.internal.storage.engine.MvTableStorage#getOrCreateIndex > * > org.apache.ignite.internal.storage.engine.MvTableStorage#getOrCreateHashIndex > * > org.apache.ignite.internal.storage.engine.MvTableStorage#getOrCreateSortedIndex > and make new ones: > * org.apache.ignite.internal.storage.engine.MvTableStorage#createIndex > * org.apache.ignite.internal.storage.engine.MvTableStorage#getIndex > * org.apache.ignite.internal.storage.engine.MvTableStorage#createHashIndex > * org.apache.ignite.internal.storage.engine.MvTableStorage#getHashIndex > * org.apache.ignite.internal.storage.engine.MvTableStorage#createSortedIndex > * org.apache.ignite.internal.storage.engine.MvTableStorage#getSortedIndex -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-20774) ItRebalanceDistributedTest#testRebalanceRetryWhenCatchupFailed is flaky on TC
Sergey Chugunov created IGNITE-20774:
Summary:
ItRebalanceDistributedTest#testRebalanceRetryWhenCatchupFailed is flaky on TC
Key: IGNITE-20774
URL: https://issues.apache.org/jira/browse/IGNITE-20774
Project: Ignite
Issue Type: Bug
Reporter: Sergey Chugunov
Attachments: _Integration_Tests_Module_Runner_18723.log.zip
Test fails with timeout in different branches including main:
{code:java}
java.util.concurrent.TimeoutException: after() timed out after 60 seconds
at
org.junit.jupiter.engine.extension.TimeoutExceptionFactory.create(TimeoutExceptionFactory.java:29)
at
org.junit.jupiter.engine.extension.SameThreadTimeoutInvocation.proceed(SameThreadTimeoutInvocation.java:58)
at
org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156)
at
org.junit.jupiter.engine.extension.TimeoutExtension.interceptLifecycleMethod(TimeoutExtension.java:128)
at
org.junit.jupiter.engine.extension.TimeoutExtension.interceptAfterEachMethod(TimeoutExtension.java:110)
at
org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103)
...
{code}
>From stack trace it is not clear where test timed out.
TC run with failed test in main is available
[here|https://ci.ignite.apache.org/viewLog.html?buildId=7593604&tab=buildResultsDiv&buildTypeId=ApacheIgnite3xGradle_Test_IntegrationTests_ModuleRunner]
test logs are attached.
Also TC reports NullPointerException in logs, it is not clear whether timeout
is caused but this NPE:
{code:java}
[2023-10-29T00:37:45,888][ERROR][%irdt_trrwcf_2%JRaft-FSMCaller-Disruptor-_stripe_4-0][StripedDisruptor]
Handle disruptor event error
[name=%irdt_trrwcf_2%JRaft-FSMCaller-Disruptor-,
event=org.apache.ignite.raft.jraft.core.FSMCallerImpl$ApplyTask@55293f4b,
hasHandler=false]
java.lang.NullPointerException: null
at
org.apache.ignite.raft.jraft.core.FSMCallerImpl.doCommitted(FSMCallerImpl.java:492)
~[ignite-raft-3.0.0-SNAPSHOT.jar:?]
at
org.apache.ignite.raft.jraft.core.FSMCallerImpl.runApplyTask(FSMCallerImpl.java:382)
~[ignite-raft-3.0.0-SNAPSHOT.jar:?]
at
org.apache.ignite.raft.jraft.core.FSMCallerImpl$ApplyTaskHandler.onEvent(FSMCallerImpl.java:136)
~[ignite-raft-3.0.0-SNAPSHOT.jar:?]
at
org.apache.ignite.raft.jraft.core.FSMCallerImpl$ApplyTaskHandler.onEvent(FSMCallerImpl.java:130)
~[ignite-raft-3.0.0-SNAPSHOT.jar:?]
at
org.apache.ignite.raft.jraft.disruptor.StripedDisruptor$StripeEntryHandler.onEvent(StripedDisruptor.java:226)
~[ignite-raft-3.0.0-SNAPSHOT.jar:?]
at
org.apache.ignite.raft.jraft.disruptor.StripedDisruptor$StripeEntryHandler.onEvent(StripedDisruptor.java:191)
~[ignite-raft-3.0.0-SNAPSHOT.jar:?]
at
com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:137)
[disruptor-3.3.7.jar:?]
at java.lang.Thread.run(Thread.java:834) [?:?]
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-20772) ItTableRaftSnapshotsTest#txSemanticsIsMaintained is flaky in different branches
[
https://issues.apache.org/jira/browse/IGNITE-20772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Chugunov updated IGNITE-20772:
-
Attachment: _Integration_Tests_Module_Runner_18692.log.zip
> ItTableRaftSnapshotsTest#txSemanticsIsMaintained is flaky in different
> branches
> ---
>
> Key: IGNITE-20772
> URL: https://issues.apache.org/jira/browse/IGNITE-20772
> Project: Ignite
> Issue Type: Bug
>Reporter: Sergey Chugunov
>Priority: Major
> Labels: ignite-3
> Attachments: _Integration_Tests_Module_Runner_18692.log.zip
>
>
> Test fails from time to time in different branches, success rate is 98%.
> Latest failure in main branch was caused by timeout in test logic:
> {code:java}
> java.lang.RuntimeException: java.util.concurrent.TimeoutException
> at org.apache.ignite.internal.Cluster.startNode(Cluster.java:336)
> at org.apache.ignite.internal.Cluster.startNode(Cluster.java:315)
> at
> org.apache.ignite.internal.raftsnapshot.ItTableRaftSnapshotsTest.reanimateNode(ItTableRaftSnapshotsTest.java:453)
> at
> org.apache.ignite.internal.raftsnapshot.ItTableRaftSnapshotsTest.reanimateNodeAndWaitForSnapshotInstalled(ItTableRaftSnapshotsTest.java:435)
> at
> org.apache.ignite.internal.raftsnapshot.ItTableRaftSnapshotsTest.reanimateNode2AndWaitForSnapshotInstalled(ItTableRaftSnapshotsTest.java:425)
> at
> org.apache.ignite.internal.raftsnapshot.ItTableRaftSnapshotsTest.txSemanticsIsMaintainedAfterInstallingSnapshot(ItTableRaftSnapshotsTest.java:494)
> at
> org.apache.ignite.internal.raftsnapshot.ItTableRaftSnapshotsTest.txSemanticsIsMaintained(ItTableRaftSnapshotsTest.java:466)
> ...
> Caused by: java.util.concurrent.TimeoutException
> at
> java.base/java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1886)
> at
> java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2021)
> at org.apache.ignite.internal.Cluster.startNode(Cluster.java:330)
> ... 93 more {code}
> Test involves node restart so multiple errors in logs about connection issues
> are expected:
> {code:java}
> [2023-10-27T08:24:44,662][WARN
> ][%itrst_tsim_2%Raft-Group-Client-4][RaftGroupServiceImpl] Recoverable error
> during the request type=GetLeaderRequestImpl occurred (will be retried on the
> randomly selected node):
> java.util.concurrent.CompletionException: java.net.ConnectException: Peer
> itrst_tsim_0 is unavailable
> at
> java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:331)
> ~[?:?]
> at
> java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:1099)
> ~[?:?]
> at
> java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2235)
> ~[?:?]
> at
> org.apache.ignite.internal.raft.RaftGroupServiceImpl.sendWithRetry(RaftGroupServiceImpl.java:523)
> ~[ignite-raft-3.0.0-SNAPSHOT.jar:?]
> at
> org.apache.ignite.internal.raft.RaftGroupServiceImpl.lambda$handleThrowable$40(RaftGroupServiceImpl.java:564)
> ~[ignite-raft-3.0.0-SNAPSHOT.jar:?]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
> at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
> [?:?]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> [?:?]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> [?:?]
> at java.lang.Thread.run(Thread.java:834) [?:?]
> Caused by: java.net.ConnectException: Peer itrst_tsim_0 is unavailable
> at
> org.apache.ignite.internal.raft.RaftGroupServiceImpl.resolvePeer(RaftGroupServiceImpl.java:761)
> ~[ignite-raft-3.0.0-SNAPSHOT.jar:?]
> at
> org.apache.ignite.internal.raft.RaftGroupServiceImpl.sendWithRetry(RaftGroupServiceImpl.java:522)
> ~[ignite-raft-3.0.0-SNAPSHOT.jar:?]
> ... 7 more {code}
> At the same time it is not clear from logs what prevented node from starting.
> Suite run with failed test is available
> [here|https://ci.ignite.apache.org/viewLog.html?buildId=7590927&buildTypeId=ApacheIgnite3xGradle_Test_IntegrationTests_ModuleRunner&tab=buildLog].
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-20772) ItTableRaftSnapshotsTest#txSemanticsIsMaintained is flaky in different branches
[
https://issues.apache.org/jira/browse/IGNITE-20772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Chugunov updated IGNITE-20772:
-
Description:
Test fails from time to time in different branches, success rate is 98%.
Latest failure in main branch was caused by timeout in test logic:
{code:java}
java.lang.RuntimeException: java.util.concurrent.TimeoutException
at org.apache.ignite.internal.Cluster.startNode(Cluster.java:336)
at org.apache.ignite.internal.Cluster.startNode(Cluster.java:315)
at
org.apache.ignite.internal.raftsnapshot.ItTableRaftSnapshotsTest.reanimateNode(ItTableRaftSnapshotsTest.java:453)
at
org.apache.ignite.internal.raftsnapshot.ItTableRaftSnapshotsTest.reanimateNodeAndWaitForSnapshotInstalled(ItTableRaftSnapshotsTest.java:435)
at
org.apache.ignite.internal.raftsnapshot.ItTableRaftSnapshotsTest.reanimateNode2AndWaitForSnapshotInstalled(ItTableRaftSnapshotsTest.java:425)
at
org.apache.ignite.internal.raftsnapshot.ItTableRaftSnapshotsTest.txSemanticsIsMaintainedAfterInstallingSnapshot(ItTableRaftSnapshotsTest.java:494)
at
org.apache.ignite.internal.raftsnapshot.ItTableRaftSnapshotsTest.txSemanticsIsMaintained(ItTableRaftSnapshotsTest.java:466)
...
Caused by: java.util.concurrent.TimeoutException
at
java.base/java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1886)
at
java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2021)
at org.apache.ignite.internal.Cluster.startNode(Cluster.java:330)
... 93 more {code}
Test involves node restart so multiple errors in logs about connection issues
are expected:
{code:java}
[2023-10-27T08:24:44,662][WARN
][%itrst_tsim_2%Raft-Group-Client-4][RaftGroupServiceImpl] Recoverable error
during the request type=GetLeaderRequestImpl occurred (will be retried on the
randomly selected node):
java.util.concurrent.CompletionException: java.net.ConnectException: Peer
itrst_tsim_0 is unavailable
at
java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:331)
~[?:?]
at
java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:1099)
~[?:?]
at
java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2235)
~[?:?]
at
org.apache.ignite.internal.raft.RaftGroupServiceImpl.sendWithRetry(RaftGroupServiceImpl.java:523)
~[ignite-raft-3.0.0-SNAPSHOT.jar:?]
at
org.apache.ignite.internal.raft.RaftGroupServiceImpl.lambda$handleThrowable$40(RaftGroupServiceImpl.java:564)
~[ignite-raft-3.0.0-SNAPSHOT.jar:?]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
[?:?]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
[?:?]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
[?:?]
at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: java.net.ConnectException: Peer itrst_tsim_0 is unavailable
at
org.apache.ignite.internal.raft.RaftGroupServiceImpl.resolvePeer(RaftGroupServiceImpl.java:761)
~[ignite-raft-3.0.0-SNAPSHOT.jar:?]
at
org.apache.ignite.internal.raft.RaftGroupServiceImpl.sendWithRetry(RaftGroupServiceImpl.java:522)
~[ignite-raft-3.0.0-SNAPSHOT.jar:?]
... 7 more {code}
At the same time it is not clear from logs what prevented node from starting.
Suite run with failed test is available
[here|https://ci.ignite.apache.org/viewLog.html?buildId=7590927&buildTypeId=ApacheIgnite3xGradle_Test_IntegrationTests_ModuleRunner&tab=buildLog],
logs are attached.
was:
Test fails from time to time in different branches, success rate is 98%.
Latest failure in main branch was caused by timeout in test logic:
{code:java}
java.lang.RuntimeException: java.util.concurrent.TimeoutException
at org.apache.ignite.internal.Cluster.startNode(Cluster.java:336)
at org.apache.ignite.internal.Cluster.startNode(Cluster.java:315)
at
org.apache.ignite.internal.raftsnapshot.ItTableRaftSnapshotsTest.reanimateNode(ItTableRaftSnapshotsTest.java:453)
at
org.apache.ignite.internal.raftsnapshot.ItTableRaftSnapshotsTest.reanimateNodeAndWaitForSnapshotInstalled(ItTableRaftSnapshotsTest.java:435)
at
org.apache.ignite.internal.raftsnapshot.ItTableRaftSnapshotsTest.reanimateNode2AndWaitForSnapshotInstalled(ItTableRaftSnapshotsTest.java:425)
at
org.apache.ignite.internal.raftsnapshot.ItTableRaftSnapshotsTest.txSemanticsIsMaintainedAfterInstallingSnapshot(ItTableRaftSnapshotsTest.java:494)
at
org.apache.ignite.internal.raftsnapshot.ItTableRaftSnapshotsTest.txSemanticsIsMaintained(ItTab
[jira] [Updated] (IGNITE-20772) ItTableRaftSnapshotsTest#txSemanticsIsMaintained is flaky in different branches
[
https://issues.apache.org/jira/browse/IGNITE-20772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Chugunov updated IGNITE-20772:
-
Description:
Test fails from time to time in different branches, success rate is 98%.
Latest failure in main branch was caused by timeout in test logic:
{code:java}
java.lang.RuntimeException: java.util.concurrent.TimeoutException
at org.apache.ignite.internal.Cluster.startNode(Cluster.java:336)
at org.apache.ignite.internal.Cluster.startNode(Cluster.java:315)
at
org.apache.ignite.internal.raftsnapshot.ItTableRaftSnapshotsTest.reanimateNode(ItTableRaftSnapshotsTest.java:453)
at
org.apache.ignite.internal.raftsnapshot.ItTableRaftSnapshotsTest.reanimateNodeAndWaitForSnapshotInstalled(ItTableRaftSnapshotsTest.java:435)
at
org.apache.ignite.internal.raftsnapshot.ItTableRaftSnapshotsTest.reanimateNode2AndWaitForSnapshotInstalled(ItTableRaftSnapshotsTest.java:425)
at
org.apache.ignite.internal.raftsnapshot.ItTableRaftSnapshotsTest.txSemanticsIsMaintainedAfterInstallingSnapshot(ItTableRaftSnapshotsTest.java:494)
at
org.apache.ignite.internal.raftsnapshot.ItTableRaftSnapshotsTest.txSemanticsIsMaintained(ItTableRaftSnapshotsTest.java:466)
...
Caused by: java.util.concurrent.TimeoutException
at
java.base/java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1886)
at
java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2021)
at org.apache.ignite.internal.Cluster.startNode(Cluster.java:330)
... 93 more {code}
Test involves node restart so multiple errors in logs about connection issues
are expected:
{code:java}
[2023-10-27T08:24:44,662][WARN
][%itrst_tsim_2%Raft-Group-Client-4][RaftGroupServiceImpl] Recoverable error
during the request type=GetLeaderRequestImpl occurred (will be retried on the
randomly selected node):
java.util.concurrent.CompletionException: java.net.ConnectException: Peer
itrst_tsim_0 is unavailable
at
java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:331)
~[?:?]
at
java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:1099)
~[?:?]
at
java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2235)
~[?:?]
at
org.apache.ignite.internal.raft.RaftGroupServiceImpl.sendWithRetry(RaftGroupServiceImpl.java:523)
~[ignite-raft-3.0.0-SNAPSHOT.jar:?]
at
org.apache.ignite.internal.raft.RaftGroupServiceImpl.lambda$handleThrowable$40(RaftGroupServiceImpl.java:564)
~[ignite-raft-3.0.0-SNAPSHOT.jar:?]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
[?:?]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
[?:?]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
[?:?]
at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: java.net.ConnectException: Peer itrst_tsim_0 is unavailable
at
org.apache.ignite.internal.raft.RaftGroupServiceImpl.resolvePeer(RaftGroupServiceImpl.java:761)
~[ignite-raft-3.0.0-SNAPSHOT.jar:?]
at
org.apache.ignite.internal.raft.RaftGroupServiceImpl.sendWithRetry(RaftGroupServiceImpl.java:522)
~[ignite-raft-3.0.0-SNAPSHOT.jar:?]
... 7 more {code}
At the same time it is not clear from logs what prevented node from starting.
Suite run with failed test is available
[here|https://ci.ignite.apache.org/viewLog.html?buildId=7590927&buildTypeId=ApacheIgnite3xGradle_Test_IntegrationTests_ModuleRunner&tab=buildLog].
was:
Test fails from time to time in different branches, success rate is 98%.
Latest failure in main branch was caused by timeout in test logic:
{code:java}
java.lang.RuntimeException: java.util.concurrent.TimeoutException
at org.apache.ignite.internal.Cluster.startNode(Cluster.java:336)
at org.apache.ignite.internal.Cluster.startNode(Cluster.java:315)
at
org.apache.ignite.internal.raftsnapshot.ItTableRaftSnapshotsTest.reanimateNode(ItTableRaftSnapshotsTest.java:453)
at
org.apache.ignite.internal.raftsnapshot.ItTableRaftSnapshotsTest.reanimateNodeAndWaitForSnapshotInstalled(ItTableRaftSnapshotsTest.java:435)
at
org.apache.ignite.internal.raftsnapshot.ItTableRaftSnapshotsTest.reanimateNode2AndWaitForSnapshotInstalled(ItTableRaftSnapshotsTest.java:425)
at
org.apache.ignite.internal.raftsnapshot.ItTableRaftSnapshotsTest.txSemanticsIsMaintainedAfterInstallingSnapshot(ItTableRaftSnapshotsTest.java:494)
at
org.apache.ignite.internal.raftsnapshot.ItTableRaftSnapshotsTest.txSemanticsIsMaintained(ItTableRaftSnapshotsTes
[jira] [Created] (IGNITE-20772) ItTableRaftSnapshotsTest#txSemanticsIsMaintained is flaky in different branches
Sergey Chugunov created IGNITE-20772:
Summary: ItTableRaftSnapshotsTest#txSemanticsIsMaintained is flaky
in different branches
Key: IGNITE-20772
URL: https://issues.apache.org/jira/browse/IGNITE-20772
Project: Ignite
Issue Type: Bug
Reporter: Sergey Chugunov
Test fails from time to time in different branches, success rate is 98%.
Latest failure in main branch was caused by timeout in test logic:
{code:java}
java.lang.RuntimeException: java.util.concurrent.TimeoutException
at org.apache.ignite.internal.Cluster.startNode(Cluster.java:336)
at org.apache.ignite.internal.Cluster.startNode(Cluster.java:315)
at
org.apache.ignite.internal.raftsnapshot.ItTableRaftSnapshotsTest.reanimateNode(ItTableRaftSnapshotsTest.java:453)
at
org.apache.ignite.internal.raftsnapshot.ItTableRaftSnapshotsTest.reanimateNodeAndWaitForSnapshotInstalled(ItTableRaftSnapshotsTest.java:435)
at
org.apache.ignite.internal.raftsnapshot.ItTableRaftSnapshotsTest.reanimateNode2AndWaitForSnapshotInstalled(ItTableRaftSnapshotsTest.java:425)
at
org.apache.ignite.internal.raftsnapshot.ItTableRaftSnapshotsTest.txSemanticsIsMaintainedAfterInstallingSnapshot(ItTableRaftSnapshotsTest.java:494)
at
org.apache.ignite.internal.raftsnapshot.ItTableRaftSnapshotsTest.txSemanticsIsMaintained(ItTableRaftSnapshotsTest.java:466)
...
Caused by: java.util.concurrent.TimeoutException
at
java.base/java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1886)
at
java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2021)
at org.apache.ignite.internal.Cluster.startNode(Cluster.java:330)
... 93 more {code}
Test involves node restart so multiple errors in logs about connection issues
are expected:
{code:java}
[2023-10-27T08:24:44,662][WARN
][%itrst_tsim_2%Raft-Group-Client-4][RaftGroupServiceImpl] Recoverable error
during the request type=GetLeaderRequestImpl occurred (will be retried on the
randomly selected node):
java.util.concurrent.CompletionException: java.net.ConnectException: Peer
itrst_tsim_0 is unavailable
at
java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:331)
~[?:?]
at
java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:1099)
~[?:?]
at
java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2235)
~[?:?]
at
org.apache.ignite.internal.raft.RaftGroupServiceImpl.sendWithRetry(RaftGroupServiceImpl.java:523)
~[ignite-raft-3.0.0-SNAPSHOT.jar:?]
at
org.apache.ignite.internal.raft.RaftGroupServiceImpl.lambda$handleThrowable$40(RaftGroupServiceImpl.java:564)
~[ignite-raft-3.0.0-SNAPSHOT.jar:?]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
[?:?]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
[?:?]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
[?:?]
at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: java.net.ConnectException: Peer itrst_tsim_0 is unavailable
at
org.apache.ignite.internal.raft.RaftGroupServiceImpl.resolvePeer(RaftGroupServiceImpl.java:761)
~[ignite-raft-3.0.0-SNAPSHOT.jar:?]
at
org.apache.ignite.internal.raft.RaftGroupServiceImpl.sendWithRetry(RaftGroupServiceImpl.java:522)
~[ignite-raft-3.0.0-SNAPSHOT.jar:?]
... 7 more {code}
At the same time it is not clear from logs what prevented node from starting.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-20608) ItIgniteNodeRestartTest#nodeWithDataTest is flaky on TC
[
https://issues.apache.org/jira/browse/IGNITE-20608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Chugunov updated IGNITE-20608:
-
Description:
Test fails in different branches with timeout in test logic, stack trace:
{code:java}
org.awaitility.core.ConditionTimeoutException: Condition with lambda expression
in org.apache.ignite.internal.runner.app.ItIgniteNodeRestartTest that uses
org.apache.ignite.table.Table, org.apache.ignite.table.Tableint was not
fulfilled within 30 seconds.
at
app//org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:167)
at
app//org.awaitility.core.CallableCondition.await(CallableCondition.java:78)
at
app//org.awaitility.core.CallableCondition.await(CallableCondition.java:26)
at
app//org.awaitility.core.ConditionFactory.until(ConditionFactory.java:985)
at
app//org.awaitility.core.ConditionFactory.until(ConditionFactory.java:954)
at
app//org.apache.ignite.internal.runner.app.ItIgniteNodeRestartTest.checkTableWithData(ItIgniteNodeRestartTest.java:1182)
at
app//org.apache.ignite.internal.runner.app.ItIgniteNodeRestartTest.nodeWithDataTest(ItIgniteNodeRestartTest.java:751)
at
java.base@11.0.17/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
//... not so important stack trace lines {code}
Its history is available
[here|https://ci.ignite.apache.org/project.html?projectId=ApacheIgnite3xGradle_Test_IntegrationTests&testNameId=7638414685760227643&tab=testDetails],
it is not clear what commit introduced this flakiness.
Some other tests from the same class fail as well, they may share the same root
cause so worth investigating them in the single ticket:
*
[ItIgniteNodeRestartTest.testTwoNodesRestartReverse|https://ci.ignite.apache.org/project.html?projectId=ApacheIgnite3xGradle_Test_IntegrationTests&testNameId=-6899510263921361541&tab=testDetails&branch_ApacheIgnite3xGradle_Test_IntegrationTests=%3Cdefault%3E]
*
[ItIgniteNodeRestartTest.nodeWithDataAndIndexRebuildTest|https://ci.ignite.apache.org/project.html?projectId=ApacheIgnite3xGradle_Test_IntegrationTests&testNameId=6204054563732065266&tab=testDetails&branch_ApacheIgnite3xGradle_Test_IntegrationTests=%3Cdefault%3E]
was:
Test fails in different branches with timeout in test logic, stack trace:
{code:java}
org.awaitility.core.ConditionTimeoutException: Condition with lambda expression
in org.apache.ignite.internal.runner.app.ItIgniteNodeRestartTest that uses
org.apache.ignite.table.Table, org.apache.ignite.table.Tableint was not
fulfilled within 30 seconds.
at
app//org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:167)
at
app//org.awaitility.core.CallableCondition.await(CallableCondition.java:78)
at
app//org.awaitility.core.CallableCondition.await(CallableCondition.java:26)
at
app//org.awaitility.core.ConditionFactory.until(ConditionFactory.java:985)
at
app//org.awaitility.core.ConditionFactory.until(ConditionFactory.java:954)
at
app//org.apache.ignite.internal.runner.app.ItIgniteNodeRestartTest.checkTableWithData(ItIgniteNodeRestartTest.java:1182)
at
app//org.apache.ignite.internal.runner.app.ItIgniteNodeRestartTest.nodeWithDataTest(ItIgniteNodeRestartTest.java:751)
at
java.base@11.0.17/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
//... not so important stack trace lines {code}
Its history is available
[here|https://ci.ignite.apache.org/project.html?projectId=ApacheIgnite3xGradle_Test_IntegrationTests&testNameId=7638414685760227643&tab=testDetails],
it is not clear what commit introduced this flakiness.
> ItIgniteNodeRestartTest#nodeWithDataTest is flaky on TC
> ---
>
> Key: IGNITE-20608
> URL: https://issues.apache.org/jira/browse/IGNITE-20608
> Project: Ignite
> Issue Type: Bug
>Reporter: Sergey Chugunov
>Priority: Major
> Labels: ignite-3
>
> Test fails in different branches with timeout in test logic, stack trace:
> {code:java}
> org.awaitility.core.ConditionTimeoutException: Condition with lambda
> expression in org.apache.ignite.internal.runner.app.ItIgniteNodeRestartTest
> that uses org.apache.ignite.table.Table, org.apache.ignite.table.Tableint was
> not fulfilled within 30 seconds.
> at
> app//org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:167)
> at
> app//org.awaitility.core.CallableCondition.await(CallableCondition.java:78)
> at
> app//org.awaitility.core.CallableCondition.await(CallableCondition.java:26)
> at
> app//org.awaitility.core.ConditionFactory.until(ConditionFactory.java:985)
> at
> app//org.awaitility.core.ConditionFactory.until(ConditionFactory.java:954)
> at
> app//org.apache.ignite.internal.runner.a
[jira] [Updated] (IGNITE-20608) ItIgniteNodeRestartTest#nodeWithDataTest is flaky on TC
[
https://issues.apache.org/jira/browse/IGNITE-20608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Chugunov updated IGNITE-20608:
-
Ignite Flags: (was: Docs Required,Release Notes Required)
> ItIgniteNodeRestartTest#nodeWithDataTest is flaky on TC
> ---
>
> Key: IGNITE-20608
> URL: https://issues.apache.org/jira/browse/IGNITE-20608
> Project: Ignite
> Issue Type: Bug
>Reporter: Sergey Chugunov
>Priority: Major
> Labels: ignite-3
>
> Test fails in different branches with timeout in test logic, stack trace:
> {code:java}
> org.awaitility.core.ConditionTimeoutException: Condition with lambda
> expression in org.apache.ignite.internal.runner.app.ItIgniteNodeRestartTest
> that uses org.apache.ignite.table.Table, org.apache.ignite.table.Tableint was
> not fulfilled within 30 seconds.
> at
> app//org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:167)
> at
> app//org.awaitility.core.CallableCondition.await(CallableCondition.java:78)
> at
> app//org.awaitility.core.CallableCondition.await(CallableCondition.java:26)
> at
> app//org.awaitility.core.ConditionFactory.until(ConditionFactory.java:985)
> at
> app//org.awaitility.core.ConditionFactory.until(ConditionFactory.java:954)
> at
> app//org.apache.ignite.internal.runner.app.ItIgniteNodeRestartTest.checkTableWithData(ItIgniteNodeRestartTest.java:1182)
> at
> app//org.apache.ignite.internal.runner.app.ItIgniteNodeRestartTest.nodeWithDataTest(ItIgniteNodeRestartTest.java:751)
> at
> java.base@11.0.17/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> //... not so important stack trace lines {code}
> Its history is available
> [here|https://ci.ignite.apache.org/project.html?projectId=ApacheIgnite3xGradle_Test_IntegrationTests&testNameId=7638414685760227643&tab=testDetails],
> it is not clear what commit introduced this flakiness.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-20678) Find a way to get a MvTableStorage on recovery the index build
[ https://issues.apache.org/jira/browse/IGNITE-20678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-20678: - Environment: (was: *strong text*) > Find a way to get a MvTableStorage on recovery the index build > -- > > Key: IGNITE-20678 > URL: https://issues.apache.org/jira/browse/IGNITE-20678 > Project: Ignite > Issue Type: Improvement >Reporter: Kirill Tkalenko >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > To implement the recovery of building indexes at the start of a node, it is > important for us to get a > *org.apache.ignite.internal.storage.engine.MvTableStorage* with an applied > replication log, how this can be done now is not entirely clear, but we need > to look at > org.apache.ignite.internal.table.distributed.TableManager#assignmentsUpdatedVv. > I also think it’s worth reviewing the existing > *org.apache.ignite.internal.index.IndexManager#getMvTableStorage* and > changing it a little, for example, using the API from > *org.apache.ignite.internal.table.distributed.TableManager*, or getting rid > of it for a different API. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-20608) ItIgniteNodeRestartTest#nodeWithDataTest is flaky on TC
Sergey Chugunov created IGNITE-20608:
Summary: ItIgniteNodeRestartTest#nodeWithDataTest is flaky on TC
Key: IGNITE-20608
URL: https://issues.apache.org/jira/browse/IGNITE-20608
Project: Ignite
Issue Type: Bug
Reporter: Sergey Chugunov
Test fails in different branches with timeout in test logic, stack trace:
{code:java}
org.awaitility.core.ConditionTimeoutException: Condition with lambda expression
in org.apache.ignite.internal.runner.app.ItIgniteNodeRestartTest that uses
org.apache.ignite.table.Table, org.apache.ignite.table.Tableint was not
fulfilled within 30 seconds.
at
app//org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:167)
at
app//org.awaitility.core.CallableCondition.await(CallableCondition.java:78)
at
app//org.awaitility.core.CallableCondition.await(CallableCondition.java:26)
at
app//org.awaitility.core.ConditionFactory.until(ConditionFactory.java:985)
at
app//org.awaitility.core.ConditionFactory.until(ConditionFactory.java:954)
at
app//org.apache.ignite.internal.runner.app.ItIgniteNodeRestartTest.checkTableWithData(ItIgniteNodeRestartTest.java:1182)
at
app//org.apache.ignite.internal.runner.app.ItIgniteNodeRestartTest.nodeWithDataTest(ItIgniteNodeRestartTest.java:751)
at
java.base@11.0.17/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
//... not so important stack trace lines {code}
Its history is available
[here|https://ci.ignite.apache.org/project.html?projectId=ApacheIgnite3xGradle_Test_IntegrationTests&testNameId=7638414685760227643&tab=testDetails],
it is not clear what commit introduced this flakiness.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-20570) ItDataSchemaSyncTest.checkSchemasCorrectlyRestore() is flaky possibly because of an issue with RAFT
Sergey Chugunov created IGNITE-20570:
Summary: ItDataSchemaSyncTest.checkSchemasCorrectlyRestore() is
flaky possibly because of an issue with RAFT
Key: IGNITE-20570
URL: https://issues.apache.org/jira/browse/IGNITE-20570
Project: Ignite
Issue Type: Bug
Reporter: Sergey Chugunov
Test failed recently with the following stack trace in logs (abridged):
{code:java}
org.apache.ignite.tx.TransactionException: IGN-REP-3
TraceId:41e00c72-74d5-4309-ba56-acdffd3a4132
org.apache.ignite.internal.replicator.exception.ReplicationTimeoutException:
IGN-REP-3 TraceId:41e00c72-74d5-4309-ba56-acdffd3a4132 Replication is timed out
[replicaGrpId=3_part_5]
at
java.base@11.0.17/java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:710)
at
app//org.apache.ignite.internal.util.ExceptionUtils$1.copy(ExceptionUtils.java:772)
at
app//org.apache.ignite.internal.util.ExceptionUtils$ExceptionFactory.createCopy(ExceptionUtils.java:706)
at
app//org.apache.ignite.internal.util.ExceptionUtils.copyExceptionWithCause(ExceptionUtils.java:543)
at
app//org.apache.ignite.internal.util.ExceptionUtils.copyExceptionWithCauseInternal(ExceptionUtils.java:641)
at
app//org.apache.ignite.internal.util.ExceptionUtils.copyExceptionWithCause(ExceptionUtils.java:494)
at
app//org.apache.ignite.internal.sql.AbstractSession.execute(AbstractSession.java:63)
at
app//org.apache.ignite.internal.runner.app.ItDataSchemaSyncTest.sql(ItDataSchemaSyncTest.java:364)
at
app//org.apache.ignite.internal.runner.app.ItDataSchemaSyncTest.checkSchemasCorrectlyRestore(ItDataSchemaSyncTest.java:273)
...{code}
Link to the failed run is
[here|https://ci.ignite.apache.org/viewLog.html?buildId=7537526&tab=buildResultsDiv&buildTypeId=ApacheIgnite3xGradle_Test_IntegrationTests_ModuleRunner].
It was failing before for other reasons (see linked ticket) but they seem to be
fixed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-20565) Test ItSchemaChangeTableViewTest.testMergeChangesAddDropAdd is flaky on TC
Sergey Chugunov created IGNITE-20565:
Summary: Test
ItSchemaChangeTableViewTest.testMergeChangesAddDropAdd is flaky on TC
Key: IGNITE-20565
URL: https://issues.apache.org/jira/browse/IGNITE-20565
Project: Ignite
Issue Type: Bug
Reporter: Sergey Chugunov
Test is flaky with low flaky rate, in the latest failure there is the following
stack trace reported by TC with NPE on top:
{code:java}
java.lang.NullPointerException
at
org.apache.ignite.internal.runner.app.ItSchemaChangeTableViewTest.testMergeChangesAddDropAdd(ItSchemaChangeTableViewTest.java:250)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at
org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:727)
at
org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
at
org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
at
org.junit.jupiter.engine.extension.SameThreadTimeoutInvocation.proceed(SameThreadTimeoutInvocation.java:45)
at
org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156)
at
org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:147)
at
org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:86)
at
org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103)
at
org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93)
at
org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
at
org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
at
org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
at
org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
at
org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:92)
at
org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:86)
at
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:217)
at
org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:213)
at
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:138)
at
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:68)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151)
at
org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
at
org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
at
org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
at
org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
at
org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:
[jira] [Updated] (IGNITE-20545) Test IgniteRpcTest.testDisconnect is flaky on TC
[
https://issues.apache.org/jira/browse/IGNITE-20545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Chugunov updated IGNITE-20545:
-
Description:
Test failed recently on main branch ([failed
run|https://ci.ignite.apache.org/viewLog.html?buildId=7536503&buildTypeId=ApacheIgnite3xGradle_Test_RunAllTests]),
there is an assertion in test logs:
{code:java}
org.opentest4j.AssertionFailedError: expected: but was:
at
app//org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
at
app//org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
at app//org.junit.jupiter.api.AssertTrue.failNotTrue(AssertTrue.java:63)
at app//org.junit.jupiter.api.AssertTrue.assertTrue(AssertTrue.java:36)
at app//org.junit.jupiter.api.AssertTrue.assertTrue(AssertTrue.java:31)
at app//org.junit.jupiter.api.Assertions.assertTrue(Assertions.java:180)
at
app//org.apache.ignite.raft.jraft.rpc.AbstractRpcTest.testDisconnect(AbstractRpcTest.java:128){code}
Test history shows that it fails occasionally in different branches with the
same error in logs.
Looks like there is some kind of race between events in test logic.
was:
Test failed recently on main branch ([failed
run|https://ci.ignite.apache.org/viewLog.html?buildId=7536503&buildTypeId=ApacheIgnite3xGradle_Test_RunAllTests]),
there is an assertion in test logs:
org.opentest4j.AssertionFailedError: expected: but was:
at
app//org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
at
app//org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
at app//org.junit.jupiter.api.AssertTrue.failNotTrue(AssertTrue.java:63)
at app//org.junit.jupiter.api.AssertTrue.assertTrue(AssertTrue.java:36)
at app//org.junit.jupiter.api.AssertTrue.assertTrue(AssertTrue.java:31)
at app//org.junit.jupiter.api.Assertions.assertTrue(Assertions.java:180)
at
app//org.apache.ignite.raft.jraft.rpc.AbstractRpcTest.testDisconnect(AbstractRpcTest.java:128)
Test history shows that it fails occasionally in different branches with the
same error in logs.
Looks like there is some kind of race between events in test logic.
> Test IgniteRpcTest.testDisconnect is flaky on TC
>
>
> Key: IGNITE-20545
> URL: https://issues.apache.org/jira/browse/IGNITE-20545
> Project: Ignite
> Issue Type: Bug
>Reporter: Sergey Chugunov
>Priority: Major
> Labels: ignite-3
>
> Test failed recently on main branch ([failed
> run|https://ci.ignite.apache.org/viewLog.html?buildId=7536503&buildTypeId=ApacheIgnite3xGradle_Test_RunAllTests]),
> there is an assertion in test logs:
> {code:java}
> org.opentest4j.AssertionFailedError: expected: but was:
> at
> app//org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
> at
> app//org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
> at app//org.junit.jupiter.api.AssertTrue.failNotTrue(AssertTrue.java:63)
> at app//org.junit.jupiter.api.AssertTrue.assertTrue(AssertTrue.java:36)
> at app//org.junit.jupiter.api.AssertTrue.assertTrue(AssertTrue.java:31)
> at app//org.junit.jupiter.api.Assertions.assertTrue(Assertions.java:180)
> at
> app//org.apache.ignite.raft.jraft.rpc.AbstractRpcTest.testDisconnect(AbstractRpcTest.java:128){code}
> Test history shows that it fails occasionally in different branches with the
> same error in logs.
> Looks like there is some kind of race between events in test logic.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-20553) Unexpected rebalancing immediately after table creation
[
https://issues.apache.org/jira/browse/IGNITE-20553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Chugunov updated IGNITE-20553:
-
Priority: Blocker (was: Major)
> Unexpected rebalancing immediately after table creation
> ---
>
> Key: IGNITE-20553
> URL: https://issues.apache.org/jira/browse/IGNITE-20553
> Project: Ignite
> Issue Type: Bug
>Reporter: Kirill Tkalenko
>Priority: Blocker
> Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>
> During the implementation of IGNITE-20330, it was discovered that when
> running
> {*}org.apache.ignite.internal.runner.app.ItDataSchemaSyncTest#checkSchemasCorrectlyRestore{*},
> a situation may occur that after creating the table, rebalancing will begin
> and this test will freeze on the first insert ({*}sql(ignite1,
> String.format("INSERT INTO " + TABLE_NAME + " VALUES(%d, %d, %d)", i, i, 2 *
> i));{*}). The situation is not reproduced often; you need to run the test
> several times.
> h3. Upd#1
> It's a known issue that node restart is broken. Before proceeding with given
> ticket metastorage compaction epic should be finished, espesially
> https://issues.apache.org/jira/browse/IGNITE-20210
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-20545) Test IgniteRpcTest.testDisconnect is flaky on TC
Sergey Chugunov created IGNITE-20545: Summary: Test IgniteRpcTest.testDisconnect is flaky on TC Key: IGNITE-20545 URL: https://issues.apache.org/jira/browse/IGNITE-20545 Project: Ignite Issue Type: Bug Reporter: Sergey Chugunov Test failed recently on main branch ([failed run|https://ci.ignite.apache.org/viewLog.html?buildId=7536503&buildTypeId=ApacheIgnite3xGradle_Test_RunAllTests]), there is an assertion in test logs: org.opentest4j.AssertionFailedError: expected: but was: at app//org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151) at app//org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132) at app//org.junit.jupiter.api.AssertTrue.failNotTrue(AssertTrue.java:63) at app//org.junit.jupiter.api.AssertTrue.assertTrue(AssertTrue.java:36) at app//org.junit.jupiter.api.AssertTrue.assertTrue(AssertTrue.java:31) at app//org.junit.jupiter.api.Assertions.assertTrue(Assertions.java:180) at app//org.apache.ignite.raft.jraft.rpc.AbstractRpcTest.testDisconnect(AbstractRpcTest.java:128) Test history shows that it fails occasionally in different branches with the same error in logs. Looks like there is some kind of race between events in test logic. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IGNITE-19925) NodeStoppingException upon stopping an embedded Ignite node
[
https://issues.apache.org/jira/browse/IGNITE-19925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Chugunov reassigned IGNITE-19925:
Assignee: Sergey Chugunov
> NodeStoppingException upon stopping an embedded Ignite node
> ---
>
> Key: IGNITE-19925
> URL: https://issues.apache.org/jira/browse/IGNITE-19925
> Project: Ignite
> Issue Type: Bug
> Components: persistence
>Reporter: Ivan Artiukhov
>Assignee: Sergey Chugunov
>Priority: Major
> Labels: ignite-3
> Attachments: NodeStoppingExceptionTest.java,
> NodeStoppingExceptionTest.log
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> See the attached reproducer.
> Steps:
> - Start an embedded Ignite node.
> - Create a table via key-vaue: 11 columns of type VARCHAR
> - Insert 10 sample rows
> - Stop the node via {{IgnitionManager#stop}}
> Expected result:
> No exceptions in the node's log
> Actual result:
> The following exception is seen:
> {noformat}
> Caused by: org.apache.ignite.lang.NodeStoppingException: IGN-CMN-1
> TraceId:65d933f8-94bd-41e6-928d-7defcf52744c Operation has been cancelled
> (node is stopping).
> at
> org.apache.ignite.network.DefaultMessagingService.invoke0(DefaultMessagingService.java:227)
> at
> org.apache.ignite.network.DefaultMessagingService.invoke(DefaultMessagingService.java:159)
> at
> org.apache.ignite.network.MessagingService.invoke(MessagingService.java:145)
> at
> org.apache.ignite.internal.raft.client.TopologyAwareRaftGroupService.sendWithRetry(TopologyAwareRaftGroupService.java:211)
> at
> org.apache.ignite.internal.raft.client.TopologyAwareRaftGroupService.sendSubscribeMessage(TopologyAwareRaftGroupService.java:197)
> at
> org.apache.ignite.internal.raft.client.TopologyAwareRaftGroupService.unsubscribeLeader(TopologyAwareRaftGroupService.java:329)
> at
> org.apache.ignite.internal.replicator.Replica.shutdown(Replica.java:278)
> at
> java.base/java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:1106)
> at
> java.base/java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2235)
> at
> org.apache.ignite.internal.replicator.ReplicaManager.stopReplicaInternal(ReplicaManager.java:410)
> at
> org.apache.ignite.internal.replicator.ReplicaManager.stopReplica(ReplicaManager.java:385)
> at
> org.apache.ignite.internal.table.distributed.TableManager.lambda$cleanUpTablesResources$30(TableManager.java:1093)
> at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
> at
> org.apache.ignite.internal.table.distributed.TableManager.cleanUpTablesResources(TableManager.java:1119)
> at
> org.apache.ignite.internal.table.distributed.TableManager.stop(TableManager.java:1045)
> at
> org.apache.ignite.internal.app.LifecycleManager.lambda$stopAllComponents$1(LifecycleManager.java:133)
> at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)
> at
> org.apache.ignite.internal.app.LifecycleManager.stopAllComponents(LifecycleManager.java:131)
> at
> org.apache.ignite.internal.app.LifecycleManager.stopNode(LifecycleManager.java:115)
> at org.apache.ignite.internal.app.IgniteImpl.stop(IgniteImpl.java:807)
> at
> org.apache.ignite.internal.app.IgnitionImpl.lambda$stop$0(IgnitionImpl.java:109)
> at
> java.base/java.util.concurrent.ConcurrentHashMap.computeIfPresent(ConcurrentHashMap.java:1822)
> at
> org.apache.ignite.internal.app.IgnitionImpl.stop(IgnitionImpl.java:108)
> at org.apache.ignite.IgnitionManager.stop(IgnitionManager.java:96)
> at org.apache.ignite.IgnitionManager.stop(IgnitionManager.java:82)
> at
> org.apache.ignite.example.AbstractExamplesTest.stopNode(AbstractExamplesTest.java:76)
> {noformat}
> {{git bisect}} says that the following commit introduced the bug (belongs to
> IGNITE-19199):
> {noformat}
> b6004047b3c3e9cd91b5ccf28c26ee206c1e3a7f is the first bad commit
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-19712) Handle rebalance wrt indexes
[ https://issues.apache.org/jira/browse/IGNITE-19712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-19712: - Epic Link: IGNITE-17766 Ignite Flags: (was: Docs Required,Release Notes Required) > Handle rebalance wrt indexes > > > Key: IGNITE-19712 > URL: https://issues.apache.org/jira/browse/IGNITE-19712 > Project: Ignite > Issue Type: Bug >Reporter: Semyon Danilov >Priority: Major > Labels: ignite-3 > > After IGNITE-19363, index storages are no longer lazily instantiated. Need to > listen to assignment changes and start new storages -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-20277) Test ItIgniteInMemoryNodeRestartTest.inMemoryNodeRestartNotLeader(TestInfo) is flaky in main branch
Sergey Chugunov created IGNITE-20277:
Summary: Test
ItIgniteInMemoryNodeRestartTest.inMemoryNodeRestartNotLeader(TestInfo) is flaky
in main branch
Key: IGNITE-20277
URL: https://issues.apache.org/jira/browse/IGNITE-20277
Project: Ignite
Issue Type: Bug
Reporter: Sergey Chugunov
An attempt was made to increase stability of the test (see linked ticket) but
fresh history for main branch shows that it is still flaky: [TC
history|https://ci.ignite.apache.org/project.html?projectId=ApacheIgnite3xGradle_Test_IntegrationTests&testNameId=2378861889774452265&tab=testDetails&branch_ApacheIgnite3xGradle_Test_IntegrationTests=%3Cdefault%3E]
Sometimes it fails with an assertion in test code:
{code:java}
org.opentest4j.AssertionFailedError: expected: but was:
...
at
app//org.apache.ignite.internal.runner.app.ItIgniteInMemoryNodeRestartTest.inMemoryNodeRestartNotLeader(ItIgniteInMemoryNodeRestartTest.java:197)
{code}
other failures are triggered by operation timeouts:
{code:java}
org.apache.ignite.tx.TransactionException: IGN-REP-3
TraceId:7ba66f02-e08d-41fb-b198-15374cfa717c Replication is timed out
[replicaGrpId=1_part_0]
at
app//org.apache.ignite.internal.util.ExceptionUtils.lambda$withCause$0(ExceptionUtils.java:378)
at
app//org.apache.ignite.internal.util.ExceptionUtils.withCauseInternal(ExceptionUtils.java:461)
at
app//org.apache.ignite.internal.util.ExceptionUtils.withCause(ExceptionUtils.java:378)
at
app//org.apache.ignite.internal.table.distributed.storage.InternalTableImpl.wrapReplicationException(InternalTableImpl.java:1584)
at
app//org.apache.ignite.internal.table.distributed.storage.InternalTableImpl.lambda$postEnlist$9(InternalTableImpl.java:506)
at
java.base@11.0.17/java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:930)
at
java.base@11.0.17/java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:907)
at
java.base@11.0.17/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
at
java.base@11.0.17/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2088)
at
app//org.apache.ignite.internal.table.distributed.storage.InternalTableImpl.lambda$enlistWithRetry$5(InternalTableImpl.java:480)
at
java.base@11.0.17/java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:930)
at
java.base@11.0.17/java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:907)
at
java.base@11.0.17/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
at
java.base@11.0.17/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2088)
at
app//org.apache.ignite.internal.replicator.ReplicaService.lambda$sendToReplica$3(ReplicaService.java:97)
at
java.base@11.0.17/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859)
at
java.base@11.0.17/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837)
at
java.base@11.0.17/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:479)
at
java.base@11.0.17/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
at
java.base@11.0.17/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
at
java.base@11.0.17/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
at
java.base@11.0.17/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
at
java.base@11.0.17/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
Caused by:
org.apache.ignite.internal.replicator.exception.ReplicationTimeoutException:
IGN-REP-3 TraceId:7ba66f02-e08d-41fb-b198-15374cfa717c Replication is timed out
[replicaGrpId=1_part_0]
... 9 more
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-20133) Compute hashes for integral/decimal columns in a stable way
[
https://issues.apache.org/jira/browse/IGNITE-20133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Chugunov updated IGNITE-20133:
-
Priority: Minor (was: Major)
> Compute hashes for integral/decimal columns in a stable way
> ---
>
> Key: IGNITE-20133
> URL: https://issues.apache.org/jira/browse/IGNITE-20133
> Project: Ignite
> Issue Type: Improvement
>Reporter: Roman Puchkovskiy
>Priority: Minor
> Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>
> The idea is to make hash computation for integral and decimal types satisfy
> the following property: if a column type is changed from an integral to a
> decimal type, the hashes for values that are already stored remain the same.
> This will allow us to permit chaning type (integral -> decimal and decimal ->
> longer decimal) of a column that is included in a HASH index.
> A hash that has this property is the following function:
> hash(val.toString(TRIM_TRAILING_ZEROS)). For instance, for 1 it will be
> hash("1"), for 1.000 it will also be hash("1"), but for 1.23 it will give
> hash("1.23").
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-19967) Design restart of Distribution Zone Manager in accordance with the new local recovery process
[ https://issues.apache.org/jira/browse/IGNITE-19967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-19967: - Epic Link: IGNITE-20166 > Design restart of Distribution Zone Manager in accordance with the new local > recovery process > - > > Key: IGNITE-19967 > URL: https://issues.apache.org/jira/browse/IGNITE-19967 > Project: Ignite > Issue Type: Task >Reporter: Mirza Aliev >Assignee: Mirza Aliev >Priority: Major > Labels: ignite-3 > > In https://issues.apache.org/jira/browse/IGNITE-19061 we have provided and > later implemented algorithm for restoring zones states after restart. > In https://issues.apache.org/jira/browse/IGNITE-19778 and other tickets new > invariants for recovery were introduced, in particular, in the new proposal > node state is recovered not until applied revision, but until some higher > revision, which brings inconsistency for restoring Distribution Zone Manager > because the process was based on saving states to the Vault and on previous > mechanism with restoring from the Vault > In this task we need to provide a new approach, taking into account new > circumstances. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19955) Fix create zone on restart rewrites existing data nodes because of trigger key inconsistnecy
[
https://issues.apache.org/jira/browse/IGNITE-19955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Chugunov updated IGNITE-19955:
-
Epic Link: IGNITE-20166
> Fix create zone on restart rewrites existing data nodes because of trigger
> key inconsistnecy
>
>
> Key: IGNITE-19955
> URL: https://issues.apache.org/jira/browse/IGNITE-19955
> Project: Ignite
> Issue Type: Bug
>Reporter: Mirza Aliev
>Priority: Major
> Labels: ignite-3
>
> Currently we have the logic for initialisation of newly created zone that it
> writes keys
> {noformat}
> zoneDataNodesKey(zoneId), zoneScaleUpChangeTriggerKey(zoneId),
> zoneScaleDownChangeTriggerKey(zoneId), zonesChangeTriggerKey(zoneId)
> {noformat}
> to metastorage, and condition is
> {noformat}
> static CompoundCondition triggerKeyConditionForZonesChanges(long
> revision, int zoneId) {
> return or(
> notExists(zonesChangeTriggerKey(zoneId)),
>
> value(zonesChangeTriggerKey(zoneId)).lt(ByteUtils.longToBytes(revision))
> );
> {noformat}
> Recovery process implies that the create zone event will be processed again,
> but with the higher revision, so data nodes will be rewritten.
> We need to handle this situation, so data nodes will be consistent after
> restart.
> Possible solution is to change condition to
> {noformat}
> static SimpleCondition triggerKeyConditionForZonesCreation(long revision,
> int zoneId) {
> return notExists(zonesChangeTriggerKey(zoneId));
> }
> static SimpleCondition triggerKeyConditionForZonesDelete(int zoneId) {
> return exists(zonesChangeTriggerKey(zoneId));
> }
> {noformat}
>
> so we could not rely on revision and check only existence of the key, when we
> create or remove zone. The problem in this solution is that reordering of the
> create and remove on some node could lead to not consistent state for zones
> key in metastorage
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-19532) Introduce happends before relation between local meta storage safe time publication and completion of corresponding meta storage listners
[
https://issues.apache.org/jira/browse/IGNITE-19532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Chugunov updated IGNITE-19532:
-
Epic Link: IGNITE-20166
> Introduce happends before relation between local meta storage safe time
> publication and completion of corresponding meta storage listners
> -
>
> Key: IGNITE-19532
> URL: https://issues.apache.org/jira/browse/IGNITE-19532
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexander Lapin
>Assignee: Semyon Danilov
>Priority: Blocker
> Labels: ignite-3
> Fix For: 3.0.0-beta2
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> h3. Motivation
> Let's assume that we have a component named LeaseTracker that locally stores
> leases in a map though meta storage updates
> {code:java}
> public class LeaseTracker implements PlacementDriver {
> /** Leases cache. */
> private final Map leases;
> ...
> private class UpdateListener implements WatchListener {
> @Override
> public CompletableFuture onUpdate(WatchEvent event) {
> for (EntryEvent entry : event.entryEvents()) {
> ...
> Lease lease = fromBytes(msEntry.value());
> leases.put(grpId, lease);
> ...
> return completedFuture(null);
> }
> }
> ...{code}
> and we want to await lease in a meta storage safe time bounded way.
> {code:java}
> public CompletableFuture getPrimaryReplica(ReplicationGroupId
> replicationGroupId, HybridTimestamp timestamp) {
> ...
> return msManager.clusterTime().waitFor(timestamp).thenApply(() -> {
> ...
> Lease lease0 = leases.get(replicationGroupId);
> if (lease.getExpirationTime().after(timestamp)) {
> return lease0;
> } else {
> return null;
> }
> ...
>}
> } {code}
> Currently that won't work properly
> because
> {code:java}
> msManager.clusterTime().waitFor(timestamp) {code}
> will be completed before local leases will be populated with data through
> meta storage events.
> Thus it'll be nice to complete clusterTime().waitFor(timestamp) futute (and
> generally speaking publish safe time) after all corresponding ms listeners
> are completed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-19778) Restore components state on metastorage recovery
[ https://issues.apache.org/jira/browse/IGNITE-19778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-19778: - Epic Link: IGNITE-20166 > Restore components state on metastorage recovery > > > Key: IGNITE-19778 > URL: https://issues.apache.org/jira/browse/IGNITE-19778 > Project: Ignite > Issue Type: Bug >Reporter: Semyon Danilov >Assignee: Semyon Danilov >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > Time Spent: 4h 40m > Remaining Estimate: 0h > > Upon metastorage signalling > (https://issues.apache.org/jira/browse/IGNITE-19777) that certain revision > that was available when recovery started is achieved by the local metastorage > storage, components should restore their state using a "snapshot" of the data > with that specific revision. For example, TableManager might want to start > tables that it sees in that snapshot. Note that tables that are missing from > the data snapshot, must be destroyed as they are now obsolete. > So state must be restored and outdated remains of the previous state must be > cleared out. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19777) Perform local metastorage recovery
[ https://issues.apache.org/jira/browse/IGNITE-19777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-19777: - Epic Link: IGNITE-20166 > Perform local metastorage recovery > --- > > Key: IGNITE-19777 > URL: https://issues.apache.org/jira/browse/IGNITE-19777 > Project: Ignite > Issue Type: Bug >Reporter: Semyon Danilov >Assignee: Semyon Danilov >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > Time Spent: 2h 10m > Remaining Estimate: 0h > > Upon recovery, metastorage should request current cluster-wide metastorage > revision and wait until this revision is achieved. > After that it is safe to continue starting other components depending on the > metastorage and perform their own recovery based on the metastorage data at > said revision. After recovery is performed, watches must be installed with > the revision + 1. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20166) Metastorage Compaction
[ https://issues.apache.org/jira/browse/IGNITE-20166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-20166: - Fix Version/s: 3.0.0-beta2 > Metastorage Compaction > -- > > Key: IGNITE-20166 > URL: https://issues.apache.org/jira/browse/IGNITE-20166 > Project: Ignite > Issue Type: Epic >Reporter: Sergey Chugunov >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > General logic of metastorage is that it supports multiple versions for the > same key corresponding to different timestamps. > At the same time there is a concept of a low watermark - a moment in time > when all timestamps falling below that mark become obsolete and can be pruned > from Metastorage. > We need to implement a mechanism to compact Metastorage to remove obsolete > keys. At the same time some components may rely on availability of history of > key's versions. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-20166) Metastorage Compaction
Sergey Chugunov created IGNITE-20166: Summary: Metastorage Compaction Key: IGNITE-20166 URL: https://issues.apache.org/jira/browse/IGNITE-20166 Project: Ignite Issue Type: Epic Reporter: Sergey Chugunov General logic of metastorage is that it supports multiple versions for the same key corresponding to different timestamps. At the same time there is a concept of a low watermark - a moment in time when all timestamps falling below that mark become obsolete and can be pruned from Metastorage. We need to implement a mechanism to compact Metastorage to remove obsolete keys. At the same time some components may rely on availability of history of key's versions. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-19883) Add a review checklist to a PR in Apache Ignite 3 repository
[
https://issues.apache.org/jira/browse/IGNITE-19883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17746866#comment-17746866
]
Sergey Chugunov commented on IGNITE-19883:
--
[~slava.koptilin], I expanded Formal Criteria section with more detailed items,
could you please take a look once again?
> Add a review checklist to a PR in Apache Ignite 3 repository
>
>
> Key: IGNITE-19883
> URL: https://issues.apache.org/jira/browse/IGNITE-19883
> Project: Ignite
> Issue Type: Task
>Reporter: Sergey Chugunov
>Assignee: Sergey Chugunov
>Priority: Major
> Labels: ignite-3
> Fix For: 3.0.0-beta2
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> We need to add a checklist of bullets to verify PR against right to the PRs
> pages in GitHub, like a list already implemented for PRs in AI2 repo:
> {noformat}
> 1. There is a single JIRA ticket related to the pull request.
> 2. The web-link to the pull request is attached to the JIRA ticket.
> 3. The JIRA ticket has the Patch Available state.
> 4. The pull request body describes changes that have been made. The
> description explains WHAT and WHY was made instead of HOW.
> 5. The pull request title is treated as the final commit message. The
> following pattern must be used: IGNITE- Change summary where -
> number of JIRA issue.
> 6. A reviewer has been mentioned through the JIRA comments (see the
> Maintainers list)
> 7. The pull request has been checked by the Teamcity Bot and the green visa
> attached to the JIRA ticket (see TC.Bot: Check PR)
> {noformat}
> Though in AI2 checklist if focused mostly on formal stuff, and we want to
> implement the list more dedicated to code quality and easy-to-review reqs.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-19717) Get rid of creating indexes at the start of a MvTableStorage
[ https://issues.apache.org/jira/browse/IGNITE-19717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-19717: - Epic Link: IGNITE-17766 (was: IGNITE-19502) > Get rid of creating indexes at the start of a MvTableStorage > > > Key: IGNITE-19717 > URL: https://issues.apache.org/jira/browse/IGNITE-19717 > Project: Ignite > Issue Type: Improvement >Reporter: Kirill Tkalenko >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > Now, at the start of the *MvTableStorage*, we create indexes, which is not > correct, this should be done by external code that should know about all > indexes. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19112) Get rid of MvTableStorage#getOrCreateIndex
[ https://issues.apache.org/jira/browse/IGNITE-19112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-19112: - Epic Link: IGNITE-17766 (was: IGNITE-19502) > Get rid of MvTableStorage#getOrCreateIndex > -- > > Key: IGNITE-19112 > URL: https://issues.apache.org/jira/browse/IGNITE-19112 > Project: Ignite > Issue Type: Improvement >Reporter: Kirill Tkalenko >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > Get rid of methods: > * org.apache.ignite.internal.storage.engine.MvTableStorage#getOrCreateIndex > * > org.apache.ignite.internal.storage.engine.MvTableStorage#getOrCreateHashIndex > * > org.apache.ignite.internal.storage.engine.MvTableStorage#getOrCreateSortedIndex > and make new ones: > * org.apache.ignite.internal.storage.engine.MvTableStorage#createIndex > * org.apache.ignite.internal.storage.engine.MvTableStorage#getOrCreateIndex > * org.apache.ignite.internal.storage.engine.MvTableStorage#createHashIndex > * org.apache.ignite.internal.storage.engine.MvTableStorage#getHashIndex > * org.apache.ignite.internal.storage.engine.MvTableStorage#createSortedIndex > * org.apache.ignite.internal.storage.engine.MvTableStorage#getSortedIndex -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19883) Add a review checklist to a PR in Apache Ignite 3 repository
[
https://issues.apache.org/jira/browse/IGNITE-19883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Chugunov updated IGNITE-19883:
-
Fix Version/s: 3.0.0-beta2
> Add a review checklist to a PR in Apache Ignite 3 repository
>
>
> Key: IGNITE-19883
> URL: https://issues.apache.org/jira/browse/IGNITE-19883
> Project: Ignite
> Issue Type: Task
>Reporter: Sergey Chugunov
>Assignee: Sergey Chugunov
>Priority: Major
> Labels: ignite-3
> Fix For: 3.0.0-beta2
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> We need to add a checklist of bullets to verify PR against right to the PRs
> pages in GitHub, like a list already implemented for PRs in AI2 repo:
> {noformat}
> 1. There is a single JIRA ticket related to the pull request.
> 2. The web-link to the pull request is attached to the JIRA ticket.
> 3. The JIRA ticket has the Patch Available state.
> 4. The pull request body describes changes that have been made. The
> description explains WHAT and WHY was made instead of HOW.
> 5. The pull request title is treated as the final commit message. The
> following pattern must be used: IGNITE- Change summary where -
> number of JIRA issue.
> 6. A reviewer has been mentioned through the JIRA comments (see the
> Maintainers list)
> 7. The pull request has been checked by the Teamcity Bot and the green visa
> attached to the JIRA ticket (see TC.Bot: Check PR)
> {noformat}
> Though in AI2 checklist if focused mostly on formal stuff, and we want to
> implement the list more dedicated to code quality and easy-to-review reqs.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-19883) Add a review checklist to a PR in Apache Ignite 3 repository
Sergey Chugunov created IGNITE-19883:
Summary: Add a review checklist to a PR in Apache Ignite 3
repository
Key: IGNITE-19883
URL: https://issues.apache.org/jira/browse/IGNITE-19883
Project: Ignite
Issue Type: Task
Reporter: Sergey Chugunov
Assignee: Sergey Chugunov
We need to add a checklist of bullets to verify PR against right to the PRs
pages in GitHub, like a list already implemented for PRs in AI2 repo:
{noformat}
1. There is a single JIRA ticket related to the pull request.
2. The web-link to the pull request is attached to the JIRA ticket.
3. The JIRA ticket has the Patch Available state.
4. The pull request body describes changes that have been made. The description
explains WHAT and WHY was made instead of HOW.
5. The pull request title is treated as the final commit message. The following
pattern must be used: IGNITE- Change summary where - number of JIRA
issue.
6. A reviewer has been mentioned through the JIRA comments (see the Maintainers
list)
7. The pull request has been checked by the Teamcity Bot and the green visa
attached to the JIRA ticket (see TC.Bot: Check PR)
{noformat}
Though in AI2 checklist if focused mostly on formal stuff, and we want to
implement the list more dedicated to code quality and easy-to-review reqs.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-19199) Idle safe time propagation for the metastorage
[ https://issues.apache.org/jira/browse/IGNITE-19199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-19199: - Epic Link: IGNITE-18733 > Idle safe time propagation for the metastorage > -- > > Key: IGNITE-19199 > URL: https://issues.apache.org/jira/browse/IGNITE-19199 > Project: Ignite > Issue Type: Improvement >Reporter: Semyon Danilov >Priority: Major > Labels: ignite-3 > > In https://issues.apache.org/jira/browse/IGNITE-19028 safe time is propagated > from the leader every 1 second, which is sub-optimal. The timeout should be > configurable + safe time should be only propagated if the metastorage is > really idle (no updates were made in this timeout period) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19199) Idle safe time propagation for the metastorage
[ https://issues.apache.org/jira/browse/IGNITE-19199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-19199: - Ignite Flags: (was: Docs Required,Release Notes Required) > Idle safe time propagation for the metastorage > -- > > Key: IGNITE-19199 > URL: https://issues.apache.org/jira/browse/IGNITE-19199 > Project: Ignite > Issue Type: Improvement >Reporter: Semyon Danilov >Priority: Major > Labels: ignite-3 > > In https://issues.apache.org/jira/browse/IGNITE-19028 safe time is propagated > from the leader every 1 second, which is sub-optimal. The timeout should be > configurable + safe time should be only propagated if the metastorage is > really idle (no updates were made in this timeout period) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19692) Design Resilient Distributed Operations mechanism
[ https://issues.apache.org/jira/browse/IGNITE-19692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-19692: - Epic Link: IGNITE-18733 > Design Resilient Distributed Operations mechanism > - > > Key: IGNITE-19692 > URL: https://issues.apache.org/jira/browse/IGNITE-19692 > Project: Ignite > Issue Type: Task >Reporter: Roman Puchkovskiy >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > We need a mechanism that would allow to do the following: > # Execute an operation on all (or some of) partitions of a table > # The whole operation is split into sub-operations (each of which operate on > a single partition) > # Each sub-operation must be resilient: that is, if the node that hosts it > restarts or the partition moves to another node, the operation should proceed > # When a sub-operation ends, it notifies the operation tracker/coordinator > # When all sub-operations end, the tracker might take some action (like > starting a subsequent operation) > # The tracker is also resilient > We need such a mechanism in a few places in the system: > # Transaction cleanup? > # Index build > # Table data validation as a part of a schema change that requires a > validation (like a narrowing type change) > Probably, more applications of the mechanism will emerge. > > On the possible implementation: the tracker could be collocated with table's > primary replica (that would guarantee that at most one tracker exists at all > times). We could store the data needed to track the operation in the > Meta-Storage under a prefix corresponding to the table, like > 'ops...'. We could store the completion status for > each of the partitions there along with some operation-wide status. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19532) Introduce happends before relation between local meta storage safe time publication and completion of corresponding meta storage listners
[
https://issues.apache.org/jira/browse/IGNITE-19532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Chugunov updated IGNITE-19532:
-
Labels: ignite-3 (was: )
> Introduce happends before relation between local meta storage safe time
> publication and completion of corresponding meta storage listners
> -
>
> Key: IGNITE-19532
> URL: https://issues.apache.org/jira/browse/IGNITE-19532
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
> Labels: ignite-3
>
> h3. Motivation
> Let's assume that we have a component named LeaseTracker that locally stores
> leases in a map though meta storage updates
> {code:java}
> public class LeaseTracker implements PlacementDriver {
> /** Leases cache. */
> private final Map leases;
> ...
> private class UpdateListener implements WatchListener {
> @Override
> public CompletableFuture onUpdate(WatchEvent event) {
> for (EntryEvent entry : event.entryEvents()) {
> ...
> Lease lease = fromBytes(msEntry.value());
> leases.put(grpId, lease);
> ...
> return completedFuture(null);
> }
> }
> ...{code}
> and we want to await lease in a meta storage safe time bounded way.
> {code:java}
> public CompletableFuture getPrimaryReplica(ReplicationGroupId
> replicationGroupId, HybridTimestamp timestamp) {
> ...
> return msManager.clusterTime().waitFor(timestamp).thenApply(() -> {
> ...
> Lease lease0 = leases.get(replicationGroupId);
> if (lease.getExpirationTime().after(timestamp)) {
> return lease0;
> } else {
> return null;
> }
> ...
>}
> } {code}
> Currently that won't work properly
> because
> {code:java}
> msManager.clusterTime().waitFor(timestamp) {code}
> will be completed before local leases will be populated with data through
> meta storage events.
> Thus it'll be nice to complete clusterTime().waitFor(timestamp) futute (and
> generally speaking publish safe time) after all corresponding ms listeners
> are completed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (IGNITE-16288) Async API for the configuration
[ https://issues.apache.org/jira/browse/IGNITE-16288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov resolved IGNITE-16288. -- Fix Version/s: (was: 3.0.0-beta2) Resolution: Won't Fix This ticket is not valid anymore as async API was implemented as part of other tickets. > Async API for the configuration > --- > > Key: IGNITE-16288 > URL: https://issues.apache.org/jira/browse/IGNITE-16288 > Project: Ignite > Issue Type: Task >Reporter: Semyon Danilov >Priority: Major > Labels: ignite-3, tech-debt > > Currently, configuration API is blocking, which creates a limitation for a > call site -- we can't call configuration methods from network threads. E.g. > TableManager#tablesAsyncInternal supposed to be non-blocking, but it does use > configuration API, so it had to be used via CompletableFuture#supplyAsync. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19223) Implement accumulation of schema updates in CatalogService
[ https://issues.apache.org/jira/browse/IGNITE-19223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-19223: - Epic Link: IGNITE-19502 (was: IGNITE-18733) > Implement accumulation of schema updates in CatalogService > -- > > Key: IGNITE-19223 > URL: https://issues.apache.org/jira/browse/IGNITE-19223 > Project: Ignite > Issue Type: Improvement >Reporter: Roman Puchkovskiy >Priority: Major > Labels: ignite-3 > > CatalogService implementation should hold the historical information about > schema updates. This means that: > # It needs to load the known history of schema updates on node start > # It should subsribe to new schema updates -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19460) Implement missed DDL commands with using Catalog
[ https://issues.apache.org/jira/browse/IGNITE-19460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-19460: - Epic Link: IGNITE-19502 (was: IGNITE-17766) > Implement missed DDL commands with using Catalog > - > > Key: IGNITE-19460 > URL: https://issues.apache.org/jira/browse/IGNITE-19460 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Andrey Mashenkov >Assignee: Andrey Mashenkov >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > Earlier in IGNITE-19081 we introduce Catalog events. > Let's add table altering, index creation/dropping commands and them to the > CatalogManager. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19112) Get rid of MvTableStorage#getOrCreateIndex
[ https://issues.apache.org/jira/browse/IGNITE-19112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-19112: - Epic Link: IGNITE-19502 (was: IGNITE-17766) > Get rid of MvTableStorage#getOrCreateIndex > -- > > Key: IGNITE-19112 > URL: https://issues.apache.org/jira/browse/IGNITE-19112 > Project: Ignite > Issue Type: Improvement >Reporter: Kirill Tkalenko >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > Get rid of methods: > * org.apache.ignite.internal.storage.engine.MvTableStorage#getOrCreateIndex > * > org.apache.ignite.internal.storage.engine.MvTableStorage#getOrCreateHashIndex > * > org.apache.ignite.internal.storage.engine.MvTableStorage#getOrCreateSortedIndex > and make new ones: > * org.apache.ignite.internal.storage.engine.MvTableStorage#createIndex > * org.apache.ignite.internal.storage.engine.MvTableStorage#getOrCreateIndex > * org.apache.ignite.internal.storage.engine.MvTableStorage#createHashIndex > * org.apache.ignite.internal.storage.engine.MvTableStorage#getHashIndex > * org.apache.ignite.internal.storage.engine.MvTableStorage#createSortedIndex > * org.apache.ignite.internal.storage.engine.MvTableStorage#getSortedIndex -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-18536) Schema synchronization feature and Catalog feature integration
[ https://issues.apache.org/jira/browse/IGNITE-18536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-18536: - Epic Link: IGNITE-19502 (was: IGNITE-17766) > Schema synchronization feature and Catalog feature integration > -- > > Key: IGNITE-18536 > URL: https://issues.apache.org/jira/browse/IGNITE-18536 > Project: Ignite > Issue Type: Improvement >Reporter: Ivan Bessonov >Priority: Major > Labels: ignite-3 > > TBD. > Metadata synchronization should be used, implemented (most likely) on top of > "transactions in the past", defined in the transactions IEP (see "Lock-free > RW Transactions" in > https://cwiki.apache.org/confluence/display/IGNITE/IEP-91:+Transaction+protocol). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19082) Describe Catalog operation flow in README and cleanup dead code.
[ https://issues.apache.org/jira/browse/IGNITE-19082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-19082: - Epic Link: IGNITE-19502 (was: IGNITE-17766) > Describe Catalog operation flow in README and cleanup dead code. > > > Key: IGNITE-19082 > URL: https://issues.apache.org/jira/browse/IGNITE-19082 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Andrey Mashenkov >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > Let's > * ensure Catalog is used by default as a single schema management point by > TableManager, IndexManager, SqlSchemaManager, SchemaRegistry. > * Describe CatalogService operations in README.md > * drop schema related code from configuration. > * drop outdated code from TableManager, IndexManager, SqlSchemaManager, > SchemaRegistry. > * make a PR for merging feature branch to main (if applicable). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-18537) Implement missed create/alter/drop Catalog operations
[ https://issues.apache.org/jira/browse/IGNITE-18537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-18537: - Epic Link: IGNITE-19502 (was: IGNITE-17766) > Implement missed create/alter/drop Catalog operations > - > > Key: IGNITE-18537 > URL: https://issues.apache.org/jira/browse/IGNITE-18537 > Project: Ignite > Issue Type: Improvement >Reporter: Ivan Bessonov >Priority: Major > Labels: ignite-3 > > Every CRETE/ALTER/DROP operation should trigger a new Catalog version. > We already have CREATE TABLE DDL operation implemented. Let's implement > others in similar way. > Please keep in mind that Distribution zones is out of scope now. -the > affinity is not the part of the schema definition. Data storage parameters > are not the part of it as well.- -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19081) Introduce Catalog events API.
[ https://issues.apache.org/jira/browse/IGNITE-19081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-19081: - Epic Link: IGNITE-19502 (was: IGNITE-17766) > Introduce Catalog events API. > - > > Key: IGNITE-19081 > URL: https://issues.apache.org/jira/browse/IGNITE-19081 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Andrey Mashenkov >Assignee: Andrey Mashenkov >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > Time Spent: 10m > Remaining Estimate: 0h > > Catalog service should trigger events when it's state is changed. > Other components should be able to subscribe on these events. > Let's > * extends CatalogService from Producer abstract class > * add events for supported DDL commands > * write tests on Catalog events > Note: Let's make these changes aside and avoid breaking the current > implementation. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-18535) Define new classes for versioned tables/indexes schemas
[
https://issues.apache.org/jira/browse/IGNITE-18535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Chugunov updated IGNITE-18535:
-
Epic Link: IGNITE-19502 (was: IGNITE-17766)
> Define new classes for versioned tables/indexes schemas
> ---
>
> Key: IGNITE-18535
> URL: https://issues.apache.org/jira/browse/IGNITE-18535
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Reporter: Ivan Bessonov
>Assignee: Andrey Mashenkov
>Priority: Major
> Labels: ignite-3
> Fix For: 3.0.0-beta2
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Current approach with schema management is faulty and can't support indexes.
> On top of that, it doesn't allow us to truly have multi-versioned historical
> data. Once the table is removed, it's removed for good, meaning that
> "current" RO transactions will not be able to finish. This is not acceptable.
> h3. Schema definitions
> What we need to have is the following:
> {code:java}
> SchemaDefinitions = map {version -> SchemaDefinition}
> SchemaDefinition = {timestamp, set {TableDefinition}, set{IndexDefinition}}
> TableDefinition = {name, id, array[ColumnDefinition], ...}
> IndexDefinition = {name, id, tableId, state, array[IdxColumnDefinition],
> ...}{code}
> Schema must be versioned, that's the first point. Well, it's already
> versioned in "main", here I mean the global versioning to tie everything to
> transactions and management of SQL indexes.
> Each definition correspond to a time period, where it represents the "actual"
> state of things. It must be used for RO queries, for example. RW transactions
> always use LATEST schema, obviously.
> Now, the meaning of defined values:
> * version - a simple auto-incrementing integer value;
> * "timestamp" - the schema is considered to be valid from this timestamp
> until the timestamp of "next" version (or "inifinity" if the next version
> doesn't yet exist);
> * most of tables and indexes properties are self-explanatory;
> * index state - RO or RW. We should differentiate the indexes that are not
> yet built frome indexes that are fully available.
> Currently, it's not too clear where to store this structure. The problem lies
> in the realm of metadata synchronization, that's not yet designed. But the
> thing is that all nodes must eventually have an up-to-date state and every
> data/index update must be consistent with the version that belongs to a
> current operation's timestamp.
> There are two likely candidates - Meta-Storage or Configuration. We'll figure
> it out later.
> h3. Seralization / storage
> It would be convenient to only store the oldest version + the collection of
> diffs. Every node would unpack that locally, but we would save a lot on the
> storage space in meta-storage in case when user has a lot of tables/indexes.
> This approach would also be beneficial for another reason: we need to know,
> what's changed between versions. It may be hard to calculate if all that we
> have are definitions themselves.
> h3. General thoughts
> This may be a good place to start using integer tableId and indexId more
> often. UUIDs are too much. What's good is that "serializability" of schemas
> gives us easy way of generating integer ids, just like it's don right now
> with configuration.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-16288) Async API for the configuration
[ https://issues.apache.org/jira/browse/IGNITE-16288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-16288: - Epic Link: (was: IGNITE-17766) > Async API for the configuration > --- > > Key: IGNITE-16288 > URL: https://issues.apache.org/jira/browse/IGNITE-16288 > Project: Ignite > Issue Type: Task >Reporter: Semyon Danilov >Priority: Major > Labels: ignite-3, tech-debt > Fix For: 3.0.0-beta2 > > > Currently, configuration API is blocking, which creates a limitation for a > call site -- we can't call configuration methods from network threads. E.g. > TableManager#tablesAsyncInternal supposed to be non-blocking, but it does use > configuration API, so it had to be used via CompletableFuture#supplyAsync. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-18747) Sql. Provide commands and handlers for alter index name operation
[ https://issues.apache.org/jira/browse/IGNITE-18747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-18747: - Epic Link: (was: IGNITE-17766) > Sql. Provide commands and handlers for alter index name operation > - > > Key: IGNITE-18747 > URL: https://issues.apache.org/jira/browse/IGNITE-18747 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Yury Gerzhedovich >Priority: Major > Labels: ignite-3 > > After implementing IGNITE-16196 need to provide DDL command and handler to > altering index name, as well as translation to the command from AST > representation. > As a result, we will be able to translate AST to a command (see > DdlSqlToCommandConverter) and execute this command in order to apply changes > to configuration (see DdlCommandHandler). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-16196) Sql. Extend SQL grammar with ALTER INDEX RENAME statement
[ https://issues.apache.org/jira/browse/IGNITE-16196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-16196: - Epic Link: (was: IGNITE-17766) > Sql. Extend SQL grammar with ALTER INDEX RENAME statement > - > > Key: IGNITE-16196 > URL: https://issues.apache.org/jira/browse/IGNITE-16196 > Project: Ignite > Issue Type: Improvement > Components: sql >Affects Versions: 3.0.0-alpha3 >Reporter: Taras Ledkov >Priority: Minor > Labels: ignite-3 > > As we can CREATE and DROP INDEX need to provide the ability to alter a name > of an index by DDL commands. > Let's extend SQL grammar with the following syntax: > ALTER INDEX [ IF EXISTS ] RENAME TO -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19501) SchemaManager should use CatalogService for building BinaryRow descriptors
[ https://issues.apache.org/jira/browse/IGNITE-19501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-19501: - Epic Link: IGNITE-19502 > SchemaManager should use CatalogService for building BinaryRow descriptors > -- > > Key: IGNITE-19501 > URL: https://issues.apache.org/jira/browse/IGNITE-19501 > Project: Ignite > Issue Type: Improvement >Reporter: Andrey Mashenkov >Priority: Major > Labels: ignite-3 > > As of now, SchemaManager uses configuration data to create > BinaryRow/BinaryTuple descriptors. > Let's make SchemaManager and SchemaRegistry using Catalog data instead. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19500) IndexManager should listen CatalogService events instead of configuration
[ https://issues.apache.org/jira/browse/IGNITE-19500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-19500: - Epic Link: IGNITE-19502 > IndexManager should listen CatalogService events instead of configuration > - > > Key: IGNITE-19500 > URL: https://issues.apache.org/jira/browse/IGNITE-19500 > Project: Ignite > Issue Type: Improvement >Reporter: Andrey Mashenkov >Priority: Major > Labels: ignite-3 > > As of now, IndexManager listens configuration events to create internal > structures. > Let's make IndexManager listens CatalogService events instead. > Note: Some tests may fails due to changed guarantees and related ticked > incompletion. So, let's do this in a separate feature branch. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19493) Sql. Query flow refactoring.
[ https://issues.apache.org/jira/browse/IGNITE-19493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-19493: - Epic Link: IGNITE-19502 > Sql. Query flow refactoring. > > > Key: IGNITE-19493 > URL: https://issues.apache.org/jira/browse/IGNITE-19493 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Andrey Mashenkov >Priority: Major > Labels: ignite-3 > > As of now, in SqlQueryProcessor.querySingle0(), we get SQL SchemaPlus from > SqlQueryManager, then create query context, then start implicit transaction > if needed. > SqlQueryManager doesn't aware of schema versions. > Let's > * add tx timestamp to the interface, it is ok to ignore the tx timestamp > parameter in current implementation unless a new one based on Catalog will be > implemented in IGNITE-19496. > * rework current flow to start implicit transaction first, then get a schema > based on Tx timestamp, then create a query context. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19499) TableManager should listen CatalogService events instead of configuration
[ https://issues.apache.org/jira/browse/IGNITE-19499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-19499: - Epic Link: IGNITE-19502 > TableManager should listen CatalogService events instead of configuration > - > > Key: IGNITE-19499 > URL: https://issues.apache.org/jira/browse/IGNITE-19499 > Project: Ignite > Issue Type: Improvement >Reporter: Andrey Mashenkov >Priority: Major > Labels: ignite-3 > > As of now, TableManager listens configuration events to create internal > structures. > Let's make TableManager listens CatalogService events instead. > Note: Some tests may fails due to changed guarantees and related ticked > incompletion. So, let's do this in a separate feature branch. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19497) Sql. Switch to SqlSchemaManager implementation on top of CatalogService
[ https://issues.apache.org/jira/browse/IGNITE-19497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-19497: - Epic Link: IGNITE-19502 > Sql. Switch to SqlSchemaManager implementation on top of CatalogService > > > Key: IGNITE-19497 > URL: https://issues.apache.org/jira/browse/IGNITE-19497 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Andrey Mashenkov >Priority: Major > Labels: ignite-3 > > Let's switch to a new SqlSchemaManager from IGNITE-19496, which uses Catalog > events, and drop old implementation. > Note: Some tests may fails due to changed guarantees and related ticked > incompletion. So, let's do this in a separate feature branch. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19493) Sql. Query flow refactoring.
[ https://issues.apache.org/jira/browse/IGNITE-19493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-19493: - Description: As of now, in SqlQueryProcessor.querySingle0(), we get SQL SchemaPlus from SqlQueryManager, then create query context, then start implicit transaction if needed. SqlQueryManager doesn't aware of schema versions. Let's * add tx timestamp to the interface, it is ok to ignore the tx timestamp parameter in current implementation unless a new one based on Catalog will be implemented in IGNITE-19496. * rework current flow to start implicit transaction first, then get a schema based on Tx timestamp, then create a query context. was: As of now, in SqlQueryProcessor.querySingle0(), we get SQL SchemaPlus from SqlQueryMaanger, then create query context, then start implicit transaction if needed. SqlQueryManager doesn't aware of schema versions. Let's * add tx timestamp to the interface, it is ok to ignore the tx timestamp parameter in current implementation unless a new one based on Catalog will be implemented in IGNITE-19496. * rework current flow to start implicit transaction first, then get a schema based on Tx timestamp, then create a query context. > Sql. Query flow refactoring. > > > Key: IGNITE-19493 > URL: https://issues.apache.org/jira/browse/IGNITE-19493 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Andrey Mashenkov >Priority: Major > Labels: ignite-3 > > As of now, in SqlQueryProcessor.querySingle0(), we get SQL SchemaPlus from > SqlQueryManager, then create query context, then start implicit transaction > if needed. > SqlQueryManager doesn't aware of schema versions. > Let's > * add tx timestamp to the interface, it is ok to ignore the tx timestamp > parameter in current implementation unless a new one based on Catalog will be > implemented in IGNITE-19496. > * rework current flow to start implicit transaction first, then get a schema > based on Tx timestamp, then create a query context. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19496) Sql. Implement SqlSchemaManager on top of CatalogService
[ https://issues.apache.org/jira/browse/IGNITE-19496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-19496: - Epic Link: IGNITE-19502 > Sql. Implement SqlSchemaManager on top of CatalogService > > > Key: IGNITE-19496 > URL: https://issues.apache.org/jira/browse/IGNITE-19496 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Andrey Mashenkov >Priority: Major > Labels: ignite-3 > > As of now, SqlSchemaManager listens configuration events to compose a new > schema, and does not support schema versioning. > Let's implement SqlSchemaManager on top of CatalogService, which is already > aware of schema versions. > The new implementation may be just a caching adapter for CatalogService, > which enriches Catalog with metadata that can be used by Calcite engine. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19483) Transform TableManager and IndexManager to internally work against Catalog event types
[ https://issues.apache.org/jira/browse/IGNITE-19483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-19483: - Epic Link: IGNITE-19502 > Transform TableManager and IndexManager to internally work against Catalog > event types > -- > > Key: IGNITE-19483 > URL: https://issues.apache.org/jira/browse/IGNITE-19483 > Project: Ignite > Issue Type: Improvement >Reporter: Roman Puchkovskiy >Assignee: Roman Puchkovskiy >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > Currently, when an event like 'table was added to configuration' happens, the > listener pools the table config by itself and then uses it to create the > table. > This should be changed: the table configuration object should be converted to > an object from Catalog domain and pushed to the listeners. > Same should be done to indices. > Requires investigation. > Also, we need to stop passing configuration to deeply-nested components (like > storages). Also requires investigation. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19482) Add a method to InternalTransaction for getting initial schema timestamp
[ https://issues.apache.org/jira/browse/IGNITE-19482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-19482: - Epic Link: IGNITE-19502 > Add a method to InternalTransaction for getting initial schema timestamp > > > Key: IGNITE-19482 > URL: https://issues.apache.org/jira/browse/IGNITE-19482 > Project: Ignite > Issue Type: Improvement >Reporter: Roman Puchkovskiy >Assignee: Roman Puchkovskiy >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > The method must return readTs for an RO transaction and beginTs for an RW > transaction. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19502) Separate component to keep database schema and manage changes in it
[ https://issues.apache.org/jira/browse/IGNITE-19502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-19502: - Description: Right now database schema information in Ignite 3.0 is stored and treated as regular configuration. All business processes related to schema changes are distributed via internal mechanisms of configuration engine and orchestrated ad-hoc by consumer components like TableManager or IndexManager. But there is a strong evidence that API and guarantees of configuration engine don't satisfy important requirements that emerge in the field of db schema management. So we need to stop treating schema as any other configuration and move it to a separate component called Catalog Service. This component will provide high-level API to modify the schema and will take responsibility for all other necessary processes like synchronizing schema changes between nodes, managing schema change events, notifying all necessary components and providing orchestration between them. was: Right now database schema information in Ignite 3.0 is stored and treated as regular configuration. All business processes related to schema changes are distributed via internal mechanisms of configuration engine and orchestrated ad-hoc by consumer components like TableManager or IndexManager. But there is a strong evidence that API and guarantees of configuration engine don't satisfy important requirements that emerge in the field of db schema management. So we need to stop treating schema as any other configuration and move it to a separate component called Catalog Service. This component will provide high-level API to modify the schema and take responsibility for all other necessary processes like synchronizing schema changes between nodes, managing schema change events, notifying all necessary components and providing orchestration between them. > Separate component to keep database schema and manage changes in it > --- > > Key: IGNITE-19502 > URL: https://issues.apache.org/jira/browse/IGNITE-19502 > Project: Ignite > Issue Type: Epic > Components: persistence, sql >Reporter: Sergey Chugunov >Priority: Major > Labels: ignite-3 > > Right now database schema information in Ignite 3.0 is stored and treated as > regular configuration. All business processes related to schema changes are > distributed via internal mechanisms of configuration engine and orchestrated > ad-hoc by consumer components like TableManager or IndexManager. > But there is a strong evidence that API and guarantees of configuration > engine don't satisfy important requirements that emerge in the field of db > schema management. > So we need to stop treating schema as any other configuration and move it to > a separate component called Catalog Service. > This component will provide high-level API to modify the schema and will take > responsibility for all other necessary processes like synchronizing schema > changes between nodes, managing schema change events, notifying all necessary > components and providing orchestration between them. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-19502) Separate component to keep database schema and manage changes in it
Sergey Chugunov created IGNITE-19502: Summary: Separate component to keep database schema and manage changes in it Key: IGNITE-19502 URL: https://issues.apache.org/jira/browse/IGNITE-19502 Project: Ignite Issue Type: Epic Components: persistence, sql Reporter: Sergey Chugunov Right now database schema information in Ignite 3.0 is stored and treated as regular configuration. All business processes related to schema changes are distributed via internal mechanisms of configuration engine and orchestrated ad-hoc by consumer components like TableManager or IndexManager. But there is a strong evidence that API and guarantees of configuration engine don't satisfy important requirements that emerge in the field of db schema management. So we need to stop treating schema as any other configuration and move it to a separate component called Catalog Service. This component will provide high-level API to modify the schema and take responsibility for all other necessary processes like synchronizing schema changes between nodes, managing schema change events, notifying all necessary components and providing orchestration between them. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19377) Remove conditional MetaStorage raft commands
[ https://issues.apache.org/jira/browse/IGNITE-19377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-19377: - Epic Link: IGNITE-18733 (was: IGNITE-17766) > Remove conditional MetaStorage raft commands > > > Key: IGNITE-19377 > URL: https://issues.apache.org/jira/browse/IGNITE-19377 > Project: Ignite > Issue Type: Improvement >Reporter: Semyon Danilov >Priority: Major > Labels: ignite-3 > > When we have replica-based MetaStorage, we should not replicate conditional > commands like IfThenElse. Replica’s leader should decide on the command’s > output and replicate simple Insert commands. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19376) Replica-based MetaStorage
[ https://issues.apache.org/jira/browse/IGNITE-19376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-19376: - Epic Link: IGNITE-18733 (was: IGNITE-17766) > Replica-based MetaStorage > - > > Key: IGNITE-19376 > URL: https://issues.apache.org/jira/browse/IGNITE-19376 > Project: Ignite > Issue Type: Improvement >Reporter: Semyon Danilov >Priority: Major > Labels: ignite-3 > > Right now any node can submit a command to the MetaStorage raft group. This > approach must be changed to the approach we have with partitions: replicas > with a primary replica that actually submits commands to the (possibly > non-colocated) raft group leader. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19376) Replica-based MetaStorage
[ https://issues.apache.org/jira/browse/IGNITE-19376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-19376: - Ignite Flags: (was: Docs Required,Release Notes Required) > Replica-based MetaStorage > - > > Key: IGNITE-19376 > URL: https://issues.apache.org/jira/browse/IGNITE-19376 > Project: Ignite > Issue Type: Improvement >Reporter: Semyon Danilov >Priority: Major > Labels: ignite-3 > > Right now any node can submit a command to the MetaStorage raft group. This > approach must be changed to the approach we have with partitions: replicas > with a primary replica that actually submits commands to the (possibly > non-colocated) raft group leader. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19377) Remove conditional MetaStorage raft commands
[ https://issues.apache.org/jira/browse/IGNITE-19377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-19377: - Epic Link: IGNITE-17766 Ignite Flags: (was: Docs Required,Release Notes Required) > Remove conditional MetaStorage raft commands > > > Key: IGNITE-19377 > URL: https://issues.apache.org/jira/browse/IGNITE-19377 > Project: Ignite > Issue Type: Improvement >Reporter: Semyon Danilov >Priority: Major > Labels: ignite-3 > > When we have replica-based MetaStorage, we should not replicate conditional > commands like IfThenElse. Replica’s leader should decide on the command’s > output and replicate simple Insert commands. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19377) Remove conditional MetaStorage raft commands
[ https://issues.apache.org/jira/browse/IGNITE-19377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-19377: - Labels: ignite-3 (was: ) > Remove conditional MetaStorage raft commands > > > Key: IGNITE-19377 > URL: https://issues.apache.org/jira/browse/IGNITE-19377 > Project: Ignite > Issue Type: Improvement >Reporter: Semyon Danilov >Priority: Major > Labels: ignite-3 > > When we have replica-based MetaStorage, we should not replicate conditional > commands like IfThenElse. Replica’s leader should decide on the command’s > output and replicate simple Insert commands. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19376) Replica-based MetaStorage
[ https://issues.apache.org/jira/browse/IGNITE-19376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-19376: - Epic Link: IGNITE-17766 > Replica-based MetaStorage > - > > Key: IGNITE-19376 > URL: https://issues.apache.org/jira/browse/IGNITE-19376 > Project: Ignite > Issue Type: Improvement >Reporter: Semyon Danilov >Priority: Major > > Right now any node can submit a command to the MetaStorage raft group. This > approach must be changed to the approach we have with partitions: replicas > with a primary replica that actually submits commands to the (possibly > non-colocated) raft group leader. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19376) Replica-based MetaStorage
[ https://issues.apache.org/jira/browse/IGNITE-19376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-19376: - Labels: ignite-3 (was: ) > Replica-based MetaStorage > - > > Key: IGNITE-19376 > URL: https://issues.apache.org/jira/browse/IGNITE-19376 > Project: Ignite > Issue Type: Improvement >Reporter: Semyon Danilov >Priority: Major > Labels: ignite-3 > > Right now any node can submit a command to the MetaStorage raft group. This > approach must be changed to the approach we have with partitions: replicas > with a primary replica that actually submits commands to the (possibly > non-colocated) raft group leader. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-18781) Strange ports used - Failed to process selector key
[ https://issues.apache.org/jira/browse/IGNITE-18781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17706339#comment-17706339 ] Sergey Chugunov commented on IGNITE-18781: -- [~ct7] , from networking point of view presence of seemingly random ports in logs is totally fine. When you configure ports for discovery and communication components of Ignite you only specify the ports these components will be *listening* to for any *incoming* connections. When a node decides to open an *outgoing* connection it opens it using any port it can establish a connection through. And there is no configuration to specify a port or a port range that should be used when opening outgoing connections. The root cause of the issue may be too restrictive rules set in the firewall or something else. > Strange ports used - Failed to process selector key > --- > > Key: IGNITE-18781 > URL: https://issues.apache.org/jira/browse/IGNITE-18781 > Project: Ignite > Issue Type: Bug > Components: networking >Affects Versions: 2.14 >Reporter: Christopher Tenter >Priority: Major > Attachments: ignitelog.txt > > > A two-node cluster occasionally cannot communicate when ignite uses a strange > local or remote port. We've opened all ports that are documented here > [https://ignite.apache.org/docs/latest/clustering/network-configuration] > > However, ignite uses a seemingly random port for connecting to the other node > sometimes. By default all ports are closed in the firewall, so the connection > fails with message "Failed to process selector key" > > Here are some connections that ignite attempts: > locAddr=/10.125.100.93:20395, rmtAddr=/10.125.100.179:47100 > locAddr=/10.125.100.93:10800, rmtAddr=/10.125.100.93:1851 > locAddr=/10.125.100.93:10800, rmtAddr=/10.125.100.93:1775 > locAddr=/10.125.100.93:1323, rmtAddr=/10.125.100.179:47100 > > So ports 20395, 1851, 1775, 1323 are used, but they are closed in the > firewall. The ports change all the time. If I add them to the firewall, it > doesn't take long until another unknown one pops up. I could not find them > anywhere in the ignite documentation. Where do they come from and how to fix > it? > > The network settings in the server config looks like this: > > > > class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder"> > > > 10.125.100.93 > 10.125.100.179 > > > > > > > > > class="org.apache.ignite.configuration.TransactionConfiguration"> > > > > > > class="org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi"> > > > > > > Full log output of the "Failed to process selector key" error attached. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-18715) B+Tree corruption error caused Ignite cluster crash and not able restart
[
https://issues.apache.org/jira/browse/IGNITE-18715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17697408#comment-17697408
]
Sergey Chugunov commented on IGNITE-18715:
--
[~nifeng2xing] , do you have a reproducer for this problem? From the stack
trace provided in the ticket it is clear that corruption has happened during
index manipulations:
org.apache.ignite.internal.cache.{*}query.index.sorted.inline.InlineIndexTree{*}.
However it is not enough to narrow down the exact scenario, a reproducer is
needed or at least persistence artifacts - partition files, index.bin file, WAL
segments - that were on disk after node crash.
At the same time it should be possible to fix the issue by deleting index.bin
file for the affected cache - in that case node should be able to start again.
It will need to rebuild all secondary indexes that are stored in index.bin
file, but restoring normal operations for that node should be possible in
theory.
> B+Tree corruption error caused Ignite cluster crash and not able restart
>
>
> Key: IGNITE-18715
> URL: https://issues.apache.org/jira/browse/IGNITE-18715
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Affects Versions: 2.14
>Reporter: Isaac Zhu
>Priority: Blocker
>
> With version 2.14, see this error during doing cache remove & put. And after
> this happens, the cluster can't be restarted, all data get lost:
> [00:48:21,922][SEVERE][sys-stripe-8-#9][] Critical system error detected.
> Will be handled accordingly to configured handler
> [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0,
> super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet
> [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]],
> failureCtx=FailureContext [type=CRITICAL_ERROR, err=class
> o.a.i.i.processors.cache.persistence.tree.CorruptedTreeException: B+Tree is
> corrupted [groupId=828437433, pageIds=[217017202749409008],
> cacheId=-595580467, cacheName=SQL_PUBLIC_URT_TESTCASE_RESULTS_VCS,
> indexName=URT_TESTCASE_RESULTS_VCS_STATUS_IDX, groupName=nav_mem_part,
> msg=Runtime failure on search row: Row@7b8be26c[ key:
> SQL_PUBLIC_URT_TESTCASE_RESULTS_VCS_070eda86_0aab_4da3_900c_8d3baf08b3a7_KEY
> [idHash=1221100184, hash=701595465, TEST_CASE_ID=610062, GROUP_ID=497], val:
> SQL_PUBLIC_URT_TESTCASE_RESULTS_VCS_070eda86_0aab_4da3_900c_8d3baf08b3a7
> [idHash=1823360128, hash=-882680090, NEW=null, STATUS=MARK_FOR_DELETION,
> DIFF_FILE_ID=610, EXEC_TIME=67, TIMED_OUT=null, RECORD_TIME=2023-02-05
> 20:26:30.234, FILE_MOD_TIME=2023-02-02 01:35:59.0, SINCE=2022-10-22
> 00:00:00.0, JIRA=, ERROR_TOOL=null, ERROR_CODE=null, BUILD_DATE=2023-02-01
> 20:00:00.0, PRODUCER_START_TIME=2023-02-03 00:10:41.0] ][ MARK_FOR_DELETION,
> 497, 610062 class
> org.apache.ignite.internal.processors.cache.persistence.tree.CorruptedTreeException:
> B+Tree is corrupted [groupId=828437433, pageIds=[217017202749409008],
> cacheId=-595580467, cacheName=SQL_PUBLIC_URT_TESTCASE_RESULTS_VCS,
> indexName=URT_TESTCASE_RESULTS_VCS_STATUS_IDX, groupName=nav_mem_part,
> msg=Runtime failure on search row: Row@7b8be26c[ key:
> SQL_PUBLIC_URT_TESTCASE_RESULTS_VCS_070eda86_0aab_4da3_900c_8d3baf08b3a7_KEY
> [idHash=1221100184, hash=701595465, TEST_CASE_ID=610062, GROUP_ID=497], val:
> SQL_PUBLIC_URT_TESTCASE_RESULTS_VCS_070eda86_0aab_4da3_900c_8d3baf08b3a7
> [idHash=1823360128, hash=-882680090, NEW=null, STATUS=MARK_FOR_DELETION,
> DIFF_FILE_ID=610, EXEC_TIME=67, TIMED_OUT=null, RECORD_TIME=2023-02-05
> 20:26:30.234, FILE_MOD_TIME=2023-02-02 01:35:59.0, SINCE=2022-10-22
> 00:00:00.0, JIRA=, ERROR_TOOL=null, ERROR_CODE=null, BUILD_DATE=2023-02-01
> 20:00:00.0, PRODUCER_START_TIME=2023-02-03 00:10:41.0] ][ MARK_FOR_DELETION,
> 497, 610062 ]] at
> org.apache.ignite.internal.cache.query.index.sorted.inline.InlineIndexTree.corruptedTreeException(InlineIndexTree.java:561)
> at
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.doRemove(BPlusTree.java:2310)
> at
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.removex(BPlusTree.java:2079)
> at
> org.apache.ignite.internal.cache.query.index.sorted.inline.InlineIndexImpl.remove(InlineIndexImpl.java:377)
> at
> org.apache.ignite.internal.cache.query.index.sorted.inline.InlineIndexImpl.onUpdate(InlineIndexImpl.java:330)
> at
> org.apache.ignite.internal.cache.query.index.IndexProcessor.updateIndex(IndexProcessor.java:465)
> at
> org.apache.ignite.internal.cache.query.index.IndexProcessor.updateIndexes(IndexProcessor.java:308)
> at
> org.apache.ignite.internal.cache.query.index.IndexProcessor.store(IndexProcessor.java:156)
> at
> org.apache.ignite.internal.processors.query.GridQueryProcessor.store(GridQueryProcesso
[jira] [Commented] (IGNITE-18905) Tool to read WAL content
[ https://issues.apache.org/jira/browse/IGNITE-18905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17697381#comment-17697381 ] Sergey Chugunov commented on IGNITE-18905: -- [~nizhikov] , I'm a bit confused about the status of this ticket. According to Jira its status is Open, but for some reason there is already a Pull Request with some implementation. It seems that it should be at least in In Progress or even in Patch Available. > Tool to read WAL content > > > Key: IGNITE-18905 > URL: https://issues.apache.org/jira/browse/IGNITE-18905 > Project: Ignite > Issue Type: Improvement >Reporter: Nikolay Izhikov >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > During post mortem, fixing production or debuggin it can be usefull to have > tool to output WAL records to log. > Ignite should provide tool to do it. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-18904) The ignite is suspended during startup
[ https://issues.apache.org/jira/browse/IGNITE-18904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17697380#comment-17697380 ] Sergey Chugunov commented on IGNITE-18904: -- [~bdq] , could you please share more context of the problem? Exact cluster configuration (how many nodes were around, their configuration etc), logs from all involved nodes and so on. >From what I see in this stack trace the node is waiting to join topology, it >is not clear why join request isn't properly handled or why node doesn't >recognize that it is the first one in the cluster. > The ignite is suspended during startup > -- > > Key: IGNITE-18904 > URL: https://issues.apache.org/jira/browse/IGNITE-18904 > Project: Ignite > Issue Type: Bug >Reporter: biandeqiang >Priority: Critical > > ignite keeps waiting when it start > > "fm.ignite.monitor-69-1" #285 prio=5 os_prio=0 cpu=35865.92ms > elapsed=83153.67s tid=0x55a415a850e0 nid=0x17f1c in Object.wait() > [0x7f3cc5fd3000] > java.lang.Thread.State: TIMED_WAITING (on object monitor) > at java.lang.Object.wait(Native Method) > at > org.apache.ignite.spi.discovery.tcp.ServerImpl.joinTopology(ServerImpl.java:1186) > - locked <0x0006f3b3f140> (a java.lang.Object) > at > org.apache.ignite.spi.discovery.tcp.ServerImpl.spiStart(ServerImpl.java:472) > at > org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.spiStart(TcpDiscoverySpi.java:2210) > at > org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:278) > at > org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:1089) > at > org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1766) > at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1147) > at > org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:1799) > at > org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1721) > - locked <0x0006924b2580> (a > org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance) > at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1160) > at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:656) > at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:578) > at org.apache.ignite.Ignition.start(Ignition.java:328) > at com.huawei.cloudsop.ignite.IgniteManager.start(IgniteManager.java:56) > at > com.huawei.iemp.fmdataservice.monitor.ignite.AlarmIgnite.getOrCreateCache(AlarmIgnite.java:115) > at > com.huawei.iemp.fmdataservice.monitor.ignite.dao.CurrentAlarmDaoImpl.startIgnite(CurrentAlarmDaoImpl.java:130) > at > com.huawei.iemp.fmdataservice.monitor.ignite.dao.IgniteMonitorTask.reStartIgnite(IgniteMonitorTask.java:75) > at > com.huawei.iemp.fmdataservice.monitor.ignite.dao.IgniteMonitorTask.run(IgniteMonitorTask.java:60) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:750) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (IGNITE-16199) Implements index build/rebuild
[ https://issues.apache.org/jira/browse/IGNITE-16199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17697366#comment-17697366 ] Sergey Chugunov edited comment on IGNITE-16199 at 3/7/23 11:05 AM: --- Ticket is clarified by others created later: IGNITE-18539, IGNITE-18595. was (Author: sergeychugunov): Ticket is clarified by others created later: [https://issues.apache.org/jira/browse/IGNITE-18539|IGNITE-18539], [https://issues.apache.org/jira/browse/IGNITE-18595|IGNITE-18595] > Implements index build/rebuild > --- > > Key: IGNITE-16199 > URL: https://issues.apache.org/jira/browse/IGNITE-16199 > Project: Ignite > Issue Type: Improvement > Components: sql >Affects Versions: 3.0.0-alpha3 >Reporter: Taras Ledkov >Priority: Major > Labels: ignite-3 > > Index must be built on exists table data: scan table's data and build an > index. > Now only update index by table updates is implemented. > May be build and rebuild tasks may be split. -- This message was sent by Atlassian Jira (v8.20.10#820010)
