[jira] [Updated] (IGNITE-21613) Make CheckpointManagerTest run on Java 21
[ https://issues.apache.org/jira/browse/IGNITE-21613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Puchkovskiy updated IGNITE-21613: --- Description: The test mocks ByteBuffer class, this does not work on Java 21. We can try to switch to spying instead of mocking. (was: The test mocks ByteBuffer class, which does not work on Java 21. We can try to switch to spying instead of mocking.) > Make CheckpointManagerTest run on Java 21 > - > > Key: IGNITE-21613 > URL: https://issues.apache.org/jira/browse/IGNITE-21613 > Project: Ignite > Issue Type: Improvement >Reporter: Roman Puchkovskiy >Assignee: Roman Puchkovskiy >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > The test mocks ByteBuffer class, this does not work on Java 21. We can try to > switch to spying instead of mocking. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-21613) Make CheckpointManagerTest run on Java 21
Roman Puchkovskiy created IGNITE-21613: -- Summary: Make CheckpointManagerTest run on Java 21 Key: IGNITE-21613 URL: https://issues.apache.org/jira/browse/IGNITE-21613 Project: Ignite Issue Type: Improvement Reporter: Roman Puchkovskiy Assignee: Roman Puchkovskiy Fix For: 3.0.0-beta2 The test mocks ByteBuffer class, which does not work on Java 21. We can try to switch to spying instead of mocking. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-21591) Add tuple updrades during index backfill process
[ https://issues.apache.org/jira/browse/IGNITE-21591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17820984#comment-17820984 ] Roman Puchkovskiy commented on IGNITE-21591: The patch looks good to me > Add tuple updrades during index backfill process > > > Key: IGNITE-21591 > URL: https://issues.apache.org/jira/browse/IGNITE-21591 > Project: Ignite > Issue Type: Improvement >Reporter: Kirill Tkalenko >Assignee: Kirill Tkalenko >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > Implement point 10 of ticket IGNITE-20117. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IGNITE-21311) Sql. Partition pruning. Introduce pruning for correlated scans
[ https://issues.apache.org/jira/browse/IGNITE-21311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maksim Zhuravkov reassigned IGNITE-21311: - Assignee: Maksim Zhuravkov > Sql. Partition pruning. Introduce pruning for correlated scans > -- > > Key: IGNITE-21311 > URL: https://issues.apache.org/jira/browse/IGNITE-21311 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Konstantin Orlov >Assignee: Maksim Zhuravkov >Priority: Major > Labels: ignite-3 > > Apart of static pruning made on preparation step, we may also introduce > dynamic pruning for correlated scans. > In case of correlated join, a predicate on the right shoulder should be > evaluated based on the context provided from the left shoulder. This context > is not known until runtime, those partitions are not trimmed on preparation > step. However, this very case doesn't differ match from the static pruning: > the only difference here is a time when pruning should be applied. > In order to support dynamic pruning, we should save pruning meta during > planning phase in a scan node. Later, in runtime, we should evaluate pruning > function in order to derive partitions satisfying the predicate. Those > partitions should be used to do a lookup into a table. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-21595) .NET: Update static analysis packages and fix new inspections
[ https://issues.apache.org/jira/browse/IGNITE-21595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Tupitsyn updated IGNITE-21595: Description: * Bump Microsoft.CodeAnalysis.NetAnalyzers from 6.0.0 to 8.0.0 * Bump Microsoft.CodeAnalysis.CSharp and Microsoft.CodeAnalysis.Analyzers * Bump StyleCop.Analyzers from 1.2.0-beta.435 to 1.2.0-beta.556 * Bump NUnit.Analyzers from 3.6.1 to 3.10.0 * Bump Microsoft.Extensions.Logging.Console and Microsoft.Extensions.Logging.Abstractions to 8.0.0 Dependabot PRs for those have failing builds because new inspections trigger warnings and errors: https://github.com/apache/ignite-3/pull/3256, https://github.com/apache/ignite-3/pull/3271, https://github.com/apache/ignite-3/pull/3268, https://github.com/apache/ignite-3/pull/3294 was: * Bump Microsoft.CodeAnalysis.NetAnalyzers from 6.0.0 to 8.0.0 * Bump Microsoft.CodeAnalysis.CSharp and Microsoft.CodeAnalysis.Analyzers * Bump StyleCop.Analyzers from 1.2.0-beta.435 to 1.2.0-beta.556 * Bump NUnit.Analyzers from 3.6.1 to 3.10.0 Dependabot PRs for those have failing builds because new inspections trigger warnings and errors: https://github.com/apache/ignite-3/pull/3256, https://github.com/apache/ignite-3/pull/3271, https://github.com/apache/ignite-3/pull/3268, https://github.com/apache/ignite-3/pull/3294 > .NET: Update static analysis packages and fix new inspections > - > > Key: IGNITE-21595 > URL: https://issues.apache.org/jira/browse/IGNITE-21595 > Project: Ignite > Issue Type: Improvement > Components: platforms >Affects Versions: 3.0.0-beta1 >Reporter: Pavel Tupitsyn >Assignee: Pavel Tupitsyn >Priority: Minor > Labels: .NET, ignite-3 > Fix For: 3.0.0-beta2 > > > * Bump Microsoft.CodeAnalysis.NetAnalyzers from 6.0.0 to 8.0.0 > * Bump Microsoft.CodeAnalysis.CSharp and Microsoft.CodeAnalysis.Analyzers > * Bump StyleCop.Analyzers from 1.2.0-beta.435 to 1.2.0-beta.556 > * Bump NUnit.Analyzers from 3.6.1 to 3.10.0 > * Bump Microsoft.Extensions.Logging.Console and > Microsoft.Extensions.Logging.Abstractions to 8.0.0 > Dependabot PRs for those have failing builds because new inspections trigger > warnings and errors: > https://github.com/apache/ignite-3/pull/3256, > https://github.com/apache/ignite-3/pull/3271, > https://github.com/apache/ignite-3/pull/3268, > https://github.com/apache/ignite-3/pull/3294 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-21595) .NET: Update static analysis packages and fix new inspections
[ https://issues.apache.org/jira/browse/IGNITE-21595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Tupitsyn updated IGNITE-21595: Description: * Bump Microsoft.CodeAnalysis.NetAnalyzers from 6.0.0 to 8.0.0 * Bump Microsoft.CodeAnalysis.CSharp and Microsoft.CodeAnalysis.Analyzers * Bump StyleCop.Analyzers from 1.2.0-beta.435 to 1.2.0-beta.556 * Bump NUnit.Analyzers from 3.6.1 to 3.10.0 Dependabot PRs for those have failing builds because new inspections trigger warnings and errors: https://github.com/apache/ignite-3/pull/3256, https://github.com/apache/ignite-3/pull/3271, https://github.com/apache/ignite-3/pull/3268, https://github.com/apache/ignite-3/pull/3294 was: * Bump Microsoft.CodeAnalysis.NetAnalyzers from 6.0.0 to 8.0.0 * Bump StyleCop.Analyzers from 1.2.0-beta.435 to 1.2.0-beta.556 * Bump NUnit.Analyzers from 3.6.1 to 3.10.0 Dependabot PRs for those have failing builds because new inspections trigger warnings and errors: https://github.com/apache/ignite-3/pull/3256, https://github.com/apache/ignite-3/pull/3271, https://github.com/apache/ignite-3/pull/3268 > .NET: Update static analysis packages and fix new inspections > - > > Key: IGNITE-21595 > URL: https://issues.apache.org/jira/browse/IGNITE-21595 > Project: Ignite > Issue Type: Improvement > Components: platforms >Affects Versions: 3.0.0-beta1 >Reporter: Pavel Tupitsyn >Assignee: Pavel Tupitsyn >Priority: Minor > Labels: .NET, ignite-3 > Fix For: 3.0.0-beta2 > > > * Bump Microsoft.CodeAnalysis.NetAnalyzers from 6.0.0 to 8.0.0 > * Bump Microsoft.CodeAnalysis.CSharp and Microsoft.CodeAnalysis.Analyzers > * Bump StyleCop.Analyzers from 1.2.0-beta.435 to 1.2.0-beta.556 > * Bump NUnit.Analyzers from 3.6.1 to 3.10.0 > Dependabot PRs for those have failing builds because new inspections trigger > warnings and errors: > https://github.com/apache/ignite-3/pull/3256, > https://github.com/apache/ignite-3/pull/3271, > https://github.com/apache/ignite-3/pull/3268, > https://github.com/apache/ignite-3/pull/3294 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-21612) Send LWM of sender on full state transfer
Kirill Tkalenko created IGNITE-21612: Summary: Send LWM of sender on full state transfer Key: IGNITE-21612 URL: https://issues.apache.org/jira/browse/IGNITE-21612 Project: Ignite Issue Type: Improvement Reporter: Kirill Tkalenko For consistency, we need to get a LWM on a full state transfer and apply it locally (if it hasn't reached it, of course). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-21611) Sending in a snapshot for BUILDING indexes the next RowId to build on a full state transfer
Kirill Tkalenko created IGNITE-21611: Summary: Sending in a snapshot for BUILDING indexes the next RowId to build on a full state transfer Key: IGNITE-21611 URL: https://issues.apache.org/jira/browse/IGNITE-21611 Project: Ignite Issue Type: Improvement Reporter: Kirill Tkalenko For consistency, we need to send in a snapshot for all indexes in status BUILDING their next *RowId* to build (*org.apache.ignite.internal.storage.index.IndexStorage#getNextRowIdToBuild*). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-21610) Upgrade tuples when inserting into indexes on a full state transfer
Kirill Tkalenko created IGNITE-21610: Summary: Upgrade tuples when inserting into indexes on a full state transfer Key: IGNITE-21610 URL: https://issues.apache.org/jira/browse/IGNITE-21610 Project: Ignite Issue Type: Improvement Reporter: Kirill Tkalenko By analogy with IGNITE-21591, we need to update tuples when inserting into the index on a full transfer state. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-21607) Code cleanup
[ https://issues.apache.org/jira/browse/IGNITE-21607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17820840#comment-17820840 ] Ignite TC Bot commented on IGNITE-21607: {panel:title=Branch: [pull/11257/head] Base: [master] : No blockers found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel} {panel:title=Branch: [pull/11257/head] Base: [master] : No new tests found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}{panel} [TeamCity *-- Run :: All* Results|https://ci2.ignite.apache.org/viewLog.html?buildId=7760808buildTypeId=IgniteTests24Java8_RunAll] > Code cleanup > > > Key: IGNITE-21607 > URL: https://issues.apache.org/jira/browse/IGNITE-21607 > Project: Ignite > Issue Type: Improvement >Reporter: Vladimir Steshin >Assignee: Vladimir Steshin >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > Fixing of typos, abbreviations, unused code and untyped generics. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-21602) Fix direct buffers allocation on Java 21
[ https://issues.apache.org/jira/browse/IGNITE-21602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17820800#comment-17820800 ] Roman Puchkovskiy commented on IGNITE-21602: Thanks! > Fix direct buffers allocation on Java 21 > > > Key: IGNITE-21602 > URL: https://issues.apache.org/jira/browse/IGNITE-21602 > Project: Ignite > Issue Type: Improvement >Reporter: Roman Puchkovskiy >Assignee: Roman Puchkovskiy >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > Time Spent: 0.5h > Remaining Estimate: 0h > > In Java 21, the signature of the constructor of DirectByteBuffer we use was > changed: now length is a long, not an int. We need to support this new > signature, still using the old one with older versions of Java. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-21603) Incorrect backward connection check with loopback address.
[ https://issues.apache.org/jira/browse/IGNITE-21603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Steshin updated IGNITE-21603: -- Summary: Incorrect backward connection check with loopback address. (was: Incorrect backward connection chech with loopback address.) > Incorrect backward connection check with loopback address. > -- > > Key: IGNITE-21603 > URL: https://issues.apache.org/jira/browse/IGNITE-21603 > Project: Ignite > Issue Type: Bug >Reporter: Vladimir Steshin >Assignee: Vladimir Steshin >Priority: Minor > Labels: ise > Time Spent: 10m > Remaining Estimate: 0h > > We may skip backward connection check to a previous node if it has the same > loopback address as the current node. > Consider: > # Neither _IgniteConfiguration#setLocalHost()_ or > _TcpDiscoverySpi#setLocalAddress()_ is set. Or the localhost parameter is set > to "_0.0.0.0_". > # Nodes start on different hosts. All the available host addresses are > resolved and > # Among the other addresses, all nodes get the loopback address > "127.0.0.1:47500" (47500 is the default tcp discovery port). > # Cluster starts and works. But > # Some node N (A) decides the connection to node N+1 (B) is lost and tries to > connect to node N+2 (C) and sends _TcpDiscoveryHandshakeRequest_. > # Before C accepts incoming A's connection, it decides to check B and pings > it with _ServerImpl#checkConnection(List addrs, int > timeout)_ > # Around here, the network is restored, and A can now connect to B anew. > # "_127.0.0.1:47500_" is last in _List_ addrs by > _IgniteUtils#inetAddressesComparator(boolean sameHost)_. But the connect > attempts in _checkConnection(...)_ are parallel. "_127.0.0.1:47500_" answers > first. > # C sees it can connect to "_127.0.0.1:47500_" and chooses it as the alive > address of B. Other pings to rest of B's addresses are ignored. > # But "_127.0.0.1:47500_" is one of C's addresses. C realizes it pinged > itself and decides that B is not reachable: > {code:java} > // If local node was able to connect to previous, confirm that it's > alive. > ok = liveAddr != null && (!liveAddr.getAddress().isLoopbackAddress() > || !locNode.socketAddresses().contains(liveAddr)); > {code} > # C accepts connection from A and answers with > _TcpDiscoveryHandshakeResponse#previousNodeAlive() == false_ > # But B is ok now. But A connects to C and B is kicked from the ring. > The problem is that C ping itself by B's address "_127.0.0.1:47500_" -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-21602) Fix direct buffers allocation on Java 21
[ https://issues.apache.org/jira/browse/IGNITE-21602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17820786#comment-17820786 ] Kirill Tkalenko commented on IGNITE-21602: -- Looks good. > Fix direct buffers allocation on Java 21 > > > Key: IGNITE-21602 > URL: https://issues.apache.org/jira/browse/IGNITE-21602 > Project: Ignite > Issue Type: Improvement >Reporter: Roman Puchkovskiy >Assignee: Roman Puchkovskiy >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > Time Spent: 0.5h > Remaining Estimate: 0h > > In Java 21, the signature of the constructor of DirectByteBuffer we use was > changed: now length is a long, not an int. We need to support this new > signature, still using the old one with older versions of Java. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IGNITE-21281) Sql. Partition pruning. Integrate static partition pruning into MODIFY statements execution pipeline.
[ https://issues.apache.org/jira/browse/IGNITE-21281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Evgeny Stanilovsky reassigned IGNITE-21281: --- Assignee: Evgeny Stanilovsky > Sql. Partition pruning. Integrate static partition pruning into MODIFY > statements execution pipeline. > - > > Key: IGNITE-21281 > URL: https://issues.apache.org/jira/browse/IGNITE-21281 > Project: Ignite > Issue Type: Improvement > Components: sql >Affects Versions: 3.0.0-beta2 >Reporter: Maksim Zhuravkov >Assignee: Evgeny Stanilovsky >Priority: Major > Labels: ignite-3 > > Update PartitionExtractor introduced in > https://issues.apache.org/jira/browse/IGNITE-21277 to compile pruning > metadata for DML operations. > In order to that we need to extract values of colocation key columns from > input operators to ModifyNodes: > - For operations that do accept scan operation (e.g. plain INSERT), such > information is included in Values and Projection operators (to get values for > DEFAULT expression). > - For that accept scan operation, we should use pruning metadata provided by > a scan operator. > *Some examples* > Plain insert: > {code:java} > INSERT INTO t (colo_col1, colo_col2) VALUES (1, 2), (3, 4) > Partition pruning metadata: t = [ (col_c1 = 1, col_c2 = 2) || (col_c1 = 3, > col_c2 = 4)] > {code} > Plain update > {code:java} > UPDATE t SET col = 100 WHERE pk = 42 > Partition pruning metadata: t = [ (pk = 42) ] // Uses metadata provided by a > scan operation > {code} > Delete > {code:java} > DELETE FROM t WHERE pk = 42 > Partition pruning metadata: t = [ (pk = 42) ] // Uses metadata provided by a > scan operation > {code} > After this issue is resolved, partition pruning should work for INSERT, > UPDATE, MERGE, and DELETE statements. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-21609) Avoid using versioned values for non-versioned objects
[ https://issues.apache.org/jira/browse/IGNITE-21609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Mashenkov updated IGNITE-21609: -- Description: As of now, TableManager/IndexManager misuses VersionedValue while holding non-versioned objects, e.g. table instances, partitions, assignments and etc. Actually, we need 1. the latest value, but not a history, which is size of 10 entries by default. 2. a future for synchronization purposes with ongoing metastorage operation. Also, VV requires a metastorage token for update. Local operations, like table/index destruction, do NOT have any token and are executed out of metastorage context, because such operations should block metastorage and further metadata elovution. was: As of now, we misuse VersionedValue in TableManager/IndexManager for holding non-versioned objects, e.g. table instances, partitions, assignments and etc. Actually, we need 1. the latest value, but not a history, which is size of 10 entries by default. 2. a future for synchronization purposes with ongoing metastorage operation. Also, VV requires a metastorage token for update. Local operations, like table/index destruction, do NOT have any token and are executed out of metastorage context, because such operations should block metastorage and further metadata elovution. > Avoid using versioned values for non-versioned objects > -- > > Key: IGNITE-21609 > URL: https://issues.apache.org/jira/browse/IGNITE-21609 > Project: Ignite > Issue Type: Improvement >Reporter: Andrey Mashenkov >Priority: Major > Labels: ignite-3 > Fix For: 3.0 > > > As of now, TableManager/IndexManager misuses VersionedValue while holding > non-versioned objects, e.g. table instances, partitions, assignments and etc. > Actually, we need > 1. the latest value, but not a history, which is size of 10 entries by > default. > 2. a future for synchronization purposes with ongoing metastorage operation. > Also, VV requires a metastorage token for update. > Local operations, like table/index destruction, do NOT have any token and are > executed out of metastorage context, because such operations should block > metastorage and further metadata elovution. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-21609) Avoid using versioned values for non-versioned objects
[ https://issues.apache.org/jira/browse/IGNITE-21609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Mashenkov updated IGNITE-21609: -- Summary: Avoid using versioned values for non-versioned objects (was: Avoid using versioned values for non-versioned objects.) > Avoid using versioned values for non-versioned objects > -- > > Key: IGNITE-21609 > URL: https://issues.apache.org/jira/browse/IGNITE-21609 > Project: Ignite > Issue Type: Improvement >Reporter: Andrey Mashenkov >Priority: Major > Labels: ignite-3 > Fix For: 3.0 > > > As of now, we misuse VersionedValue in TableManager/IndexManager for holding > non-versioned objects, e.g. table instances, partitions, assignments and etc. > Actually, we need > 1. the latest value, but not a history, which is size of 10 entries by > default. > 2. a future for synchronization purposes with ongoing metastorage operation. > Also, VV requires a metastorage token for update. > Local operations, like table/index destruction, do NOT have any token and are > executed out of metastorage context, because such operations should block > metastorage and further metadata elovution. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-21346) Removal of MVCC code from GridCacheEntryEx and IgniteCacheOffheapManager
[ https://issues.apache.org/jira/browse/IGNITE-21346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anton Vinogradov updated IGNITE-21346: -- Fix Version/s: 2.17 > Removal of MVCC code from GridCacheEntryEx and IgniteCacheOffheapManager > > > Key: IGNITE-21346 > URL: https://issues.apache.org/jira/browse/IGNITE-21346 > Project: Ignite > Issue Type: Sub-task > Components: mvcc >Reporter: Ilya Shishkov >Assignee: Ilya Shishkov >Priority: Minor > Labels: ise > Fix For: 2.17 > > Time Spent: 10m > Remaining Estimate: 0h > > Remove all MVCC code from GridCacheEntryEx and IgniteCacheOffheapManager and > their succesors. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-21609) Avoid using versioned values for non-versioned objects.
Andrey Mashenkov created IGNITE-21609: - Summary: Avoid using versioned values for non-versioned objects. Key: IGNITE-21609 URL: https://issues.apache.org/jira/browse/IGNITE-21609 Project: Ignite Issue Type: Improvement Reporter: Andrey Mashenkov Fix For: 3.0 As of now, we misuse VersionedValue in TableManager/IndexManager for holding non-versioned objects, e.g. table instances, partitions, assignments and etc. Actually, we need 1. the latest value, but not a history, which is size of 10 entries by default. 2. a future for synchronization purposes with ongoing metastorage operation. Also, VV requires a metastorage token for update. Local operations, like table/index destruction, do NOT have any token and are executed out of metastorage context, because such operations should block metastorage and further metadata elovution. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-21608) Fix catalog compaction.
Andrey Mashenkov created IGNITE-21608: - Summary: Fix catalog compaction. Key: IGNITE-21608 URL: https://issues.apache.org/jira/browse/IGNITE-21608 Project: Ignite Issue Type: Improvement Reporter: Andrey Mashenkov Fix For: 3.0 Catalog compaction was disabled in IGNITE-21585, because we need consensus on all the nodes about safe timestamp/version the Catalog` history could be truncated to. Let's enable compations and fix all the issues. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-19410) Node failure in case multiple nodes join and leave a cluster simultaneously with security is enabled.
[ https://issues.apache.org/jira/browse/IGNITE-19410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17820722#comment-17820722 ] Nikolay commented on IGNITE-19410: -- [~PetrovMikhail] still observe this issue with ignite 2.16. Steps to reproduce are the same as you described. {code:java} [TcpDiscoveryNode [id=22670289-ad93-49f7-3b2e-e66e1abfb3af, consistentId=client, isClient=true, ver=2.16.0#20231215-sha1:7bde6a42] JVM will be halted immediately due to the failure: [failureCtx=FailureContext [type=SYSTEM_WORKER_TERMINATION, err=java.lang.IllegalStateException: Failed to find security context for subject with given ID : 22670289-ad93-49f7-3b2e-e66e1abfb3af]]{code} > Node failure in case multiple nodes join and leave a cluster simultaneously > with security is enabled. > -- > > Key: IGNITE-19410 > URL: https://issues.apache.org/jira/browse/IGNITE-19410 > Project: Ignite > Issue Type: Bug > Components: security >Reporter: Mikhail Petrov >Assignee: Mikhail Petrov >Priority: Major > Labels: ise > Fix For: 2.16 > > Attachments: NodeSecurityContextTest.java > > Time Spent: 3h > Remaining Estimate: 0h > > The case when nodes with security enabled join and leave the cluster > simultaneously can cause the joining nodes to fail with the following > exception: > {code:java} > [2023-05-03T14:54:31,208][ERROR][disco-notifier-worker-#332%ignite.NodeSecurityContextTest2%][IgniteTestResources] > Critical system error detected. Will be handled accordingly to configured > handler [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, > super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet > [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], > failureCtx=FailureContext [type=SYSTEM_WORKER_TERMINATION, > err=java.lang.IllegalStateException: Failed to find security context for > subject with given ID : 4725544a-f144-4486-a705-46b2ac200011]] > java.lang.IllegalStateException: Failed to find security context for subject > with given ID : 4725544a-f144-4486-a705-46b2ac200011 > at > org.apache.ignite.internal.processors.security.IgniteSecurityProcessor.withContext(IgniteSecurityProcessor.java:164) > ~[classes/:?] > at > org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$3$SecurityAwareNotificationTask.run(GridDiscoveryManager.java:949) > ~[classes/:?] > at > org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$DiscoveryMessageNotifierWorker.body0(GridDiscoveryManager.java:2822) > ~[classes/:?] > at > org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$DiscoveryMessageNotifierWorker.body(GridDiscoveryManager.java:2860) > [classes/:?] > at > org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:125) > [classes/:?] > at java.lang.Thread.run(Thread.java:750) [?:1.8.0_351] {code} > Reproducer is attached. > Simplified steps that leads to the failure: > 1. The client node sends an arbitrary discovery message which produces an > acknowledgement message when it processed by the all cluster nodes . > 2. The client node gracefully leaves the cluster. > 3. The new node joins the cluster and receives a topology snapshot that does > not include the left client node. > 4. The new node receives an acknowledgment for the message from the step 1 > and fails during its processing because message originator node is not listed > in the current discovery cache or discovery cache history (see > IgniteSecurityProcessor#withContext(java.util.UUID)) . This is because > currently the GridDiscoveryManager#historicalNode method only aware of the > topology history that occurs after a node has joined the cluster. The > complete cluster topology history that exists at the time a new node joined > the cluster is stored in GridDiscoveryManager#topHist and is not taken into > account by the GridDiscoveryManager#historicalNode method. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IGNITE-21607) Code cleanup
[ https://issues.apache.org/jira/browse/IGNITE-21607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Steshin reassigned IGNITE-21607: - Assignee: Vladimir Steshin > Code cleanup > > > Key: IGNITE-21607 > URL: https://issues.apache.org/jira/browse/IGNITE-21607 > Project: Ignite > Issue Type: Improvement >Reporter: Vladimir Steshin >Assignee: Vladimir Steshin >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > Fixing of typos, abbreviations, unused code and untyped generics. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-21591) Add tuple updrades during index backfill process
[ https://issues.apache.org/jira/browse/IGNITE-21591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirill Tkalenko updated IGNITE-21591: - Summary: Add tuple updrades during index backfill process (was: Add tuple updates during index backfill process) > Add tuple updrades during index backfill process > > > Key: IGNITE-21591 > URL: https://issues.apache.org/jira/browse/IGNITE-21591 > Project: Ignite > Issue Type: Improvement >Reporter: Kirill Tkalenko >Assignee: Kirill Tkalenko >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > Time Spent: 10m > Remaining Estimate: 0h > > Implement point 10 of ticket IGNITE-20117. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-21607) Code cleanup
Vladimir Steshin created IGNITE-21607: - Summary: Code cleanup Key: IGNITE-21607 URL: https://issues.apache.org/jira/browse/IGNITE-21607 Project: Ignite Issue Type: Improvement Reporter: Vladimir Steshin Fixing of typos, abbreviations, unused code and untyped generics. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-21606) Missing tuple update during filtering index scans
[ https://issues.apache.org/jira/browse/IGNITE-21606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirill Tkalenko updated IGNITE-21606: - Fix Version/s: 3.0.0-beta2 > Missing tuple update during filtering index scans > - > > Key: IGNITE-21606 > URL: https://issues.apache.org/jira/browse/IGNITE-21606 > Project: Ignite > Issue Type: Bug >Reporter: Kirill Tkalenko >Assignee: Kirill Tkalenko >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > Consider the following test scenario: > # Create a table with two columns. > # We insert into this table. > # Add a column to the table. > # Create an index on the new column. > # We make a “select” with a condition on the new column. > We get an error: > {noformat} > [2024-02-26T11:30:37,822][WARN > ][%ibiont_n_0%scan-query-executor-0][ReplicaManager] Failed to process > replica request [request=ReadOnlyScanRetrieveBatchReplicaRequestImpl > [batchSize=100, columnsToInclude={0, 1, 2, 3}, exactKey=null, flags=3, > groupId=7_part_0, indexToUse=10, lowerBoundPrefix=BinaryTupleMessageImpl > [elementCount=1, tuple=java.nio.HeapByteBuffer[pos=0 lim=9 cap=9]], > readTimestampLong=111996845269843968, scanId=1, > transactionId=018de489-92b1--41c5-8f650001, > upperBoundPrefix=BinaryTupleMessageImpl [elementCount=1, > tuple=java.nio.HeapByteBuffer[pos=0 lim=9 cap=9. > java.util.concurrent.CompletionException: java.lang.AssertionError: > schemaVersion=1, column=SURNAME > at > java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:314) > [?:?] > at > java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:319) > [?:?] > at > java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:1081) > [?:?] > at > java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:478) > [?:?] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) > [?:?] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) > [?:?] > at java.lang.Thread.run(Thread.java:834) [?:?] > Caused by: java.lang.AssertionError: schemaVersion=1, column=SURNAME > at > org.apache.ignite.internal.index.IndexManager$TableRowToIndexKeyConverter.resolveColumnIndexes(IndexManager.java:289) > ~[main/:?] > at > org.apache.ignite.internal.index.IndexManager$TableRowToIndexKeyConverter.createConverter(IndexManager.java:276) > ~[main/:?] > at > org.apache.ignite.internal.index.IndexManager$TableRowToIndexKeyConverter.converter(IndexManager.java:262) > ~[main/:?] > at > org.apache.ignite.internal.index.IndexManager$TableRowToIndexKeyConverter.extractColumns(IndexManager.java:249) > ~[main/:?] > at > org.apache.ignite.internal.table.distributed.replicator.PartitionReplicaListener.indexRowMatches(PartitionReplicaListener.java:1369) > ~[main/:?] > at > org.apache.ignite.internal.table.distributed.replicator.PartitionReplicaListener.lambda$continueReadOnlyIndexScan$51(PartitionReplicaListener.java:1295) > ~[main/:?] > at > java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:1072) > ~[?:?] > ... 4 more > {noformat} > This happens because when filtering index scans (IGNITE-18518), we do not > update the tuple to the required schema version. > Look TODOs in the code. > We also need to understand which schema version to use to update the tuple. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IGNITE-21606) Missing tuple update during filtering index scans
[ https://issues.apache.org/jira/browse/IGNITE-21606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirill Tkalenko reassigned IGNITE-21606: Assignee: Kirill Tkalenko > Missing tuple update during filtering index scans > - > > Key: IGNITE-21606 > URL: https://issues.apache.org/jira/browse/IGNITE-21606 > Project: Ignite > Issue Type: Bug >Reporter: Kirill Tkalenko >Assignee: Kirill Tkalenko >Priority: Major > Labels: ignite-3 > > Consider the following test scenario: > # Create a table with two columns. > # We insert into this table. > # Add a column to the table. > # Create an index on the new column. > # We make a “select” with a condition on the new column. > We get an error: > {noformat} > [2024-02-26T11:30:37,822][WARN > ][%ibiont_n_0%scan-query-executor-0][ReplicaManager] Failed to process > replica request [request=ReadOnlyScanRetrieveBatchReplicaRequestImpl > [batchSize=100, columnsToInclude={0, 1, 2, 3}, exactKey=null, flags=3, > groupId=7_part_0, indexToUse=10, lowerBoundPrefix=BinaryTupleMessageImpl > [elementCount=1, tuple=java.nio.HeapByteBuffer[pos=0 lim=9 cap=9]], > readTimestampLong=111996845269843968, scanId=1, > transactionId=018de489-92b1--41c5-8f650001, > upperBoundPrefix=BinaryTupleMessageImpl [elementCount=1, > tuple=java.nio.HeapByteBuffer[pos=0 lim=9 cap=9. > java.util.concurrent.CompletionException: java.lang.AssertionError: > schemaVersion=1, column=SURNAME > at > java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:314) > [?:?] > at > java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:319) > [?:?] > at > java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:1081) > [?:?] > at > java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:478) > [?:?] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) > [?:?] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) > [?:?] > at java.lang.Thread.run(Thread.java:834) [?:?] > Caused by: java.lang.AssertionError: schemaVersion=1, column=SURNAME > at > org.apache.ignite.internal.index.IndexManager$TableRowToIndexKeyConverter.resolveColumnIndexes(IndexManager.java:289) > ~[main/:?] > at > org.apache.ignite.internal.index.IndexManager$TableRowToIndexKeyConverter.createConverter(IndexManager.java:276) > ~[main/:?] > at > org.apache.ignite.internal.index.IndexManager$TableRowToIndexKeyConverter.converter(IndexManager.java:262) > ~[main/:?] > at > org.apache.ignite.internal.index.IndexManager$TableRowToIndexKeyConverter.extractColumns(IndexManager.java:249) > ~[main/:?] > at > org.apache.ignite.internal.table.distributed.replicator.PartitionReplicaListener.indexRowMatches(PartitionReplicaListener.java:1369) > ~[main/:?] > at > org.apache.ignite.internal.table.distributed.replicator.PartitionReplicaListener.lambda$continueReadOnlyIndexScan$51(PartitionReplicaListener.java:1295) > ~[main/:?] > at > java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:1072) > ~[?:?] > ... 4 more > {noformat} > This happens because when filtering index scans (IGNITE-18518), we do not > update the tuple to the required schema version. > Look TODOs in the code. > We also need to understand which schema version to use to update the tuple. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-21602) Fix direct buffers allocation on Java 21
[ https://issues.apache.org/jira/browse/IGNITE-21602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Puchkovskiy updated IGNITE-21602: --- Description: In Java 21, the signature of the constructor of DirectByteBuffer we use was changed: now length is a long, not an int. We need to support this new signature, still using the old one with older versions of Java. > Fix direct buffers allocation on Java 21 > > > Key: IGNITE-21602 > URL: https://issues.apache.org/jira/browse/IGNITE-21602 > Project: Ignite > Issue Type: Improvement >Reporter: Roman Puchkovskiy >Assignee: Roman Puchkovskiy >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > Time Spent: 10m > Remaining Estimate: 0h > > In Java 21, the signature of the constructor of DirectByteBuffer we use was > changed: now length is a long, not an int. We need to support this new > signature, still using the old one with older versions of Java. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (IGNITE-19274) Sql. Jdbc client. Support TIMESTAMP WITH LOCAL TIME ZONE type
[ https://issues.apache.org/jira/browse/IGNITE-19274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17716595#comment-17716595 ] Pavel Pereslegin edited comment on IGNITE-19274 at 2/26/24 12:14 PM: - outdated comment (see update section in issue) was (Author: xtern): The issue described in the task is one of the problems in terms of supporting work with the client's timezone. *The problem - Ignite-3 do not take into account the client's/initiator's timezone in any way.* We have the following "data types mapping": ||Calcite type||JDBC Type||Current Ignite type|| |TIME|java.sql.Time|java.time.LocalTime| |TIME_WITH_LOCAL_TIME_ZONE|java.sql.Time|java.time.LocalTime| |TIMESTAMP|java.sql.Timestamp|java.time.LocalDateTime| |TIMESTAMP_WITH_LOCAL_TIME_ZONE|java.sql.Timestamp|java.time.Instant| *_WITH_LOCAL_TIME_ZONE types must take into account the client's local time zone. Those. if the client is in the "GMT+3" time zone and is passing the value "14:00", the value must be converted from GMT+3 to UTC and the value "11:00" must be stored in the database. It seems that *from KV side* there is no problem with {{TIMESTAMP_WITH_LOCAL_TIME_ZONE}}, because client must set {{Instant}} value for the field (and {{instant}} value is always in UTC) *From sql side* client must pass somehow (per-connection or per-query (check other DB)) client's timezone, that must be taken into account by sql-engine on server side. {{TIME_WITH_LOCAL_TIME_ZONE}} must be mapped to another type (not the same as {{TIME}})). We also need to design how default values for these types will be materialized. > Sql. Jdbc client. Support TIMESTAMP WITH LOCAL TIME ZONE type > -- > > Key: IGNITE-19274 > URL: https://issues.apache.org/jira/browse/IGNITE-19274 > Project: Ignite > Issue Type: Bug > Components: sql >Affects Versions: 3.0.0-beta1 >Reporter: Evgeny Stanilovsky >Priority: Major > Labels: ignite-3 > > The {{TIMESTAMP WITH LOCAL TIME ZONE}} data type is a variant of > {{TIMESTAMP}} that includes a time zone offset in its value. Data stored in > the database is normalized to the database time zone (UTC) and time zone > offset is not stored as part of the column data. When the data is retrieved, > it to be returned in the user's local session time zone. > i.e: > {noformat} > CREATE TABLE timestamp(ts TIMESTAMP, t_tz TIMESTAMP WITH LOCAL TIME ZONE); > SET TIME ZONE 'tz1'; > INSERT INTO timestamp VALUES ('2011-01-01 01:01:01', TIMESTAMP WITH LOCAL > TIME ZONE '2011-01-01 01:01:01'); > SET TIME ZONE 'tz2'; > INSERT INTO timestamp VALUES ('2011-01-01 01:01:01', TIMESTAMP WITH LOCAL > TIME ZONE '2011-01-01 01:01:01'); > ... > select * from timestamp;{noformat} > returned rows need to be different in case of different tz1 and tz2 offsets > but they are equals for now. Also returned representation need to be present > in user session time zone. > h5. Update from 26.02.2024: > Definition of done for this task: > * Client time zone is passed to server (check other database implementations > to decide how and when to pass it). > * Data of type "TIMESTAMP With LOCAL TIME ZONE" can be written/read correctly > using the dynamic parameter. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19274) Sql. Jdbc client. Support TIMESTAMP WITH LOCAL TIME ZONE type
[ https://issues.apache.org/jira/browse/IGNITE-19274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Pereslegin updated IGNITE-19274: -- Description: The {{TIMESTAMP WITH LOCAL TIME ZONE}} data type is a variant of {{TIMESTAMP}} that includes a time zone offset in its value. Data stored in the database is normalized to the database time zone (UTC) and time zone offset is not stored as part of the column data. When the data is retrieved, it to be returned in the user's local session time zone. i.e: {noformat} CREATE TABLE timestamp(ts TIMESTAMP, t_tz TIMESTAMP WITH LOCAL TIME ZONE); SET TIME ZONE 'tz1'; INSERT INTO timestamp VALUES ('2011-01-01 01:01:01', TIMESTAMP WITH LOCAL TIME ZONE '2011-01-01 01:01:01'); SET TIME ZONE 'tz2'; INSERT INTO timestamp VALUES ('2011-01-01 01:01:01', TIMESTAMP WITH LOCAL TIME ZONE '2011-01-01 01:01:01'); ... select * from timestamp;{noformat} returned rows need to be different in case of different tz1 and tz2 offsets but they are equals for now. Also returned representation need to be present in user session time zone. h5. Update from 26.02.2024: Definition of done for this task: * Client time zone is passed to server (check other database implementations to decide how and when to pass it). * Data of type "TIMESTAMP With LOCAL TIME ZONE" can be written/read correctly using the dynamic parameter. was: The {{TIMESTAMP WITH LOCAL TIME ZONE}} data type is a variant of {{TIMESTAMP}} that includes a time zone offset in its value. Data stored in the database is normalized to the database time zone (UTC) and time zone offset is not stored as part of the column data. When the data is retrieved, it to be returned in the user's local session time zone. i.e: {noformat} CREATE TABLE timestamp(ts TIMESTAMP, t_tz TIMESTAMP WITH LOCAL TIME ZONE); SET TIME ZONE 'tz1'; INSERT INTO timestamp VALUES ('2011-01-01 01:01:01', TIMESTAMP WITH LOCAL TIME ZONE '2011-01-01 01:01:01'); SET TIME ZONE 'tz2'; INSERT INTO timestamp VALUES ('2011-01-01 01:01:01', TIMESTAMP WITH LOCAL TIME ZONE '2011-01-01 01:01:01'); ... select * from timestamp;{noformat} returned rows need to be different in case of different tz1 and tz2 offsets but they are equals for now. Also returned representation need to be present in user session time zone. > Sql. Jdbc client. Support TIMESTAMP WITH LOCAL TIME ZONE type > -- > > Key: IGNITE-19274 > URL: https://issues.apache.org/jira/browse/IGNITE-19274 > Project: Ignite > Issue Type: Bug > Components: sql >Affects Versions: 3.0.0-beta1 >Reporter: Evgeny Stanilovsky >Priority: Major > Labels: ignite-3 > > The {{TIMESTAMP WITH LOCAL TIME ZONE}} data type is a variant of > {{TIMESTAMP}} that includes a time zone offset in its value. Data stored in > the database is normalized to the database time zone (UTC) and time zone > offset is not stored as part of the column data. When the data is retrieved, > it to be returned in the user's local session time zone. > i.e: > {noformat} > CREATE TABLE timestamp(ts TIMESTAMP, t_tz TIMESTAMP WITH LOCAL TIME ZONE); > SET TIME ZONE 'tz1'; > INSERT INTO timestamp VALUES ('2011-01-01 01:01:01', TIMESTAMP WITH LOCAL > TIME ZONE '2011-01-01 01:01:01'); > SET TIME ZONE 'tz2'; > INSERT INTO timestamp VALUES ('2011-01-01 01:01:01', TIMESTAMP WITH LOCAL > TIME ZONE '2011-01-01 01:01:01'); > ... > select * from timestamp;{noformat} > returned rows need to be different in case of different tz1 and tz2 offsets > but they are equals for now. Also returned representation need to be present > in user session time zone. > h5. Update from 26.02.2024: > Definition of done for this task: > * Client time zone is passed to server (check other database implementations > to decide how and when to pass it). > * Data of type "TIMESTAMP With LOCAL TIME ZONE" can be written/read correctly > using the dynamic parameter. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-21545) Introduce a cursor manager
[ https://issues.apache.org/jira/browse/IGNITE-21545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17820694#comment-17820694 ] Vladislav Pyatkov commented on IGNITE-21545: Merged 4915e292808560e1c3eb995d6a03466bbdf3 > Introduce a cursor manager > -- > > Key: IGNITE-21545 > URL: https://issues.apache.org/jira/browse/IGNITE-21545 > Project: Ignite > Issue Type: Improvement >Reporter: Denis Chudov >Assignee: Kirill Sizov >Priority: Major > Labels: ignite-3 > Time Spent: 50m > Remaining Estimate: 0h > > Introduce a cursor manager that would maintain all cursors created on a node, > instead of maintaining them in partition replica listeners. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IGNITE-19274) Sql. Jdbc client. Support TIMESTAMP WITH LOCAL TIME ZONE type
[ https://issues.apache.org/jira/browse/IGNITE-19274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Pereslegin reassigned IGNITE-19274: - Assignee: (was: Pavel Pereslegin) > Sql. Jdbc client. Support TIMESTAMP WITH LOCAL TIME ZONE type > -- > > Key: IGNITE-19274 > URL: https://issues.apache.org/jira/browse/IGNITE-19274 > Project: Ignite > Issue Type: Bug > Components: sql >Affects Versions: 3.0.0-beta1 >Reporter: Evgeny Stanilovsky >Priority: Major > Labels: ignite-3 > > The {{TIMESTAMP WITH LOCAL TIME ZONE}} data type is a variant of > {{TIMESTAMP}} that includes a time zone offset in its value. Data stored in > the database is normalized to the database time zone (UTC) and time zone > offset is not stored as part of the column data. When the data is retrieved, > it to be returned in the user's local session time zone. > i.e: > {noformat} > CREATE TABLE timestamp(ts TIMESTAMP, t_tz TIMESTAMP WITH LOCAL TIME ZONE); > SET TIME ZONE 'tz1'; > INSERT INTO timestamp VALUES ('2011-01-01 01:01:01', TIMESTAMP WITH LOCAL > TIME ZONE '2011-01-01 01:01:01'); > SET TIME ZONE 'tz2'; > INSERT INTO timestamp VALUES ('2011-01-01 01:01:01', TIMESTAMP WITH LOCAL > TIME ZONE '2011-01-01 01:01:01'); > ... > select * from timestamp;{noformat} > returned rows need to be different in case of different tz1 and tz2 offsets > but they are equals for now. Also returned representation need to be present > in user session time zone. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19274) Sql. Jdbc client. Support TIMESTAMP WITH LOCAL TIME ZONE type
[ https://issues.apache.org/jira/browse/IGNITE-19274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Pereslegin updated IGNITE-19274: -- Summary: Sql. Jdbc client. Support TIMESTAMP WITH LOCAL TIME ZONE type (was: Sql. Jdbc side working with TIMESTAMP WITH LOCAL TIME ZONE did not take into account current tz while storing data.) > Sql. Jdbc client. Support TIMESTAMP WITH LOCAL TIME ZONE type > -- > > Key: IGNITE-19274 > URL: https://issues.apache.org/jira/browse/IGNITE-19274 > Project: Ignite > Issue Type: Bug > Components: sql >Affects Versions: 3.0.0-beta1 >Reporter: Evgeny Stanilovsky >Assignee: Pavel Pereslegin >Priority: Major > Labels: ignite-3 > > The {{TIMESTAMP WITH LOCAL TIME ZONE}} data type is a variant of > {{TIMESTAMP}} that includes a time zone offset in its value. Data stored in > the database is normalized to the database time zone (UTC) and time zone > offset is not stored as part of the column data. When the data is retrieved, > it to be returned in the user's local session time zone. > i.e: > {noformat} > CREATE TABLE timestamp(ts TIMESTAMP, t_tz TIMESTAMP WITH LOCAL TIME ZONE); > SET TIME ZONE 'tz1'; > INSERT INTO timestamp VALUES ('2011-01-01 01:01:01', TIMESTAMP WITH LOCAL > TIME ZONE '2011-01-01 01:01:01'); > SET TIME ZONE 'tz2'; > INSERT INTO timestamp VALUES ('2011-01-01 01:01:01', TIMESTAMP WITH LOCAL > TIME ZONE '2011-01-01 01:01:01'); > ... > select * from timestamp;{noformat} > returned rows need to be different in case of different tz1 and tz2 offsets > but they are equals for now. Also returned representation need to be present > in user session time zone. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-21606) Missing tuple update during filtering index scans
[ https://issues.apache.org/jira/browse/IGNITE-21606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirill Tkalenko updated IGNITE-21606: - Description: Consider the following test scenario: # Create a table with two columns. # We insert into this table. # Add a column to the table. # Create an index on the new column. # We make a “select” with a condition on the new column. We get an error: {noformat} [2024-02-26T11:30:37,822][WARN ][%ibiont_n_0%scan-query-executor-0][ReplicaManager] Failed to process replica request [request=ReadOnlyScanRetrieveBatchReplicaRequestImpl [batchSize=100, columnsToInclude={0, 1, 2, 3}, exactKey=null, flags=3, groupId=7_part_0, indexToUse=10, lowerBoundPrefix=BinaryTupleMessageImpl [elementCount=1, tuple=java.nio.HeapByteBuffer[pos=0 lim=9 cap=9]], readTimestampLong=111996845269843968, scanId=1, transactionId=018de489-92b1--41c5-8f650001, upperBoundPrefix=BinaryTupleMessageImpl [elementCount=1, tuple=java.nio.HeapByteBuffer[pos=0 lim=9 cap=9. java.util.concurrent.CompletionException: java.lang.AssertionError: schemaVersion=1, column=SURNAME at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:314) [?:?] at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:319) [?:?] at java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:1081) [?:?] at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:478) [?:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] at java.lang.Thread.run(Thread.java:834) [?:?] Caused by: java.lang.AssertionError: schemaVersion=1, column=SURNAME at org.apache.ignite.internal.index.IndexManager$TableRowToIndexKeyConverter.resolveColumnIndexes(IndexManager.java:289) ~[main/:?] at org.apache.ignite.internal.index.IndexManager$TableRowToIndexKeyConverter.createConverter(IndexManager.java:276) ~[main/:?] at org.apache.ignite.internal.index.IndexManager$TableRowToIndexKeyConverter.converter(IndexManager.java:262) ~[main/:?] at org.apache.ignite.internal.index.IndexManager$TableRowToIndexKeyConverter.extractColumns(IndexManager.java:249) ~[main/:?] at org.apache.ignite.internal.table.distributed.replicator.PartitionReplicaListener.indexRowMatches(PartitionReplicaListener.java:1369) ~[main/:?] at org.apache.ignite.internal.table.distributed.replicator.PartitionReplicaListener.lambda$continueReadOnlyIndexScan$51(PartitionReplicaListener.java:1295) ~[main/:?] at java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:1072) ~[?:?] ... 4 more {noformat} This happens because when filtering index scans (IGNITE-18518), we do not update the tuple to the required schema version. Look TODOs in the code. We also need to understand which schema version to use to update the tuple. was: Consider the following test scenario: # Create a table with two columns. # We insert into this table. # Add a column to the table. # Create an index on the new column. # We make a “select” with a condition on the new column. We get an error: {noformat} [2024-02-26T11:30:37,822][WARN ][%ibiont_n_0%scan-query-executor-0][ReplicaManager] Failed to process replica request [request=ReadOnlyScanRetrieveBatchReplicaRequestImpl [batchSize=100, columnsToInclude={0, 1, 2, 3}, exactKey=null, flags=3, groupId=7_part_0, indexToUse=10, lowerBoundPrefix=BinaryTupleMessageImpl [elementCount=1, tuple=java.nio.HeapByteBuffer[pos=0 lim=9 cap=9]], readTimestampLong=111996845269843968, scanId=1, transactionId=018de489-92b1--41c5-8f650001, upperBoundPrefix=BinaryTupleMessageImpl [elementCount=1, tuple=java.nio.HeapByteBuffer[pos=0 lim=9 cap=9. java.util.concurrent.CompletionException: java.lang.AssertionError: schemaVersion=1, column=SURNAME at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:314) [?:?] at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:319) [?:?] at java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:1081) [?:?] at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:478) [?:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] at java.lang.Thread.run(Thread.java:834) [?:?] Caused by: java.lang.AssertionError: schemaVersion=1, column=SURNAME at org.apache.ignite.internal.index.IndexManager$TableRowToIndexKeyConverter.resolveColumnIndexes(IndexManager.java:289)
[jira] [Updated] (IGNITE-21606) Missing tuple update during filtering index scans
[ https://issues.apache.org/jira/browse/IGNITE-21606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirill Tkalenko updated IGNITE-21606: - Summary: Missing tuple update during filtering index scans (was: Missing tuple update during index scans) > Missing tuple update during filtering index scans > - > > Key: IGNITE-21606 > URL: https://issues.apache.org/jira/browse/IGNITE-21606 > Project: Ignite > Issue Type: Bug >Reporter: Kirill Tkalenko >Priority: Major > Labels: ignite-3 > > Consider the following test scenario: > # Create a table with two columns. > # We insert into this table. > # Add a column to the table. > # Create an index on the new column. > # We make a “select” with a condition on the new column. > We get an error: > {noformat} > [2024-02-26T11:30:37,822][WARN > ][%ibiont_n_0%scan-query-executor-0][ReplicaManager] Failed to process > replica request [request=ReadOnlyScanRetrieveBatchReplicaRequestImpl > [batchSize=100, columnsToInclude={0, 1, 2, 3}, exactKey=null, flags=3, > groupId=7_part_0, indexToUse=10, lowerBoundPrefix=BinaryTupleMessageImpl > [elementCount=1, tuple=java.nio.HeapByteBuffer[pos=0 lim=9 cap=9]], > readTimestampLong=111996845269843968, scanId=1, > transactionId=018de489-92b1--41c5-8f650001, > upperBoundPrefix=BinaryTupleMessageImpl [elementCount=1, > tuple=java.nio.HeapByteBuffer[pos=0 lim=9 cap=9. > java.util.concurrent.CompletionException: java.lang.AssertionError: > schemaVersion=1, column=SURNAME > at > java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:314) > [?:?] > at > java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:319) > [?:?] > at > java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:1081) > [?:?] > at > java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:478) > [?:?] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) > [?:?] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) > [?:?] > at java.lang.Thread.run(Thread.java:834) [?:?] > Caused by: java.lang.AssertionError: schemaVersion=1, column=SURNAME > at > org.apache.ignite.internal.index.IndexManager$TableRowToIndexKeyConverter.resolveColumnIndexes(IndexManager.java:289) > ~[main/:?] > at > org.apache.ignite.internal.index.IndexManager$TableRowToIndexKeyConverter.createConverter(IndexManager.java:276) > ~[main/:?] > at > org.apache.ignite.internal.index.IndexManager$TableRowToIndexKeyConverter.converter(IndexManager.java:262) > ~[main/:?] > at > org.apache.ignite.internal.index.IndexManager$TableRowToIndexKeyConverter.extractColumns(IndexManager.java:249) > ~[main/:?] > at > org.apache.ignite.internal.table.distributed.replicator.PartitionReplicaListener.indexRowMatches(PartitionReplicaListener.java:1369) > ~[main/:?] > at > org.apache.ignite.internal.table.distributed.replicator.PartitionReplicaListener.lambda$continueReadOnlyIndexScan$51(PartitionReplicaListener.java:1295) > ~[main/:?] > at > java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:1072) > ~[?:?] > ... 4 more > {noformat} > This happens because when filtering index scans (IGNITE-18518), we do not > update the tuple to the required schema version. > Look TODOs in the code. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-21606) Missing tuple update during index scans
Kirill Tkalenko created IGNITE-21606: Summary: Missing tuple update during index scans Key: IGNITE-21606 URL: https://issues.apache.org/jira/browse/IGNITE-21606 Project: Ignite Issue Type: Bug Reporter: Kirill Tkalenko Consider the following test scenario: # Create a table with two columns. # We insert into this table. # Add a column to the table. # Create an index on the new column. # We make a “select” with a condition on the new column. We get an error: {noformat} [2024-02-26T11:30:37,822][WARN ][%ibiont_n_0%scan-query-executor-0][ReplicaManager] Failed to process replica request [request=ReadOnlyScanRetrieveBatchReplicaRequestImpl [batchSize=100, columnsToInclude={0, 1, 2, 3}, exactKey=null, flags=3, groupId=7_part_0, indexToUse=10, lowerBoundPrefix=BinaryTupleMessageImpl [elementCount=1, tuple=java.nio.HeapByteBuffer[pos=0 lim=9 cap=9]], readTimestampLong=111996845269843968, scanId=1, transactionId=018de489-92b1--41c5-8f650001, upperBoundPrefix=BinaryTupleMessageImpl [elementCount=1, tuple=java.nio.HeapByteBuffer[pos=0 lim=9 cap=9. java.util.concurrent.CompletionException: java.lang.AssertionError: schemaVersion=1, column=SURNAME at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:314) [?:?] at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:319) [?:?] at java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:1081) [?:?] at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:478) [?:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] at java.lang.Thread.run(Thread.java:834) [?:?] Caused by: java.lang.AssertionError: schemaVersion=1, column=SURNAME at org.apache.ignite.internal.index.IndexManager$TableRowToIndexKeyConverter.resolveColumnIndexes(IndexManager.java:289) ~[main/:?] at org.apache.ignite.internal.index.IndexManager$TableRowToIndexKeyConverter.createConverter(IndexManager.java:276) ~[main/:?] at org.apache.ignite.internal.index.IndexManager$TableRowToIndexKeyConverter.converter(IndexManager.java:262) ~[main/:?] at org.apache.ignite.internal.index.IndexManager$TableRowToIndexKeyConverter.extractColumns(IndexManager.java:249) ~[main/:?] at org.apache.ignite.internal.table.distributed.replicator.PartitionReplicaListener.indexRowMatches(PartitionReplicaListener.java:1369) ~[main/:?] at org.apache.ignite.internal.table.distributed.replicator.PartitionReplicaListener.lambda$continueReadOnlyIndexScan$51(PartitionReplicaListener.java:1295) ~[main/:?] at java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:1072) ~[?:?] ... 4 more {noformat} This happens because when filtering index scans (IGNITE-18518), we do not update the tuple to the required schema version. Look TODOs in the code. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IGNITE-21556) ODBC 3.0: Use after free in SQLExecDirect
[ https://issues.apache.org/jira/browse/IGNITE-21556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Sapego reassigned IGNITE-21556: Assignee: Igor Sapego > ODBC 3.0: Use after free in SQLExecDirect > - > > Key: IGNITE-21556 > URL: https://issues.apache.org/jira/browse/IGNITE-21556 > Project: Ignite > Issue Type: Bug > Components: platforms >Affects Versions: 3.0 >Reporter: Dmitrii Zabotlin >Assignee: Igor Sapego >Priority: Major > Labels: ignite-3 > > There is a use-after-free bug in the ODBC platforms code. > Steps to reproduce: > 1. Execute query with parameters previously bound. > 2. Execute another query on top of the same statement without parameters (for > example, SELECT). > If previous parameter arrays are already freed crash will occur here: > {code:java} > void parameter::reset_stored_data() { > m_stored_data.clear(); > if (m_buffer.is_data_at_exec()) > m_stored_data.reserve(m_buffer.get_data_at_exec_size()); > } {code} > Method is_data_at_exec reads buffer content: > {code:java} > bool application_data_buffer::is_data_at_exec() const { > const SQLLEN *res_len_ptr = get_result_len(); > if (!res_len_ptr) > return false; > auto s_len = static_cast(*res_len_ptr); > return s_len <= SQL_LEN_DATA_AT_EXEC_OFFSET || s_len == SQL_DATA_AT_EXEC; > }{code} > If parameter type is variable length type, for example, string, res_len_ptr > will not be null and we will read from freed memory. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-21604) .NET: Thin 3.0: Pass client time zone to server
Igor Sapego created IGNITE-21604: Summary: .NET: Thin 3.0: Pass client time zone to server Key: IGNITE-21604 URL: https://issues.apache.org/jira/browse/IGNITE-21604 Project: Ignite Issue Type: Improvement Components: thin client Reporter: Pavel Pereslegin Once IGNITE-21551 is implemented, it will be possible to set the time zone in a user session. The session time zone is taken into account when calling functions for obtaining the current time (CURRENT_TIMESTAMP, etc), as well as when processing a string literal for the 'TIMESTAMP WITH LOCAL TIME ZONE' type. For this to work correctly, you need to transfer the client's time zone through the thin client. p.s. check the TODOs that point to this ticket. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IGNITE-21605) C++ 3.0: Pass client time zone to server
[ https://issues.apache.org/jira/browse/IGNITE-21605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Sapego reassigned IGNITE-21605: Assignee: Igor Sapego > C++ 3.0: Pass client time zone to server > > > Key: IGNITE-21605 > URL: https://issues.apache.org/jira/browse/IGNITE-21605 > Project: Ignite > Issue Type: Improvement > Components: thin client >Reporter: Pavel Pereslegin >Assignee: Igor Sapego >Priority: Major > Labels: ignite-3 > > Once IGNITE-21551 is implemented, it will be possible to set the time zone in > a user session. > The session time zone is taken into account when calling functions for > obtaining the current time (CURRENT_TIMESTAMP, etc), as well as when > processing a string literal for the 'TIMESTAMP WITH LOCAL TIME ZONE' type. > For this to work correctly, you need to transfer the client's time zone > through the thin client. > p.s. check the TODOs that point to this ticket. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-21605) C++ 3.0: Pass client time zone to server
Igor Sapego created IGNITE-21605: Summary: C++ 3.0: Pass client time zone to server Key: IGNITE-21605 URL: https://issues.apache.org/jira/browse/IGNITE-21605 Project: Ignite Issue Type: Improvement Components: thin client Reporter: Pavel Pereslegin Once IGNITE-21551 is implemented, it will be possible to set the time zone in a user session. The session time zone is taken into account when calling functions for obtaining the current time (CURRENT_TIMESTAMP, etc), as well as when processing a string literal for the 'TIMESTAMP WITH LOCAL TIME ZONE' type. For this to work correctly, you need to transfer the client's time zone through the thin client. p.s. check the TODOs that point to this ticket. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-21602) Fix direct buffers allocation on Java 21
[ https://issues.apache.org/jira/browse/IGNITE-21602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Puchkovskiy updated IGNITE-21602: --- Summary: Fix direct buffers allocation on Java 21 (was: Support Java 21) > Fix direct buffers allocation on Java 21 > > > Key: IGNITE-21602 > URL: https://issues.apache.org/jira/browse/IGNITE-21602 > Project: Ignite > Issue Type: Improvement >Reporter: Roman Puchkovskiy >Assignee: Roman Puchkovskiy >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IGNITE-21603) Incorrect backward connection chech with loopback address.
[ https://issues.apache.org/jira/browse/IGNITE-21603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Steshin reassigned IGNITE-21603: - Assignee: Vladimir Steshin > Incorrect backward connection chech with loopback address. > -- > > Key: IGNITE-21603 > URL: https://issues.apache.org/jira/browse/IGNITE-21603 > Project: Ignite > Issue Type: Bug >Reporter: Vladimir Steshin >Assignee: Vladimir Steshin >Priority: Minor > Labels: ise > Time Spent: 10m > Remaining Estimate: 0h > > We may skip backward connection check to a previous node if it has the same > loopback address as the current node. > Consider: > # Neither _IgniteConfiguration#setLocalHost()_ or > _TcpDiscoverySpi#setLocalAddress()_ is set. Or the localhost parameter is set > to "_0.0.0.0_". > # Nodes start on different hosts. All the available host addresses are > resolved and > # Among the other addresses, all nodes get the loopback address > "127.0.0.1:47500" (47500 is the default tcp discovery port). > # Cluster starts and works. But > # Some node N (A) decides the connection to node N+1 (B) is lost and tries to > connect to node N+2 (C) and sends _TcpDiscoveryHandshakeRequest_. > # Before C accepts incoming A's connection, it decides to check B and pings > it with _ServerImpl#checkConnection(List addrs, int > timeout)_ > # Around here, the network is restored, and A can now connect to B anew. > # "_127.0.0.1:47500_" is last in _List_ addrs by > _IgniteUtils#inetAddressesComparator(boolean sameHost)_. But the connect > attempts in _checkConnection(...)_ are parallel. "_127.0.0.1:47500_" answers > first. > # C sees it can connect to "_127.0.0.1:47500_" and chooses it as the alive > address of B. Other pings to rest of B's addresses are ignored. > # But "_127.0.0.1:47500_" is one of C's addresses. C realizes it pinged > itself and decides that B is not reachable: > {code:java} > // If local node was able to connect to previous, confirm that it's > alive. > ok = liveAddr != null && (!liveAddr.getAddress().isLoopbackAddress() > || !locNode.socketAddresses().contains(liveAddr)); > {code} > # C accepts connection from A and answers with > _TcpDiscoveryHandshakeResponse#previousNodeAlive() == false_ > # But B is ok now. But A connects to C and B is kicked from the ring. > The problem is that C ping itself by B's address "_127.0.0.1:47500_" -- This message was sent by Atlassian Jira (v8.20.10#820010)