[jira] [Commented] (IGNITE-15348) Checkstyle should check whitespace after cast token.
[
https://issues.apache.org/jira/browse/IGNITE-15348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17445659#comment-17445659
]
Ignite TC Bot commented on IGNITE-15348:
{panel:title=Branch: [pull/9568/head] Base: [master] : Possible Blockers
(1)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}Platform C++ CMake (Win x64 / Release){color} [[tests 0 Exit
Code |https://ci.ignite.apache.org/viewLog.html?buildId=6275183]]
{panel}
{panel:title=Branch: [pull/9568/head] Base: [master] : No new tests
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=6274110buildTypeId=IgniteTests24Java8_RunAll]
> Checkstyle should check whitespace after cast token.
> -
>
> Key: IGNITE-15348
> URL: https://issues.apache.org/jira/browse/IGNITE-15348
> Project: Ignite
> Issue Type: New Feature
>Reporter: Maksim Timonin
>Assignee: Andrei Demidov
>Priority: Major
> Labels: newbie
> Time Spent: 10m
> Remaining Estimate: 0h
>
> According to Ignite code style [1] there shouldn't a whitespace after the
> ")" token. Let's add check for that.
>
> Solution is add the TYPECAST token to the NoWhitespaceAfter module
>
>
>
>
> Also it is required to fix all issues within repo.
>
> [1]
> https://cwiki.apache.org/confluence/display/IGNITE/Coding+Guidelines#CodingGuidelines-Whitespacesandemptylines
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (IGNITE-14120) select count * returns multiple rows
[ https://issues.apache.org/jira/browse/IGNITE-14120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YuJue Li updated IGNITE-14120: -- Fix Version/s: 2.12 > select count * returns multiple rows > > > Key: IGNITE-14120 > URL: https://issues.apache.org/jira/browse/IGNITE-14120 > Project: Ignite > Issue Type: Bug > Components: sql >Affects Versions: 2.8.1 >Reporter: Isaac Zhu >Assignee: Vladimir Ermakov >Priority: Major > Fix For: 2.12 > > Time Spent: 1h 50m > Remaining Estimate: 0h > > I have a partitioned table which has 1 backup, the *queryParallelism* is set > to 4. > The table primary key is column "ID", > If I do this query: > select count( * ) from my_table where ID = 1000; > It will return 4 rows: > 1 > 0 > 0 > 0 > > If I query by other not primary-key columns of this table, the result is > good, like: > select count( *) from my_table where name = 'abcd' > result is: > 0 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (IGNITE-15942) Add support for h2 1.4.200 version to Ignite and Spring Ignite Data packages
[ https://issues.apache.org/jira/browse/IGNITE-15942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17445470#comment-17445470 ] Ilya Kasnacheev commented on IGNITE-15942: -- I'm not sure it is feasible, I know that H2 has dropped some of APIs needed by Ignite integration. > Add support for h2 1.4.200 version to Ignite and Spring Ignite Data packages > > > Key: IGNITE-15942 > URL: https://issues.apache.org/jira/browse/IGNITE-15942 > Project: Ignite > Issue Type: Improvement >Reporter: Julius Kabugu >Priority: Major > > Today, when we create Ignite thick clients, the ignite client packages and > Spring Ignite Data require h2 database version 1.4.197. Upgrading to 1.4.200 > breaks the clients. However, h2 1.4.197 has been flagged by security scans > due to a number of vulnerabilities. > The request is to update Ignite client packages to use H2 database version > 1.4.200. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (IGNITE-15942) Add support for h2 1.4.200 version to Ignite and Spring Ignite Data packages
Julius Kabugu created IGNITE-15942: -- Summary: Add support for h2 1.4.200 version to Ignite and Spring Ignite Data packages Key: IGNITE-15942 URL: https://issues.apache.org/jira/browse/IGNITE-15942 Project: Ignite Issue Type: Improvement Reporter: Julius Kabugu Today, when we create Ignite thick clients, the ignite client packages and Spring Ignite Data require h2 database version 1.4.197. Upgrading to 1.4.200 breaks the clients. However, h2 1.4.197 has been flagged by security scans due to a number of vulnerabilities. The request is to update Ignite client packages to use H2 database version 1.4.200. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (IGNITE-15941) Flaky test FailureProcessorThreadDumpThrottlingTest.testThrottlingPerFailureType
Andrey N. Gura created IGNITE-15941:
---
Summary: Flaky test
FailureProcessorThreadDumpThrottlingTest.testThrottlingPerFailureType
Key: IGNITE-15941
URL: https://issues.apache.org/jira/browse/IGNITE-15941
Project: Ignite
Issue Type: Bug
Reporter: Andrey N. Gura
Assignee: Andrey N. Gura
Fix For: 2.12
The test
{{FailureProcessorThreadDumpThrottlingTest.testThrottlingPerFailureType}} is
flaky. See stack trace:
{noformat}
java.lang.AssertionError
at org.junit.Assert.fail(Assert.java:86)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertTrue(Assert.java:52)
at
org.apache.ignite.testframework.junits.JUnitAssertAware.assertTrue(JUnitAssertAware.java:33)
at
org.apache.ignite.internal.processors.failure.FailureProcessorThreadDumpThrottlingTest.testThrottlingPerFailureType(FailureProcessorThreadDumpThrottlingTest.java:183)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.apache.ignite.testframework.junits.GridAbstractTest$7.run(GridAbstractTest.java:2402)
at java.lang.Thread.run(Thread.java:748)
{noformat}
The problem is usage of different sources of time in the test and in a
production code. While the test uses system timer for sleeping
({{Thread.sleep}}) the production code uses timer thread which refreshes a
cached time value periodically ({{IgniteUtils.currentTimeMillis}}). As result
the following situation is possible: the test's thread sleeps some given time
(3000 ms), wakes up and check cached time from timer thread which is still lags
behind the test's thread because timer thread didn't get quantum from thread
scheduler. Such situation breaks invariant and test fails.
This could be fixed in two ways:
1. fix the test in such way when the test's thread waits proper time value from
the timer thread
2. fix production code by changing {{U.currentTimeMillis}} call to
{{System.currentTimeMillis}}.
Because failure processor is not performance critical component the second way
is better the the first way because such solution allows to avoid similar bugs
in other tests.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (IGNITE-15922) Create numa-aware allocator for data regions.
[
https://issues.apache.org/jira/browse/IGNITE-15922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17445404#comment-17445404
]
Ignite TC Bot commented on IGNITE-15922:
{panel:title=Branch: [pull/9569/head] Base: [master] : Possible Blockers
(2)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}AWS{color} [[tests 0 Exit Code
|https://ci2.ignite.apache.org/viewLog.html?buildId=6209506]]
{color:#d04437}GCE{color} [[tests 0 Exit Code
|https://ci2.ignite.apache.org/viewLog.html?buildId=6209508]]
{panel}
{panel:title=Branch: [pull/9569/head] Base: [master] : New Tests
(30)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}
{color:#8b}PDS 2{color} [[tests
30|https://ci2.ignite.apache.org/viewLog.html?buildId=6209469]]
* {color:#013220}IgnitePdsTestSuite2:
CdcSelfTest.testMultiNodeConsumption[specificConsistentId=false, walMode=FSYNC,
metricExporter=org.apache.ignite.cdc.CdcSelfTest$$Lambda$17/798639105@3e5fd2b1]
- PASSED{color}
* {color:#013220}IgnitePdsTestSuite2:
CdcSelfTest.testReReadWhenStateWasNotStored[specificConsistentId=false,
walMode=FSYNC,
metricExporter=org.apache.ignite.cdc.CdcSelfTest$$Lambda$17/798639105@3e5fd2b1]
- PASSED{color}
* {color:#013220}IgnitePdsTestSuite2:
CdcSelfTest.testCdcSingleton[specificConsistentId=false, walMode=FSYNC,
metricExporter=org.apache.ignite.cdc.CdcSelfTest$$Lambda$17/798639105@3e5fd2b1]
- PASSED{color}
* {color:#013220}IgnitePdsTestSuite2:
CdcSelfTest.testCdcSingleton[specificConsistentId=true, walMode=FSYNC,
metricExporter=org.apache.ignite.cdc.CdcSelfTest$$Lambda$17/798639105@3e5fd2b1]
- PASSED{color}
* {color:#013220}IgnitePdsTestSuite2:
CdcSelfTest.testMultiNodeConsumption[specificConsistentId=true, walMode=FSYNC,
metricExporter=org.apache.ignite.cdc.CdcSelfTest$$Lambda$17/798639105@3e5fd2b1]
- PASSED{color}
* {color:#013220}IgnitePdsTestSuite2:
CdcSelfTest.testReadBeforeGracefulShutdown[specificConsistentId=false,
walMode=FSYNC,
metricExporter=org.apache.ignite.cdc.CdcSelfTest$$Lambda$17/798639105@3e5fd2b1]
- PASSED{color}
* {color:#013220}IgnitePdsTestSuite2:
CdcSelfTest.testReadAllKeys[specificConsistentId=false, walMode=FSYNC,
metricExporter=org.apache.ignite.cdc.CdcSelfTest$$Lambda$17/798639105@3e5fd2b1]
- PASSED{color}
* {color:#013220}IgnitePdsTestSuite2:
CdcSelfTest.testReadAllKeys[specificConsistentId=true, walMode=FSYNC,
metricExporter=org.apache.ignite.cdc.CdcSelfTest$$Lambda$17/798639105@3e5fd2b1]
- PASSED{color}
* {color:#013220}IgnitePdsTestSuite2:
CdcSelfTest.testCdcSingleton[specificConsistentId=false, walMode=LOG_ONLY,
metricExporter=org.apache.ignite.cdc.CdcSelfTest$$Lambda$17/798639105@3e5fd2b1]
- PASSED{color}
* {color:#013220}IgnitePdsTestSuite2:
CdcSelfTest.testReReadWhenStateWasNotStored[specificConsistentId=true,
walMode=FSYNC,
metricExporter=org.apache.ignite.cdc.CdcSelfTest$$Lambda$17/798639105@3e5fd2b1]
- PASSED{color}
* {color:#013220}IgnitePdsTestSuite2:
CdcSelfTest.testReadBeforeGracefulShutdown[specificConsistentId=true,
walMode=FSYNC,
metricExporter=org.apache.ignite.cdc.CdcSelfTest$$Lambda$17/798639105@3e5fd2b1]
- PASSED{color}
... and 19 new tests
{panel}
[TeamCity *-- Run :: All*
Results|https://ci2.ignite.apache.org/viewLog.html?buildId=6209504buildTypeId=IgniteTests24Java8_RunAll]
> Create numa-aware allocator for data regions.
> -
>
> Key: IGNITE-15922
> URL: https://issues.apache.org/jira/browse/IGNITE-15922
> Project: Ignite
> Issue Type: Improvement
>Reporter: Ivan Daschinsky
>Assignee: Ivan Daschinsky
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Sometimes it is worth to allocate data region on specific numa node or on
> interleaved node (i.e. Intel Optane can be set up as a little bit slower, but
> cheaper than and have more capacity, than DDR, and unified as NUMA nodes).
> Need to write C++ wrapper around {{libnuma}} and extract common allocator
> interface and add as a configuration parameter to {{DataRegionConfiguration}}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (IGNITE-15394) Calcite integration. Query contexts refactoring
[ https://issues.apache.org/jira/browse/IGNITE-15394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Taras Ledkov updated IGNITE-15394: -- Labels: calcite (was: calcite3-required ignite-3) > Calcite integration. Query contexts refactoring > --- > > Key: IGNITE-15394 > URL: https://issues.apache.org/jira/browse/IGNITE-15394 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Taras Ledkov >Assignee: Taras Ledkov >Priority: Major > Labels: calcite > Time Spent: 2h > Remaining Estimate: 0h > > As a part of the query cancel implementation we have to clear query contexts > abstractions: > - separate planning & execution context; > - introduce common parent context that will contain query info. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (IGNITE-15940) [Networking] Class parsing API and simple class descriptor format
[ https://issues.apache.org/jira/browse/IGNITE-15940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Semyon Danilov updated IGNITE-15940: Description: To enable User Object Serialization we need to define class descriptor format and API for parsing java classes into this format. As the first step, we consider simple cases - Serializable/Externalizable classes with primitive or Serializable/Externalizable fields. Look here for reference: https://github.com/gridgain/gridgain-9-ce/blob/iep-67/modules/network/README.md#class-descriptor was: To enable User Object Serialization we need to define class descriptor format and API for parsing java classes into this format. As the first step we consider simple cases - Serializable/Externalizable classes with primitive or Serializable/Externalizable fields. > [Networking] Class parsing API and simple class descriptor format > - > > Key: IGNITE-15940 > URL: https://issues.apache.org/jira/browse/IGNITE-15940 > Project: Ignite > Issue Type: Task > Components: networking >Reporter: Sergey Chugunov >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-alpha4 > > > To enable User Object Serialization we need to define class descriptor format > and API for parsing java classes into this format. > As the first step, we consider simple cases - Serializable/Externalizable > classes with primitive or Serializable/Externalizable fields. > Look here for reference: > https://github.com/gridgain/gridgain-9-ce/blob/iep-67/modules/network/README.md#class-descriptor -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (IGNITE-15781) CacheGroupMetrics spams to heap with java.lang.Integer
[ https://issues.apache.org/jira/browse/IGNITE-15781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maksim Timonin updated IGNITE-15781: Affects Version/s: 2.11 > CacheGroupMetrics spams to heap with java.lang.Integer > -- > > Key: IGNITE-15781 > URL: https://issues.apache.org/jira/browse/IGNITE-15781 > Project: Ignite > Issue Type: Bug >Affects Versions: 2.11 >Reporter: Maksim Timonin >Assignee: Maksim Timonin >Priority: Major > > Check GridDhtPartitionMap.get() and similar methods that expect Integer, but > CacheMetrics uses prmitive types. Autoboxing pollutes heap much if metrics > asks frequently. > > Metrics to check: getMinimumNumberOfPartitionCopies() > this metrics iterates over all nodes and partitions and invokes get(int) on > every iteration. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (IGNITE-15781) CacheGroupMetrics spams to heap with java.lang.Integer
[ https://issues.apache.org/jira/browse/IGNITE-15781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maksim Timonin updated IGNITE-15781: Ignite Flags: (was: Docs Required,Release Notes Required) > CacheGroupMetrics spams to heap with java.lang.Integer > -- > > Key: IGNITE-15781 > URL: https://issues.apache.org/jira/browse/IGNITE-15781 > Project: Ignite > Issue Type: Bug >Reporter: Maksim Timonin >Assignee: Maksim Timonin >Priority: Major > > Check GridDhtPartitionMap.get() and similar methods that expect Integer, but > CacheMetrics uses prmitive types. Autoboxing pollutes heap much if metrics > asks frequently. > > Metrics to check: getMinimumNumberOfPartitionCopies() > this metrics iterates over all nodes and partitions and invokes get(int) on > every iteration. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (IGNITE-15940) [Networking] Class parsing API and simple class descriptor format
[ https://issues.apache.org/jira/browse/IGNITE-15940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-15940: - Labels: ignite-3 (was: ) > [Networking] Class parsing API and simple class descriptor format > - > > Key: IGNITE-15940 > URL: https://issues.apache.org/jira/browse/IGNITE-15940 > Project: Ignite > Issue Type: Task > Components: networking >Reporter: Sergey Chugunov >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-alpha4 > > > To enable User Object Serialization we need to define class descriptor format > and API for parsing java classes into this format. > As the first step we consider simple cases - Serializable/Externalizable > classes with primitive or Serializable/Externalizable fields. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (IGNITE-15912) Merge SQL calcite query contexts refactoring to 3.0
[ https://issues.apache.org/jira/browse/IGNITE-15912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17445256#comment-17445256 ] Konstantin Orlov commented on IGNITE-15912: --- [~tledkov-gridgain] , LGTM! > Merge SQL calcite query contexts refactoring to 3.0 > --- > > Key: IGNITE-15912 > URL: https://issues.apache.org/jira/browse/IGNITE-15912 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Taras Ledkov >Assignee: Taras Ledkov >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-alpha4 > > > Adopt the query contexts refactoring patch (IGNITE-15394) for the Ignite 3.0. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (IGNITE-15940) [Networking] Class parsing API and simple class descriptor format
Sergey Chugunov created IGNITE-15940: Summary: [Networking] Class parsing API and simple class descriptor format Key: IGNITE-15940 URL: https://issues.apache.org/jira/browse/IGNITE-15940 Project: Ignite Issue Type: Task Components: networking Reporter: Sergey Chugunov Fix For: 3.0.0-alpha4 To enable User Object Serialization we need to define class descriptor format and API for parsing java classes into this format. As the first step we consider simple cases - Serializable/Externalizable classes with primitive or Serializable/Externalizable fields. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (IGNITE-15932) Implement storage for locks
[ https://issues.apache.org/jira/browse/IGNITE-15932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Scherbakov updated IGNITE-15932: --- Ignite Flags: (was: Docs Required,Release Notes Required) > Implement storage for locks > --- > > Key: IGNITE-15932 > URL: https://issues.apache.org/jira/browse/IGNITE-15932 > Project: Ignite > Issue Type: Improvement > Environment: Currently locks are stored in the heap and can take > unlimited amount of memory. > Need to implement storage for a locks with a configurable limits. It can even > store locks off-heap of use disk spilling. > Lock attempt must fail if a storage is overflowed. > >Reporter: Alexey Scherbakov >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (IGNITE-15289) Collect global statistics
[ https://issues.apache.org/jira/browse/IGNITE-15289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yury Gerzhedovich updated IGNITE-15289: --- Labels: calcite calcite3-required (was: ) > Collect global statistics > - > > Key: IGNITE-15289 > URL: https://issues.apache.org/jira/browse/IGNITE-15289 > Project: Ignite > Issue Type: Bug > Components: sql >Reporter: Alexander Belyak >Assignee: Alexander Belyak >Priority: Major > Labels: calcite, calcite3-required > Time Spent: 7h > Remaining Estimate: 0h > > Collect global statistics by request to use in Calcite cost model. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (IGNITE-15868) Unexpected command: PROBE when authorization is enabled
[
https://issues.apache.org/jira/browse/IGNITE-15868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vyacheslav Koptilin reassigned IGNITE-15868:
Assignee: Dmitriy Borunov
> Unexpected command: PROBE when authorization is enabled
> ---
>
> Key: IGNITE-15868
> URL: https://issues.apache.org/jira/browse/IGNITE-15868
> Project: Ignite
> Issue Type: Bug
> Components: platforms, thin client
>Affects Versions: 2.12
>Reporter: Dmitriy Borunov
>Assignee: Dmitriy Borunov
>Priority: Major
> Fix For: 2.13
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Rest API Probe is not working when Control Center authorisation is enabled
> {noformat}
> [2021-11-01T13:35:33,550][ERROR][rest-#125%nebula-node%][GridRestProcessor]
> Runtime error caught during grid runnable execution: GridWorker
> [name=rest-proc-work
> er, igniteInstanceName=nebula-node, finished=false,
> heartbeatTs=1635773733544, hashCode=1386655371, interrupted=false,
> runner=rest-#125%nebula-node%]
> java.lang.AssertionError: Unexpected command: PROBE
> at
> org.apache.ignite.internal.processors.rest.GridRestProcessor.authorize(GridRestProcessor.java:968)
> ~[ignite-core-8.8.10.jar:8.8.10]
> at
> org.apache.ignite.internal.processors.rest.GridRestProcessor.handleRequest(GridRestProcessor.java:286)
> ~[ignite-core-8.8.10.jar:8.8.10]
> at
> org.apache.ignite.internal.processors.rest.GridRestProcessor.access$100(GridRestProcessor.java:108)
> ~[ignite-core-8.8.10.jar:8.8.10]
> at
> org.apache.ignite.internal.processors.rest.GridRestProcessor$2.body(GridRestProcessor.java:183)
> ~[ignite-core-8.8.10.jar:8.8.10]
> at
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:119)
> [ignite-core-8.8.10.jar:8.8.10]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> [?:?]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> [?:?]
> at java.lang.Thread.run(Thread.java:829) [?:?]{noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (IGNITE-13558) GridCacheProcessor should implement better parallelization when restoring partition states on startup
[ https://issues.apache.org/jira/browse/IGNITE-13558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17445110#comment-17445110 ] Denis Chudov commented on IGNITE-13558: --- [~sergeychugunov] seems we are ready to merge. > GridCacheProcessor should implement better parallelization when restoring > partition states on startup > - > > Key: IGNITE-13558 > URL: https://issues.apache.org/jira/browse/IGNITE-13558 > Project: Ignite > Issue Type: Improvement > Components: persistence >Reporter: Sergey Chugunov >Assignee: Denis Chudov >Priority: Major > Time Spent: 4h 10m > Remaining Estimate: 0h > > GridCacheProcessor#restorePartitionStates method tries to employ striped pool > to restore partition states in parallel but level of parallelization is down > only to cache group per thread. > It is not enough and not utilizes resources effectively in case of one cache > group much bigger than the others. > We need to parallel restore process down to individual partitions to get the > most from the available resources and speed up node startup. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (IGNITE-13558) GridCacheProcessor should implement better parallelization when restoring partition states on startup
[ https://issues.apache.org/jira/browse/IGNITE-13558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17445109#comment-17445109 ] Denis Chudov commented on IGNITE-13558: --- PR 9565 is almost equal to PR 9327 except of last merge commit. > GridCacheProcessor should implement better parallelization when restoring > partition states on startup > - > > Key: IGNITE-13558 > URL: https://issues.apache.org/jira/browse/IGNITE-13558 > Project: Ignite > Issue Type: Improvement > Components: persistence >Reporter: Sergey Chugunov >Assignee: Denis Chudov >Priority: Major > Time Spent: 4h 10m > Remaining Estimate: 0h > > GridCacheProcessor#restorePartitionStates method tries to employ striped pool > to restore partition states in parallel but level of parallelization is down > only to cache group per thread. > It is not enough and not utilizes resources effectively in case of one cache > group much bigger than the others. > We need to parallel restore process down to individual partitions to get the > most from the available resources and speed up node startup. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (IGNITE-15093) MVP for transactional protocol
[ https://issues.apache.org/jira/browse/IGNITE-15093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Scherbakov updated IGNITE-15093: --- Description: It is necessary to develop an MVP for transactional protocol components. It seems we need at least IGNITE-15085 IGNITE-15086 and IGNITE-15057 for this. The simplest implementation will do. was: It is necessary to develop an MVP for transactional protocol components. It seems we need at least IGNITE-15083 IGNITE-15085 IGNITE-15086 and IGNITE-15057 for this. The simplest implementation will do. > MVP for transactional protocol > -- > > Key: IGNITE-15093 > URL: https://issues.apache.org/jira/browse/IGNITE-15093 > Project: Ignite > Issue Type: Task >Reporter: Alexey Scherbakov >Assignee: Alexey Scherbakov >Priority: Major > Labels: iep-61, ignite-3 > > It is necessary to develop an MVP for transactional protocol components. > It seems we need at least IGNITE-15085 IGNITE-15086 and IGNITE-15057 for this. > The simplest implementation will do. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (IGNITE-15937) Fix transactions exception model
[ https://issues.apache.org/jira/browse/IGNITE-15937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Scherbakov updated IGNITE-15937: --- Description: Make sure a transaction always use TransactionException as root cause. Implement error codes. was: Make sure a transaction always throws TransactionException as root cause. Implement error codes. > Fix transactions exception model > > > Key: IGNITE-15937 > URL: https://issues.apache.org/jira/browse/IGNITE-15937 > Project: Ignite > Issue Type: Improvement >Reporter: Alexey Scherbakov >Priority: Major > > Make sure a transaction always use TransactionException as root cause. > Implement error codes. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (IGNITE-15937) Fix transactions exception model
[ https://issues.apache.org/jira/browse/IGNITE-15937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Scherbakov updated IGNITE-15937: --- Summary: Fix transactions exception model (was: Fix tx exception model) > Fix transactions exception model > > > Key: IGNITE-15937 > URL: https://issues.apache.org/jira/browse/IGNITE-15937 > Project: Ignite > Issue Type: Improvement >Reporter: Alexey Scherbakov >Priority: Major > > Make sure a transaction always throws TransactionException as root cause. > Implement error codes. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (IGNITE-15939) Fix .net tests related to reading non existing keys
Alexey Scherbakov created IGNITE-15939: -- Summary: Fix .net tests related to reading non existing keys Key: IGNITE-15939 URL: https://issues.apache.org/jira/browse/IGNITE-15939 Project: Ignite Issue Type: Improvement Reporter: Alexey Scherbakov Currenlty getAll will return null if a value is not found for a key (undelying collection is a List) Need to adapt .net tests [1] [2] [1] TestGetAllReturnsRecordsForExistingKeys [2] TestGetAllNonExistentKeysReturnsEmptyList -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (IGNITE-15938) A race between async ops and tx finish
Alexey Scherbakov created IGNITE-15938: -- Summary: A race between async ops and tx finish Key: IGNITE-15938 URL: https://issues.apache.org/jira/browse/IGNITE-15938 Project: Ignite Issue Type: Improvement Reporter: Alexey Scherbakov There is a race between tx async ops and tx finish ops (commit, rollback). User op futures are not tracked in any way. Possible solution: deny commit or rollback if some async ops are on the fly. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (IGNITE-15937) Fix tx exception model
Alexey Scherbakov created IGNITE-15937: -- Summary: Fix tx exception model Key: IGNITE-15937 URL: https://issues.apache.org/jira/browse/IGNITE-15937 Project: Ignite Issue Type: Improvement Reporter: Alexey Scherbakov Make sure a transaction always throws TransactionException as root cause. Implement error codes. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (IGNITE-15936) Implement timeouts
Alexey Scherbakov created IGNITE-15936: -- Summary: Implement timeouts Key: IGNITE-15936 URL: https://issues.apache.org/jira/browse/IGNITE-15936 Project: Ignite Issue Type: Improvement Reporter: Alexey Scherbakov Tx should be rolled back if a timeout is exceeded [1] [1] org.apache.ignite.tx.IgniteTransactions#withTimeout -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (IGNITE-15935) Implement tx near cache
Alexey Scherbakov created IGNITE-15935: -- Summary: Implement tx near cache Key: IGNITE-15935 URL: https://issues.apache.org/jira/browse/IGNITE-15935 Project: Ignite Issue Type: Improvement Reporter: Alexey Scherbakov Currently all tx reads are routed to a raft group leader. It is possible to use a local cache on a near tx side for ad-hoc reads. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (IGNITE-15934) Optimize versioned row store
Alexey Scherbakov created IGNITE-15934: -- Summary: Optimize versioned row store Key: IGNITE-15934 URL: https://issues.apache.org/jira/browse/IGNITE-15934 Project: Ignite Issue Type: Improvement Reporter: Alexey Scherbakov Git rid of data copying. Do not store keys twice. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (IGNITE-15933) Implement asynchronous rollback
Alexey Scherbakov created IGNITE-15933: -- Summary: Implement asynchronous rollback Key: IGNITE-15933 URL: https://issues.apache.org/jira/browse/IGNITE-15933 Project: Ignite Issue Type: Improvement Reporter: Alexey Scherbakov Need to make sure a transaction can be asynchronously rolled back, concurrently with a user activity. This requires additional test scenarios covering the aforementioned feature. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (IGNITE-15932) Implement storage for locks
[ https://issues.apache.org/jira/browse/IGNITE-15932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Scherbakov updated IGNITE-15932: --- Environment: Currently locks are stored in the heap and can take unlimited amount of memory. Need to implement storage for a locks with a configurable limits. It can even store locks off-heap of use disk spilling. Lock attempt must fail if a storage is overflowed. was: Currently locks are stored in the heap and can take unlimited amount of memory. Need to implement storage for a locks with a configurable limits. It can even store locks off-heap of use disk spilling. > Implement storage for locks > --- > > Key: IGNITE-15932 > URL: https://issues.apache.org/jira/browse/IGNITE-15932 > Project: Ignite > Issue Type: Improvement > Environment: Currently locks are stored in the heap and can take > unlimited amount of memory. > Need to implement storage for a locks with a configurable limits. It can even > store locks off-heap of use disk spilling. > Lock attempt must fail if a storage is overflowed. > >Reporter: Alexey Scherbakov >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (IGNITE-15932) Implement storage for locks
Alexey Scherbakov created IGNITE-15932: -- Summary: Implement storage for locks Key: IGNITE-15932 URL: https://issues.apache.org/jira/browse/IGNITE-15932 Project: Ignite Issue Type: Improvement Environment: Currently locks are stored in the heap and can take unlimited amount of memory. Need to implement storage for a locks with a configurable limits. It can even store locks off-heap of use disk spilling. Reporter: Alexey Scherbakov -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (IGNITE-15931) Implement storage for tx states
Alexey Scherbakov created IGNITE-15931: -- Summary: Implement storage for tx states Key: IGNITE-15931 URL: https://issues.apache.org/jira/browse/IGNITE-15931 Project: Ignite Issue Type: Improvement Environment: Currently tx states [1] are stored in memory and will be lost on node failure, but they shouldn't. Need to store states in the Storage (similar to PartitionStorage). [1] org.apache.ignite.internal.tx.impl.TxManagerImpl#states Reporter: Alexey Scherbakov -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (IGNITE-15930) Implement explicit tx argument in table API
Alexey Scherbakov created IGNITE-15930: -- Summary: Implement explicit tx argument in table API Key: IGNITE-15930 URL: https://issues.apache.org/jira/browse/IGNITE-15930 Project: Ignite Issue Type: Improvement Reporter: Alexey Scherbakov All data related public methods on table APIs must accept Transaction as an argument. null value means "autocommit" mode. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (IGNITE-15929) Local node id generation might not work
Alexey Scherbakov created IGNITE-15929: -- Summary: Local node id generation might not work Key: IGNITE-15929 URL: https://issues.apache.org/jira/browse/IGNITE-15929 Project: Ignite Issue Type: Bug Reporter: Alexey Scherbakov Current implementation of org.apache.ignite.internal.tx.Timestamp#getLocalNodeId is error prone, because java.net.NetworkInterface#getByInetAddress can return null. Need to fallback to random bytes in this case (or maybe some other strategy) -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (IGNITE-15087) Implement support for transactional SQL
[ https://issues.apache.org/jira/browse/IGNITE-15087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Scherbakov updated IGNITE-15087: --- Description: SQL execution engine must interact with LockManager and TxManager to achieve desired isolation, as described in [1] Indexes must be aware of tx isolation. [1] [https://github.com/apache/ignite-3/tree/main/modules/transactions] was: SQL execution engine must interact with LockManager and TxManager to achieve desired isolation, as described in [1] [1] https://github.com/apache/ignite-3/tree/main/modules/transactions > Implement support for transactional SQL > --- > > Key: IGNITE-15087 > URL: https://issues.apache.org/jira/browse/IGNITE-15087 > Project: Ignite > Issue Type: Task >Reporter: Alexey Scherbakov >Priority: Major > Labels: iep-61, ignite-3 > > SQL execution engine must interact with LockManager and TxManager to achieve > desired isolation, as described in [1] > Indexes must be aware of tx isolation. > [1] [https://github.com/apache/ignite-3/tree/main/modules/transactions] -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (IGNITE-15928) Optimize transactional IT tests to reuse nodes between different scenarios
Alexey Scherbakov created IGNITE-15928: -- Summary: Optimize transactional IT tests to reuse nodes between different scenarios Key: IGNITE-15928 URL: https://issues.apache.org/jira/browse/IGNITE-15928 Project: Ignite Issue Type: Improvement Reporter: Alexey Scherbakov Avoid restarting nodes between tests -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (IGNITE-15927) Implement one phase commit
Alexey Scherbakov created IGNITE-15927: -- Summary: Implement one phase commit Key: IGNITE-15927 URL: https://issues.apache.org/jira/browse/IGNITE-15927 Project: Ignite Issue Type: Improvement Reporter: Alexey Scherbakov If all keys in the implicit transaction belong to a same partition in can be committed in one round-trip. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (IGNITE-15094) Support for transactions in a client
[ https://issues.apache.org/jira/browse/IGNITE-15094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Scherbakov updated IGNITE-15094: --- Epic Link: IGNITE-15081 > Support for transactions in a client > > > Key: IGNITE-15094 > URL: https://issues.apache.org/jira/browse/IGNITE-15094 > Project: Ignite > Issue Type: Task >Reporter: Alexey Scherbakov >Priority: Major > Labels: iep-61, ignite-3 > > Implement the support for transactions (interop with tx coordinator) in a > thin client. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (IGNITE-15013) Design the transaction protocol for Ignite 3
[ https://issues.apache.org/jira/browse/IGNITE-15013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Scherbakov updated IGNITE-15013: --- Epic Link: IGNITE-15081 > Design the transaction protocol for Ignite 3 > > > Key: IGNITE-15013 > URL: https://issues.apache.org/jira/browse/IGNITE-15013 > Project: Ignite > Issue Type: Task >Affects Versions: 3.0.0-alpha2 >Reporter: Alexey Scherbakov >Assignee: Alexey Scherbakov >Priority: Major > Labels: iep-61, ignite-3 > Fix For: 3.0.0-alpha3 > > Time Spent: 0.5h > Remaining Estimate: 0h > > As a result of this task, the transactional protocol design must be proposed. > It's ok to have a protocol design to be in some intermediate state, so, not > all desired features can be available. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (IGNITE-15093) MVP for transactional protocol
[ https://issues.apache.org/jira/browse/IGNITE-15093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Scherbakov updated IGNITE-15093: --- Epic Link: IGNITE-15081 > MVP for transactional protocol > -- > > Key: IGNITE-15093 > URL: https://issues.apache.org/jira/browse/IGNITE-15093 > Project: Ignite > Issue Type: Task >Reporter: Alexey Scherbakov >Assignee: Alexey Scherbakov >Priority: Major > Labels: iep-61, ignite-3 > > It is necessary to develop an MVP for transactional protocol components. > It seems we need at least IGNITE-15083 IGNITE-15085 IGNITE-15086 and > IGNITE-15057 for this. > The simplest implementation will do. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (IGNITE-15057) Implement LockManager with deadlock prevention based on operation ordering
[ https://issues.apache.org/jira/browse/IGNITE-15057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Scherbakov updated IGNITE-15057: --- Epic Link: IGNITE-15081 > Implement LockManager with deadlock prevention based on operation ordering > -- > > Key: IGNITE-15057 > URL: https://issues.apache.org/jira/browse/IGNITE-15057 > Project: Ignite > Issue Type: Task >Affects Versions: 3.0.0-alpha2 >Reporter: Alexey Scherbakov >Assignee: Alexey Scherbakov >Priority: Major > Labels: iep-61, ignite-3 > Fix For: 3.0.0-alpha3 > > Time Spent: 50m > Remaining Estimate: 0h > > We assume two phase locking concurrency control in the first version of tx > protocol in Ignite 3. > We need a LockManager implementing this functionality. > If a shared/exclusive lock is acquired in incompatible mode, only "newest" > operations are allowed to wait for "oldest", according to ordering based on > globally comparable timestamp. > More specifically, suppose a transaction Ti tries to wait for Tj => lock > order is [Tj, Ti]. If Ti has lower priority than Tj (i.e., Ti is younger than > Tj), then Ti is permitted to wait => [Tj=10, Ti=20]. Otherwise [Tj=20, Ti=10] > => Ti is aborted. > Aborted transactions must be restarted preserving it's timestamp. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (IGNITE-15086) Design a public tx API
[
https://issues.apache.org/jira/browse/IGNITE-15086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Scherbakov updated IGNITE-15086:
---
Epic Link: IGNITE-15081
> Design a public tx API
> --
>
> Key: IGNITE-15086
> URL: https://issues.apache.org/jira/browse/IGNITE-15086
> Project: Ignite
> Issue Type: Task
>Reporter: Alexey Scherbakov
>Assignee: Alexey Scherbakov
>Priority: Major
> Labels: iep-61, ignite-3
> Fix For: 3.0.0-alpha3
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Design a public tx API.
> The proposed design includes async and sync APIs to execute a transaction.
> The transaction facade looks like this:
> {code:java}
> /**
> * Ignite Transactions facade.
> */
> public interface IgniteTransactions {
> /**
> * Begins the transaction.
> *
> * @return The future.
> */
> CompletableFuture beginAsync();
> /**
> * Synchronously executes a closure within a transaction.
> *
> * If the closure is executed normally (no exceptions), the transaction
> is automatically committed.
> *
> * @param clo The closure.
> */
> void runInTransaction(Consumer clo);
> }
> {code}
> Transacton interface:
> {code:java}
> /**
> * The transaction.
> */
> public interface Transaction {
> /**
> * Synchronously commits a transaction.
> * Does nothing if it's already finished by committing or rolling back.
> */
> void commit();
> /**
> * Asynchronously commits a transaction.
> * Does nothing if it's already finished by committing or rolling back.
> *
> * @return The future.
> */
> CompletableFuture commitAsync();
> /**
> * Synchronously rolls back a transaction.
> * Does nothing if it's already finished by committing or rolling back.
> */
> void rollback();
> /**
> * Asynchronously rolls back a transaction.
> * Does nothing if it's already finished by committing or rolling back.
> *
> * @return The future.
> */
> CompletableFuture rollbackAsync();
> }
> {code}
> Example of synchronous transaction:
> {code:java}
> igniteTransactions.runInTransaction(tx -> {
> Table txTable = table.withTransaction(tx); // table view enlisted
> in the transaction.
> txTable.upsertAsync(...);
> tx.commit();
> });
> {code}
> Example of asynchronous transaction:
> {code:java}
> igniteTransactions.beginAsync()
> .thenApply(tx -> accounts.withTransaction(tx))
> .thenCompose(txTbl -> txTbl.upsertAsync(...).thenApply(i ->
> txTbl.transaction()))
> .thenCompose(Transaction::commitAsync);
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (IGNITE-15092) Implement a vacuum
[ https://issues.apache.org/jira/browse/IGNITE-15092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Scherbakov updated IGNITE-15092: --- Epic Link: IGNITE-15081 > Implement a vacuum > -- > > Key: IGNITE-15092 > URL: https://issues.apache.org/jira/browse/IGNITE-15092 > Project: Ignite > Issue Type: Task >Reporter: Alexey Scherbakov >Priority: Major > Labels: iep-61, ignite-3 > > During pre-writes some excessive data is written to support tx aborting. > This data must be eventually removed. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (IGNITE-15091) Implement tx recovery protocol
[ https://issues.apache.org/jira/browse/IGNITE-15091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Scherbakov updated IGNITE-15091: --- Epic Link: IGNITE-15081 > Implement tx recovery protocol > -- > > Key: IGNITE-15091 > URL: https://issues.apache.org/jira/browse/IGNITE-15091 > Project: Ignite > Issue Type: Task >Reporter: Alexey Scherbakov >Priority: Major > Labels: iep-61, ignite-3 > > This protocol should keep consistency on different kind of node failures. > It will be similar to Ignite 2, see [1] > [1] https://github.com/apache/ignite-3/tree/main/modules/transactions -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (IGNITE-15087) Implement support for transactional SQL
[ https://issues.apache.org/jira/browse/IGNITE-15087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Scherbakov updated IGNITE-15087: --- Epic Link: IGNITE-15081 > Implement support for transactional SQL > --- > > Key: IGNITE-15087 > URL: https://issues.apache.org/jira/browse/IGNITE-15087 > Project: Ignite > Issue Type: Task >Reporter: Alexey Scherbakov >Priority: Major > Labels: iep-61, ignite-3 > > SQL execution engine must interact with LockManager and TxManager to achieve > desired isolation, as described in [1] > [1] https://github.com/apache/ignite-3/tree/main/modules/transactions -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (IGNITE-15081) Implement transactions support in Ignite-3
[ https://issues.apache.org/jira/browse/IGNITE-15081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Scherbakov updated IGNITE-15081: --- Epic Name: Ignite 3 transactions > Implement transactions support in Ignite-3 > -- > > Key: IGNITE-15081 > URL: https://issues.apache.org/jira/browse/IGNITE-15081 > Project: Ignite > Issue Type: Epic >Reporter: Alexey Scherbakov >Priority: Major > Labels: iep-61, ignite-3 > Fix For: 3.0 > > > Umbrella: Design and implement transactions support. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (IGNITE-15129) Improve timestamp generation for transactions
[ https://issues.apache.org/jira/browse/IGNITE-15129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Scherbakov updated IGNITE-15129: --- Epic Link: IGNITE-15081 > Improve timestamp generation for transactions > - > > Key: IGNITE-15129 > URL: https://issues.apache.org/jira/browse/IGNITE-15129 > Project: Ignite > Issue Type: Improvement >Reporter: Alexey Scherbakov >Priority: Major > Labels: iep-61, ignite-3 > > Current timestamp implementation doesn't include globalId part, so is useful > only then all transactions ids are generated on single node. > It should be improved by adding global unique jvm id to the timestamp. > Additionally, current timestamp uses nanoSeconds, which is far away from real > time. > Instead, NTP timestamp can be used as a localTime part. > IgniteUuid can be used for this purpose as well - it's the simplest solution. > Also, the performance of current implementation may not be optimal - probably > it should be CAS powered or/and use optimistic locking (StampedLock). -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (IGNITE-15085) Implement tx coordination
[
https://issues.apache.org/jira/browse/IGNITE-15085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Scherbakov updated IGNITE-15085:
---
Epic Link: IGNITE-15081
> Implement tx coordination
> -
>
> Key: IGNITE-15085
> URL: https://issues.apache.org/jira/browse/IGNITE-15085
> Project: Ignite
> Issue Type: Task
>Reporter: Alexey Scherbakov
>Assignee: Alexey Scherbakov
>Priority: Major
> Labels: iep-61, ignite-3
> Time Spent: 7h 50m
> Remaining Estimate: 0h
>
> This ticket implies the implementation of tx coordination, as described in
> [1].
> This includes:
> * TxManager - top-level manager for tx state and coordination
> * pre-writes - each tx write is pre-written to partition store in special
> format
> * replicated tx state - tx state is stored in the partition's raft group
> * integration with lock manager (see *precaution* chapter in [1])
> The example of single key tx:
> {noformat}
> Tx client TxCoordinator
> Partition leaseholder.
> tx.start
> ->
> assign timestamp (id)
> txstate = PENDING
> <-
> table.put(k,v)
> ->
> enlist(partition(k));
> lh = getLeaseholder(partition(k))
> send UpsertCommand(k) to lh
>
> >
>
>replicate txstate = PENDING
>
>lockManager.tryAcquire(k,timestamp);
>
>wait for completion async
>
>prewrite(k, v) -- replicate to all replicas
> repeat for each enlisted partition...
> <-
> tx.finish - commit or rollback
> ->
> send finish request to all remote enlisted nodes
>
> >
>
>replicate txstate = COMMITTED/ABORTED
> txState = COMMITTED/ABORTED
>lockManager.tryRelease(k, timestamp)
>
> <
>
> when all leasholders are replied,
> reply to initiator
> <
> {noformat}
> [1] [https://github.com/apache/ignite-3/tree/main/modules/transactions]
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (IGNITE-15083) Implement leaseholder selection and failover
[ https://issues.apache.org/jira/browse/IGNITE-15083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Scherbakov updated IGNITE-15083: --- Epic Link: IGNITE-15081 > Implement leaseholder selection and failover > > > Key: IGNITE-15083 > URL: https://issues.apache.org/jira/browse/IGNITE-15083 > Project: Ignite > Issue Type: Task >Reporter: Alexey Scherbakov >Priority: Major > Labels: iep-61, ignite-3 > > The tx design [1] assumes the presence of leaseholder (stronger form of a > leader) for each partition raft group. > * Leaseholder is used for reads without majority heartbeat > * Leaseholder is used for lock requests processing. > * Only one leaseholder should exists at a time. > * Leaseholders map is stored in metastore (can be discovered by asking group > members in the first approach) > * Tx coordinators are subscribed for leaseholder status updates. > The lease acquisiotion implementation details are open for now [2]. > Jraft includes expiration based leases (similar to yugabyte's), and this > should be enough for the first approach. > [1] https://github.com/apache/ignite-3/tree/main/modules/transactions > [2] Cocroach and tikv use raft log based approach. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (IGNITE-15081) Implement transactions support in Ignite-3
[ https://issues.apache.org/jira/browse/IGNITE-15081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Scherbakov updated IGNITE-15081: --- Issue Type: Epic (was: New Feature) > Implement transactions support in Ignite-3 > -- > > Key: IGNITE-15081 > URL: https://issues.apache.org/jira/browse/IGNITE-15081 > Project: Ignite > Issue Type: Epic >Reporter: Alexey Scherbakov >Priority: Major > Labels: iep-61, ignite-3 > Fix For: 3.0 > > > Umbrella: Design and implement transactions support. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (IGNITE-15081) Implement transactions support in Ignite-3
[ https://issues.apache.org/jira/browse/IGNITE-15081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Scherbakov updated IGNITE-15081: --- Issue Type: New Feature (was: Epic) > Implement transactions support in Ignite-3 > -- > > Key: IGNITE-15081 > URL: https://issues.apache.org/jira/browse/IGNITE-15081 > Project: Ignite > Issue Type: New Feature >Reporter: Alexey Scherbakov >Priority: Major > Labels: iep-61, ignite-3 > Fix For: 3.0 > > > Umbrella: Design and implement transactions support. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (IGNITE-15081) Implement transactions support in Ignite-3
[ https://issues.apache.org/jira/browse/IGNITE-15081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Scherbakov updated IGNITE-15081: --- Issue Type: Epic (was: New Feature) > Implement transactions support in Ignite-3 > -- > > Key: IGNITE-15081 > URL: https://issues.apache.org/jira/browse/IGNITE-15081 > Project: Ignite > Issue Type: Epic >Reporter: Alexey Scherbakov >Priority: Major > Labels: iep-61, ignite-3 > Fix For: 3.0 > > > Umbrella: Design and implement transactions support. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (IGNITE-15081) Implement transactions support in Ignite-3
[ https://issues.apache.org/jira/browse/IGNITE-15081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Scherbakov updated IGNITE-15081: --- Issue Type: New Feature (was: Epic) > Implement transactions support in Ignite-3 > -- > > Key: IGNITE-15081 > URL: https://issues.apache.org/jira/browse/IGNITE-15081 > Project: Ignite > Issue Type: New Feature >Reporter: Alexey Scherbakov >Priority: Major > Labels: iep-61, ignite-3 > Fix For: 3.0 > > > Umbrella: Design and implement transactions support. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (IGNITE-15081) Implement transactions support in Ignite-3
[ https://issues.apache.org/jira/browse/IGNITE-15081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Scherbakov updated IGNITE-15081: --- Issue Type: Epic (was: New Feature) > Implement transactions support in Ignite-3 > -- > > Key: IGNITE-15081 > URL: https://issues.apache.org/jira/browse/IGNITE-15081 > Project: Ignite > Issue Type: Epic >Reporter: Alexey Scherbakov >Priority: Major > Labels: iep-61, ignite-3 > Fix For: 3.0 > > > Umbrella: Design and implement transactions support. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (IGNITE-15096) Schema events processing refactoring.
[ https://issues.apache.org/jira/browse/IGNITE-15096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yury Gerzhedovich resolved IGNITE-15096. Resolution: Invalid not an actual > Schema events processing refactoring. > - > > Key: IGNITE-15096 > URL: https://issues.apache.org/jira/browse/IGNITE-15096 > Project: Ignite > Issue Type: Improvement >Reporter: Andrey Mashenkov >Assignee: Andrey Mashenkov >Priority: Minor > Labels: ignite-3 > Time Spent: 40m > Remaining Estimate: 0h > > All table events with schema events are processed in TableManager listener. > Let's > * -Replace ConfigurationListener with ConfigurationNamedListListener to > simplify the diff calculation between old/new configs.- > Resolved in IGNITE-15404 > * -Register a separate per-table listener for schema events on table > creation.- > Resolved in IGNITE-15409 > Update: > Now we register a per-table listener for schema changes + listener for > partition affinity assignment. > A single listener for schema changes and single listener for affinity changes > will be enough, > but table registration may be tricky a bit. > Let's discuss and fix this. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (IGNITE-13558) GridCacheProcessor should implement better parallelization when restoring partition states on startup
[
https://issues.apache.org/jira/browse/IGNITE-13558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17445024#comment-17445024
]
Ignite TC Bot commented on IGNITE-13558:
{panel:title=Branch: [pull/9565/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/9565/head] Base: [master] : New Tests
(1023)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}
{color:#8b}Platform C++ CMake (Win x64 / Release){color} [[tests
1023|https://ci.ignite.apache.org/viewLog.html?buildId=6271258]]
* {color:#013220}IgniteThinClientTest: SqlFieldsQueryTestSuite: SelectTwoValues
- PASSED{color}
* {color:#013220}IgniteOdbcTest: SqlNumericFunctionTestSuite:
TestNumericFunctionFloor - PASSED{color}
* {color:#013220}IgniteThinClientTest: SqlFieldsQueryTestSuite:
SelectSingleValue - PASSED{color}
* {color:#013220}IgniteOdbcTest: SqlNumericFunctionTestSuite:
TestNumericFunctionLog - PASSED{color}
* {color:#013220}IgniteThinClientTest: SqlFieldsQueryTestSuite:
CreateTableInsertSelect - PASSED{color}
* {color:#013220}IgniteOdbcTest: SqlDateTimeFunctionTestSuite: TestCurrentDate
- PASSED{color}
* {color:#013220}IgniteThinClientTest: SqlFieldsQueryTestSuite:
SelectTwoValuesInDifferentOrder - PASSED{color}
* {color:#013220}IgniteCoreTest: ClusterTestSuite: IgniteForCacheNodes -
PASSED{color}
* {color:#013220}IgniteCoreTest: ClusterTestSuite: IgniteForClientNodes -
PASSED{color}
* {color:#013220}IgniteCoreTest: CacheQueryTestSuite:
TestFieldsQueryByteArrayInsertSelect - PASSED{color}
* {color:#013220}IgniteCoreTest: ClusterTestSuite: IgniteForDaemons -
PASSED{color}
... and 1012 new tests
{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=6271351buildTypeId=IgniteTests24Java8_RunAll]
> GridCacheProcessor should implement better parallelization when restoring
> partition states on startup
> -
>
> Key: IGNITE-13558
> URL: https://issues.apache.org/jira/browse/IGNITE-13558
> Project: Ignite
> Issue Type: Improvement
> Components: persistence
>Reporter: Sergey Chugunov
>Assignee: Denis Chudov
>Priority: Major
> Time Spent: 4h 10m
> Remaining Estimate: 0h
>
> GridCacheProcessor#restorePartitionStates method tries to employ striped pool
> to restore partition states in parallel but level of parallelization is down
> only to cache group per thread.
> It is not enough and not utilizes resources effectively in case of one cache
> group much bigger than the others.
> We need to parallel restore process down to individual partitions to get the
> most from the available resources and speed up node startup.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (IGNITE-15926) Implement two methods for dropping a table for the table existing and not
[
https://issues.apache.org/jira/browse/IGNITE-15926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vladislav Pyatkov updated IGNITE-15926:
---
Description:
ANSI standard requires two methods for dropping table for the two DDL:
_DROP TABLE [ IF EXISTS ] syntax_
for the purpose, Table manager should provide two methods with similar semantic.
{code}
TableManager#dropTable(String) throws TableNotExists
TableManager#dropTableIfExists(String)
{code}
was:
ANSI standard requires two methods for dropping table for the two DDL:
_DROP TABLE [ IF EXISTS ] syntax_
for the purpose, Table manager should provide two methods with similar semantic.
> Implement two methods for dropping a table for the table existing and not
> -
>
> Key: IGNITE-15926
> URL: https://issues.apache.org/jira/browse/IGNITE-15926
> Project: Ignite
> Issue Type: Bug
>Reporter: Vladislav Pyatkov
>Priority: Major
> Labels: ignite-3
>
> ANSI standard requires two methods for dropping table for the two DDL:
> _DROP TABLE [ IF EXISTS ] syntax_
> for the purpose, Table manager should provide two methods with similar
> semantic.
> {code}
> TableManager#dropTable(String) throws TableNotExists
> TableManager#dropTableIfExists(String)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (IGNITE-15926) Implement two methods for dropping a table for the table existing and not
[ https://issues.apache.org/jira/browse/IGNITE-15926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladislav Pyatkov updated IGNITE-15926: --- Description: ANSI standard requires two methods for dropping table for the two DDL: _DROP TABLE [ IF EXISTS ] syntax_ for the purpose, Table manager should provide two methods with similar semantic. was: ANSI standard required a two method for dropping table for the two DDL: _DROP TABLE [ IF EXISTS ] syntax_ for the purpose, Table manager should provide two methods with similar semantic. > Implement two methods for dropping a table for the table existing and not > - > > Key: IGNITE-15926 > URL: https://issues.apache.org/jira/browse/IGNITE-15926 > Project: Ignite > Issue Type: Bug >Reporter: Vladislav Pyatkov >Priority: Major > Labels: ignite-3 > > ANSI standard requires two methods for dropping table for the two DDL: > _DROP TABLE [ IF EXISTS ] syntax_ > for the purpose, Table manager should provide two methods with similar > semantic. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (IGNITE-15926) Implement two methods for dropping a table for the table existing and not
[ https://issues.apache.org/jira/browse/IGNITE-15926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladislav Pyatkov updated IGNITE-15926: --- Description: ANSI standard required a two method for dropping table for the two DDL: _DROP TABLE [ IF EXISTS ] syntax_ for the purpose, Table manager should provide two methods with similar semantic. was: ANC standard required a two method for dropping table for the two DDL: _DROP TABLE [ IF EXISTS ] syntax_ for the purpose, Table manager should provide two methods with similar semantic. > Implement two methods for dropping a table for the table existing and not > - > > Key: IGNITE-15926 > URL: https://issues.apache.org/jira/browse/IGNITE-15926 > Project: Ignite > Issue Type: Bug >Reporter: Vladislav Pyatkov >Priority: Major > Labels: ignite-3 > > ANSI standard required a two method for dropping table for the two DDL: > _DROP TABLE [ IF EXISTS ] syntax_ > for the purpose, Table manager should provide two methods with similar > semantic. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (IGNITE-15926) Implement two methods for dropping a table for the table existing and not
[ https://issues.apache.org/jira/browse/IGNITE-15926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladislav Pyatkov updated IGNITE-15926: --- Summary: Implement two methods for dropping a table for the table existing and not (was: Implement two method for dropping a table for the table existing and not) > Implement two methods for dropping a table for the table existing and not > - > > Key: IGNITE-15926 > URL: https://issues.apache.org/jira/browse/IGNITE-15926 > Project: Ignite > Issue Type: Bug >Reporter: Vladislav Pyatkov >Priority: Major > Labels: ignite-3 > > ANC standard required a two method for dropping table for the two DDL: > _DROP TABLE [ IF EXISTS ] syntax_ > for the purpose, Table manager should provide two methods with similar > semantic. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (IGNITE-15926) Implement two method for dropping a table for the table existing and not
Vladislav Pyatkov created IGNITE-15926: -- Summary: Implement two method for dropping a table for the table existing and not Key: IGNITE-15926 URL: https://issues.apache.org/jira/browse/IGNITE-15926 Project: Ignite Issue Type: Bug Reporter: Vladislav Pyatkov ANC standard required a two method for dropping table for the two DDL: _DROP TABLE [ IF EXISTS ] syntax_ for the purpose, Table manager should provide two methods with similar semantic. -- This message was sent by Atlassian Jira (v8.20.1#820001)
