[jira] [Updated] (IGNITE-21098) Increase additional wait after DDL by MaxClockSkew

2023-12-15 Thread Roman Puchkovskiy (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-21098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Roman Puchkovskiy updated IGNITE-21098:
---
Description: When executing an RO transaction, readTimestamp has to be 
determined. If MaxObservableTimestamp is not available...

> Increase additional wait after DDL by MaxClockSkew
> --
>
> Key: IGNITE-21098
> URL: https://issues.apache.org/jira/browse/IGNITE-21098
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Roman Puchkovskiy
>Assignee: Roman Puchkovskiy
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>
> When executing an RO transaction, readTimestamp has to be determined. If 
> MaxObservableTimestamp is not available...



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-21098) Increase additional wait after DDL by MaxClockSkew

2023-12-15 Thread Roman Puchkovskiy (Jira)
Roman Puchkovskiy created IGNITE-21098:
--

 Summary: Increase additional wait after DDL by MaxClockSkew
 Key: IGNITE-21098
 URL: https://issues.apache.org/jira/browse/IGNITE-21098
 Project: Ignite
  Issue Type: Improvement
Reporter: Roman Puchkovskiy
Assignee: Roman Puchkovskiy
 Fix For: 3.0.0-beta2






--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Assigned] (IGNITE-21065) Enhance granularity of authentication events

2023-12-15 Thread Ivan Gagarkin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-21065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ivan Gagarkin reassigned IGNITE-21065:
--

Assignee: Ivan Gagarkin

> Enhance granularity of authentication events
> 
>
> Key: IGNITE-21065
> URL: https://issues.apache.org/jira/browse/IGNITE-21065
> Project: Ignite
>  Issue Type: Bug
>  Components: security, thin client
>Reporter: Ivan Gagarkin
>Assignee: Ivan Gagarkin
>Priority: Major
>  Labels: ignite-3
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Since the basic authenticator stores a list of users, we need to extend 
> authentication events to improve granularity. This is to ensure that the 
> connection is not closed for all users if just one of them changes their 
> password. Update the tests in 
> {{org.apache.ignite.client.handler.ClientInboundMessageHandlerTest}} 
> accordingly.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IGNITE-21096) Fix standalone examples compilation

2023-12-15 Thread Ignite TC Bot (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-21096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17797301#comment-17797301
 ] 

Ignite TC Bot commented on IGNITE-21096:


{panel:title=Branch: [pull/11103/head] Base: [master] : No blockers 
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/11103/head] Base: [master] : No new tests 
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}{panel}
[TeamCity *-- Run :: All* 
Results|https://ci2.ignite.apache.org/viewLog.html?buildId=7658719buildTypeId=IgniteTests24Java8_RunAll]

> Fix standalone examples compilation
> ---
>
> Key: IGNITE-21096
> URL: https://issues.apache.org/jira/browse/IGNITE-21096
> Project: Ignite
>  Issue Type: Bug
>Reporter: Nikita Amelchev
>Assignee: Nikita Amelchev
>Priority: Major
>  Labels: ise
> Fix For: 2.16
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> The standalone examples compilation fails:
> {noformat}
> [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) 
> on project ignite-examples: Compilation failure
> [ERROR] 
> /Users/user/Downloads/release-2.16.0-rc0/svn/vote/apache-ignite-2.16.0-bin/examples/src/main/java/org/apache/ignite/examples/opencensus/OpenCensusMetricsExporterExample.java:[29,29]
>  package org.apache.commons.io does not exist
> {noformat}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-21097) Sql. Infer dynamic parameter types in scripts.

2023-12-15 Thread Maksim Zhuravkov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-21097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Maksim Zhuravkov updated IGNITE-21097:
--
Summary: Sql. Infer dynamic parameter types in scripts.  (was: Sql. Infer 
dynamic parameter types for scripts.)

> Sql. Infer dynamic parameter types in scripts.
> --
>
> Key: IGNITE-21097
> URL: https://issues.apache.org/jira/browse/IGNITE-21097
> Project: Ignite
>  Issue Type: Improvement
>  Components: sql
>Reporter: Maksim Zhuravkov
>Priority: Minor
>  Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>
> After https://issues.apache.org/jira/browse/IGNITE-20778 was resolved it is 
> now possible to infer types of dynamic parameters for a single SQL statement. 
> Since SQL engine also provides API to execute SQL scripts, it should be 
> possible to extend dynamic parameter type inference to scripts as well.
> There some limitations on supported statement types in SQL scripts to support 
> type inference of dynamic parameters - type inference won't work if a script 
> includes a DDL statement (because DDL statements can make add/remove/change 
> type columns, thus making type inference without DDL interpretation 
> impossible).
> So if a script contains any DDL statement a process of inference of dynamic 
> parameter type should report an error, indicating that type inference cannot 
> be perform in presence of DDL statements.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-21097) Sql. Infer dynamic parameter types for scripts.

2023-12-15 Thread Maksim Zhuravkov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-21097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Maksim Zhuravkov updated IGNITE-21097:
--
Summary: Sql. Infer dynamic parameter types for scripts.  (was: Sql. Infer 
parameter types for scripts.)

> Sql. Infer dynamic parameter types for scripts.
> ---
>
> Key: IGNITE-21097
> URL: https://issues.apache.org/jira/browse/IGNITE-21097
> Project: Ignite
>  Issue Type: Improvement
>  Components: sql
>Reporter: Maksim Zhuravkov
>Priority: Minor
>  Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>
> After https://issues.apache.org/jira/browse/IGNITE-20778 was resolved it is 
> now possible to infer types of dynamic parameters for a single SQL statement. 
> Since SQL engine also provides API to execute SQL scripts, it should be 
> possible to extend dynamic parameter type inference to scripts as well.
> There some limitations on supported statement types in SQL scripts to support 
> type inference of dynamic parameters - type inference won't work if a script 
> includes a DDL statement (because DDL statements can make add/remove/change 
> type columns, thus making type inference without DDL interpretation 
> impossible).
> So if a script contains any DDL statement a process of inference of dynamic 
> parameter type should report an error, indicating that type inference cannot 
> be perform in presence of DDL statements.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-21097) Sql. Infer parameter types for scripts.

2023-12-15 Thread Maksim Zhuravkov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-21097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Maksim Zhuravkov updated IGNITE-21097:
--
Description: 
After https://issues.apache.org/jira/browse/IGNITE-20778 was resolved it is now 
possible to infer types of dynamic parameters for a single SQL statement. Since 
SQL engine also provides API to execute SQL scripts, it should be possible to 
extend dynamic parameter type inference to scripts as well.

There some limitations on supported statement types in SQL scripts to support 
type inference of dynamic parameters - type inference won't work if a script 
includes a DDL statement (because DDL statements can make add/remove/change 
type columns, thus making type inference without DDL interpretation impossible).

So if a script contains any DDL statement a process of inference of dynamic 
parameter type should report an error, indicating that type inference cannot be 
perform in presence of DDL statements.

  was:
After https://issues.apache.org/jira/browse/IGNITE-20778 was resolved it is now 
possible to infer types of dynamic parameters for a single SQL statement. 
Since SQL engine also provides API to execute SQL scripts, it should be 
possible to extend dynamic parameter type inference to scripts as well.

There some limitations on supported statement types in SQL scripts to support 
type inference of dynamic parameters - type inference won't work if a script
includes DDL statment (because DDL statement can make add/remove/change type 
columns, thus making type inference without DDL interpretation impossible).

So if a script contains any DDL statement a process of inference of dynamic 
parameter type should report an error, indicating that type inference can not be
perform in presence of DDL statements.


> Sql. Infer parameter types for scripts.
> ---
>
> Key: IGNITE-21097
> URL: https://issues.apache.org/jira/browse/IGNITE-21097
> Project: Ignite
>  Issue Type: Improvement
>  Components: sql
>Reporter: Maksim Zhuravkov
>Priority: Minor
>  Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>
> After https://issues.apache.org/jira/browse/IGNITE-20778 was resolved it is 
> now possible to infer types of dynamic parameters for a single SQL statement. 
> Since SQL engine also provides API to execute SQL scripts, it should be 
> possible to extend dynamic parameter type inference to scripts as well.
> There some limitations on supported statement types in SQL scripts to support 
> type inference of dynamic parameters - type inference won't work if a script 
> includes a DDL statement (because DDL statements can make add/remove/change 
> type columns, thus making type inference without DDL interpretation 
> impossible).
> So if a script contains any DDL statement a process of inference of dynamic 
> parameter type should report an error, indicating that type inference cannot 
> be perform in presence of DDL statements.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-21097) Sql. Infer parameter types for scripts.

2023-12-15 Thread Maksim Zhuravkov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-21097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Maksim Zhuravkov updated IGNITE-21097:
--
Labels: ignite-3  (was: )

> Sql. Infer parameter types for scripts.
> ---
>
> Key: IGNITE-21097
> URL: https://issues.apache.org/jira/browse/IGNITE-21097
> Project: Ignite
>  Issue Type: Improvement
>  Components: sql
>Reporter: Maksim Zhuravkov
>Priority: Minor
>  Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>
> After https://issues.apache.org/jira/browse/IGNITE-20778 was resolved it is 
> now possible to infer types of dynamic parameters for a single SQL statement. 
> Since SQL engine also provides API to execute SQL scripts, it should be 
> possible to extend dynamic parameter type inference to scripts as well.
> There some limitations on supported statement types in SQL scripts to support 
> type inference of dynamic parameters - type inference won't work if a script
> includes DDL statment (because DDL statement can make add/remove/change type 
> columns, thus making type inference without DDL interpretation impossible).
> So if a script contains any DDL statement a process of inference of dynamic 
> parameter type should report an error, indicating that type inference can not 
> be
> perform in presence of DDL statements.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-21097) Sql. Infer parameter types for scripts.

2023-12-15 Thread Maksim Zhuravkov (Jira)
Maksim Zhuravkov created IGNITE-21097:
-

 Summary: Sql. Infer parameter types for scripts.
 Key: IGNITE-21097
 URL: https://issues.apache.org/jira/browse/IGNITE-21097
 Project: Ignite
  Issue Type: Improvement
  Components: sql
Reporter: Maksim Zhuravkov
 Fix For: 3.0.0-beta2


After https://issues.apache.org/jira/browse/IGNITE-20778 was resolved it is now 
possible to infer types of dynamic parameters for a single SQL statement. 
Since SQL engine also provides API to execute SQL scripts, it should be 
possible to extend dynamic parameter type inference to scripts as well.

There some limitations on supported statement types in SQL scripts to support 
type inference of dynamic parameters - type inference won't work if a script
includes DDL statment (because DDL statement can make add/remove/change type 
columns, thus making type inference without DDL interpretation impossible).

So if a script contains any DDL statement a process of inference of dynamic 
parameter type should report an error, indicating that type inference can not be
perform in presence of DDL statements.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-21096) Fix standalone examples compilation

2023-12-15 Thread Aleksandr Nikolaev (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-21096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksandr Nikolaev updated IGNITE-21096:

Labels: ise  (was: )

> Fix standalone examples compilation
> ---
>
> Key: IGNITE-21096
> URL: https://issues.apache.org/jira/browse/IGNITE-21096
> Project: Ignite
>  Issue Type: Bug
>Reporter: Nikita Amelchev
>Assignee: Nikita Amelchev
>Priority: Major
>  Labels: ise
> Fix For: 2.16
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> The standalone examples compilation fails:
> {noformat}
> [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) 
> on project ignite-examples: Compilation failure
> [ERROR] 
> /Users/user/Downloads/release-2.16.0-rc0/svn/vote/apache-ignite-2.16.0-bin/examples/src/main/java/org/apache/ignite/examples/opencensus/OpenCensusMetricsExporterExample.java:[29,29]
>  package org.apache.commons.io does not exist
> {noformat}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IGNITE-21084) Account for differences of logical part of now() on different nodes when waiting after a DDL

2023-12-15 Thread Roman Puchkovskiy (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-21084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17797268#comment-17797268
 ] 

Roman Puchkovskiy commented on IGNITE-21084:


Thanks!

> Account for differences of logical part of now() on different nodes when 
> waiting after a DDL
> 
>
> Key: IGNITE-21084
> URL: https://issues.apache.org/jira/browse/IGNITE-21084
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Roman Puchkovskiy
>Assignee: Roman Puchkovskiy
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Client must see the results of a DDL they just executed, no matter through 
> which node of the cluster a subsequent request is made. To maintain this, we 
> must make sure that ActivationTimestamp(DDL) becomes non-future on all nodes 
> of the cluster and only then can we complete the DDL future. As nodes' 
> physical clocks might have some skew relatively to each other, we add 
> MaxClockSkew to the timestamp till which we wait to compensate for this 
> difference.
> Analogously to HLC.now() being different because its physical part differs on 
> different nodes, it might be different because *logical* parts are different 
> on different nodes.
> Let's assume we don't have any physical clock skew, and MaxClockSkew is 0. 
> ActivationTimestamp(DDL) is (100, 5); (100 is physical part, 5 is logical 
> part). We wait on node 1 (through which the DDL was executed) till its 
> HLC.now() reaches (100, 5), then we complete the DDL future. The user goes to 
> node 2 at which HLC.now() is (100, 2). The user executes a query at 'now', 
> and the query sees the state before the DDL was executed, which breaks our 
> requirement.
> We can fix this by 'rounding' the timestamp-to-wait-for up in the following 
> way:
>  # If logical part is not 0, increment the physical part and set the logical 
> part to 0
>  # If the logical part is 0, leave the timestamp as is
> As a result, for (100, 0) we will wait for (100, 0), and at node 1 HLC is at 
> least (100, 0), so it cannot see the previous schema. For (100, 5) we would 
> wait till (101, 0), which would also guarantee that a query executed after 
> waiting sees the new schema version.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IGNITE-21083) Java thin 3.0: Sporadic IllegalReferenceCountException when reading jdbc messages

2023-12-15 Thread Pavel Tupitsyn (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-21083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17797235#comment-17797235
 ] 

Pavel Tupitsyn commented on IGNITE-21083:
-

Merged to main: 2efc454f21460ec31a7828d8927ed113d0c80655

> Java thin 3.0: Sporadic IllegalReferenceCountException when reading jdbc 
> messages
> -
>
> Key: IGNITE-21083
> URL: https://issues.apache.org/jira/browse/IGNITE-21083
> Project: Ignite
>  Issue Type: Bug
>  Components: thin client
>Reporter: Maksim Zhuravkov
>Assignee: Pavel Tupitsyn
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Some jdbc test fail  with IllegalReferenceCountException refCnt 0 
> (`org.apache.ignite.jdbc.ItJdbcResultSetSelfTest.testFindColumn`).
> This issue maybe caused by the following code in `TcpClientChannel` , because 
> a buffer is sent to another thread w/o calling `ByteBuf::retain`:
> {code:java}
> /** {@inheritDoc} */
> @Override
> public void onMessage(ByteBuf buf) {
> asyncContinuationExecutor.execute(() -> {
> try {
> processNextMessage(buf);
> } catch (Throwable t) {
> close(t, false);
> } finally {
> buf.release();
> }
> });
> }
> {code}
> Error:
> {code:java}
> java.sql.SQLException: Unable to close result set.
>   at 
> org.apache.ignite.internal.jdbc.JdbcResultSet.close0(JdbcResultSet.java:296)
>   at 
> org.apache.ignite.internal.jdbc.JdbcStatement.closeResults(JdbcStatement.java:717)
>   at 
> org.apache.ignite.internal.jdbc.JdbcStatement.close(JdbcStatement.java:259)
>   at 
> org.apache.ignite.jdbc.AbstractJdbcSelfTest.tearDownBase(AbstractJdbcSelfTest.java:130)
>   at jdk.internal.reflect.GeneratedMethodAccessor40.invoke(Unknown Source)
>   at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.base/java.lang.reflect.Method.invoke(Method.java:566)
>   at 
> org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:727)
>   at 
> org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
>   at 
> org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
>   at 
> org.junit.jupiter.engine.extension.SameThreadTimeoutInvocation.proceed(SameThreadTimeoutInvocation.java:45)
>   at 
> org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156)
>   at 
> org.junit.jupiter.engine.extension.TimeoutExtension.interceptLifecycleMethod(TimeoutExtension.java:128)
>   at 
> org.junit.jupiter.engine.extension.TimeoutExtension.interceptAfterEachMethod(TimeoutExtension.java:110)
>   at 
> org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103)
>   at 
> org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93)
>   at 
> org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
>   at 
> org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
>   at 
> org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
>   at 
> org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
>   at 
> org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:92)
>   at 
> org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:86)
>   at 
> org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.invokeMethodInExtensionContext(ClassBasedTestDescriptor.java:520)
>   at 
> org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.lambda$synthesizeAfterEachMethodAdapter$24(ClassBasedTestDescriptor.java:510)
>   at 
> org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeAfterEachMethods$10(TestMethodTestDescriptor.java:243)
>   at 
> org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeAllAfterMethodsOrCallbacks$13(TestMethodTestDescriptor.java:276)
>   at 
> org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
>   at 
> org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeAllAfterMethodsOrCallbacks$14(TestMethodTestDescriptor.java:276)
>   at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
>   at 
> 

[jira] [Created] (IGNITE-21096) Fix standalone examples compilation

2023-12-15 Thread Nikita Amelchev (Jira)
Nikita Amelchev created IGNITE-21096:


 Summary: Fix standalone examples compilation
 Key: IGNITE-21096
 URL: https://issues.apache.org/jira/browse/IGNITE-21096
 Project: Ignite
  Issue Type: Bug
Reporter: Nikita Amelchev
Assignee: Nikita Amelchev
 Fix For: 2.16


The standalone examples compilation fails:
{noformat}
[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on 
project ignite-examples: Compilation failure
[ERROR] 
/Users/user/Downloads/release-2.16.0-rc0/svn/vote/apache-ignite-2.16.0-bin/examples/src/main/java/org/apache/ignite/examples/opencensus/OpenCensusMetricsExporterExample.java:[29,29]
 package org.apache.commons.io does not exist
{noformat}




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-21083) Java thin 3.0: Sporadic IllegalReferenceCountException when reading jdbc messages

2023-12-15 Thread Pavel Tupitsyn (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-21083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Tupitsyn updated IGNITE-21083:

Ignite Flags:   (was: Docs Required,Release Notes Required)

> Java thin 3.0: Sporadic IllegalReferenceCountException when reading jdbc 
> messages
> -
>
> Key: IGNITE-21083
> URL: https://issues.apache.org/jira/browse/IGNITE-21083
> Project: Ignite
>  Issue Type: Bug
>  Components: thin client
>Reporter: Maksim Zhuravkov
>Assignee: Pavel Tupitsyn
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Some jdbc test fail  with IllegalReferenceCountException refCnt 0 
> (`org.apache.ignite.jdbc.ItJdbcResultSetSelfTest.testFindColumn`).
> This issue maybe caused by the following code in `TcpClientChannel` , because 
> a buffer is sent to another thread w/o calling `ByteBuf::retain`:
> {code:java}
> /** {@inheritDoc} */
> @Override
> public void onMessage(ByteBuf buf) {
> asyncContinuationExecutor.execute(() -> {
> try {
> processNextMessage(buf);
> } catch (Throwable t) {
> close(t, false);
> } finally {
> buf.release();
> }
> });
> }
> {code}
> Error:
> {code:java}
> java.sql.SQLException: Unable to close result set.
>   at 
> org.apache.ignite.internal.jdbc.JdbcResultSet.close0(JdbcResultSet.java:296)
>   at 
> org.apache.ignite.internal.jdbc.JdbcStatement.closeResults(JdbcStatement.java:717)
>   at 
> org.apache.ignite.internal.jdbc.JdbcStatement.close(JdbcStatement.java:259)
>   at 
> org.apache.ignite.jdbc.AbstractJdbcSelfTest.tearDownBase(AbstractJdbcSelfTest.java:130)
>   at jdk.internal.reflect.GeneratedMethodAccessor40.invoke(Unknown Source)
>   at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.base/java.lang.reflect.Method.invoke(Method.java:566)
>   at 
> org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:727)
>   at 
> org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
>   at 
> org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
>   at 
> org.junit.jupiter.engine.extension.SameThreadTimeoutInvocation.proceed(SameThreadTimeoutInvocation.java:45)
>   at 
> org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156)
>   at 
> org.junit.jupiter.engine.extension.TimeoutExtension.interceptLifecycleMethod(TimeoutExtension.java:128)
>   at 
> org.junit.jupiter.engine.extension.TimeoutExtension.interceptAfterEachMethod(TimeoutExtension.java:110)
>   at 
> org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103)
>   at 
> org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93)
>   at 
> org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
>   at 
> org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
>   at 
> org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
>   at 
> org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
>   at 
> org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:92)
>   at 
> org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:86)
>   at 
> org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.invokeMethodInExtensionContext(ClassBasedTestDescriptor.java:520)
>   at 
> org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.lambda$synthesizeAfterEachMethodAdapter$24(ClassBasedTestDescriptor.java:510)
>   at 
> org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeAfterEachMethods$10(TestMethodTestDescriptor.java:243)
>   at 
> org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeAllAfterMethodsOrCallbacks$13(TestMethodTestDescriptor.java:276)
>   at 
> org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
>   at 
> org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeAllAfterMethodsOrCallbacks$14(TestMethodTestDescriptor.java:276)
>   at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
>   at 
> 

[jira] [Updated] (IGNITE-21081) Sql. Support empty statements.

2023-12-15 Thread Pavel Pereslegin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-21081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Pereslegin updated IGNITE-21081:
--
Description: 
Statements starting by ';', for example

{code:java};SELECT 1{code}
{code:java} ; SELECT 1{code}
{code:java}\n ; \n SELECT 1{code}

produces parser exception: {{Failed to parse query: Encountered ";"}}.


  was:
Statements starting by ';', for example

{code:java};SELECT 1{code}
{code:java} ; SELECT 1{code}
{code:java}\n ; \n SELECT 1{code}

produces parser exception {{Failed to parse query: Encountered ";"}}.



> Sql. Support empty statements.
> --
>
> Key: IGNITE-21081
> URL: https://issues.apache.org/jira/browse/IGNITE-21081
> Project: Ignite
>  Issue Type: Improvement
>  Components: sql
>Affects Versions: 3.0.0-beta1
>Reporter: Evgeny Stanilovsky
>Priority: Major
>  Labels: ignite-3
>
> Statements starting by ';', for example
> {code:java};SELECT 1{code}
> {code:java} ; SELECT 1{code}
> {code:java}\n ; \n SELECT 1{code}
> produces parser exception: {{Failed to parse query: Encountered ";"}}.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-21081) Sql. Support empty statements.

2023-12-15 Thread Pavel Pereslegin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-21081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Pereslegin updated IGNITE-21081:
--
Description: 
Statements starting with ';', for example

{code:java};SELECT 1{code}
{code:java} ; SELECT 1{code}
{code:java}\n ; \n SELECT 1{code}

produces parser exception: {{Failed to parse query: Encountered ";"}}.


  was:
Statements starting by ';', for example

{code:java};SELECT 1{code}
{code:java} ; SELECT 1{code}
{code:java}\n ; \n SELECT 1{code}

produces parser exception: {{Failed to parse query: Encountered ";"}}.



> Sql. Support empty statements.
> --
>
> Key: IGNITE-21081
> URL: https://issues.apache.org/jira/browse/IGNITE-21081
> Project: Ignite
>  Issue Type: Improvement
>  Components: sql
>Affects Versions: 3.0.0-beta1
>Reporter: Evgeny Stanilovsky
>Priority: Major
>  Labels: ignite-3
>
> Statements starting with ';', for example
> {code:java};SELECT 1{code}
> {code:java} ; SELECT 1{code}
> {code:java}\n ; \n SELECT 1{code}
> produces parser exception: {{Failed to parse query: Encountered ";"}}.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-21081) Sql. Support empty statements.

2023-12-15 Thread Pavel Pereslegin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-21081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Pereslegin updated IGNITE-21081:
--
Description: 
Statements starting by ';', for example

{code:java};SELECT 1{code}
{code:java} ; SELECT 1{code}
{code:java}\n ; \n SELECT 1{code}

produces parser exception {{Failed to parse query: Encountered ";"}}.


  was:
Statements like : 

{code:java};;;SELECT 1 + 2{code}
{code:java}-- comment; SELECT * FROM T1{code}

are not supported for now from jdbc\multistatement api, seems need to be fixed.



> Sql. Support empty statements.
> --
>
> Key: IGNITE-21081
> URL: https://issues.apache.org/jira/browse/IGNITE-21081
> Project: Ignite
>  Issue Type: Improvement
>  Components: sql
>Affects Versions: 3.0.0-beta1
>Reporter: Evgeny Stanilovsky
>Priority: Major
>  Labels: ignite-3
>
> Statements starting by ';', for example
> {code:java};SELECT 1{code}
> {code:java} ; SELECT 1{code}
> {code:java}\n ; \n SELECT 1{code}
> produces parser exception {{Failed to parse query: Encountered ";"}}.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IGNITE-20894) Fix deadlock recovery on commit

2023-12-15 Thread Vladislav Pyatkov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-20894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17797185#comment-17797185
 ] 

Vladislav Pyatkov commented on IGNITE-20894:


There are tests that demonstrate exactly one transaction commit preset for SQL:
_ItSqlApiBaseTest#checkTransactionsWithDml_
_ItSqlApiBaseTest#testLockIsNotReleasedAfterTxRollback_
If a transaction coordinator catches a lock exception, the transaction will be 
automatically rolled back.

> Fix deadlock recovery on commit
> ---
>
> Key: IGNITE-20894
> URL: https://issues.apache.org/jira/browse/IGNITE-20894
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 3.0
>Reporter: Alexey Scherbakov
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0
>
>
> Reproduced by [1]
> In short, two transaction are put into deadlock, using no prevention policy.
> First tx is committed, but can't acquire locks.
> Expected behavior - tx is rolled back on commit, releasing associated locks, 
> tx2 succefully commits.
> Actual behavior - commit hangs.
> [1] org.apache.ignite.distributed.ItLockTableTest#testDeadlockRecovery



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-21095) Preserve NetworkMessages immutability

2023-12-15 Thread Alexander Lapin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-21095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Lapin updated IGNITE-21095:
-
Description: 
h3. Motivation

Currently it's possible to modify NetworkMessages using setters that'll be 
generated after @WithSetter. Given approach is fragile, e.g. check Safe time 
reordering in partitions and should be reworked with some sort of factory 
methods that will not modify initial object but will update newly created 
instance.
h3. Definition of Done
 * @WithSetter is removed.
 * NetworkMessage modification returns newly created (and modified) object 
instead of in-place modification over initial one.

> Preserve NetworkMessages immutability
> -
>
> Key: IGNITE-21095
> URL: https://issues.apache.org/jira/browse/IGNITE-21095
> Project: Ignite
>  Issue Type: New Feature
>Reporter: Alexander Lapin
>Priority: Major
>  Labels: ignite-3
>
> h3. Motivation
> Currently it's possible to modify NetworkMessages using setters that'll be 
> generated after @WithSetter. Given approach is fragile, e.g. check Safe time 
> reordering in partitions and should be reworked with some sort of factory 
> methods that will not modify initial object but will update newly created 
> instance.
> h3. Definition of Done
>  * @WithSetter is removed.
>  * NetworkMessage modification returns newly created (and modified) object 
> instead of in-place modification over initial one.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-21095) Preserve NetworkMessages immutability

2023-12-15 Thread Alexander Lapin (Jira)
Alexander Lapin created IGNITE-21095:


 Summary: Preserve NetworkMessages immutability
 Key: IGNITE-21095
 URL: https://issues.apache.org/jira/browse/IGNITE-21095
 Project: Ignite
  Issue Type: New Feature
Reporter: Alexander Lapin






--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-21062) Safe time reordering in partitions

2023-12-15 Thread Alexander Lapin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-21062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Lapin updated IGNITE-21062:
-
Reviewer: Ivan Bessonov

> Safe time reordering in partitions
> --
>
> Key: IGNITE-21062
> URL: https://issues.apache.org/jira/browse/IGNITE-21062
> Project: Ignite
>  Issue Type: Bug
>Reporter: Ivan Bessonov
>Assignee: Alexander Lapin
>Priority: Major
>  Labels: ignite-3
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> In the scenario of creating a lot of table and having slow system 
> (presumably), it's possible to notice {{Safe time reordering detected 
> [current=...}} assertion error in logs.
> It happens with safe-time sync commands, in the absence of transactional load.
> h3. UPD #1
> Following steps will bring us to the "Safe time reordering detected" problem.
> 0. PartitionReplicaListener and PartitionListener are located on the same 
> Ignite node. No serialization/deserialization within raft itself. 
> maxObservableSafeTime = -1, maxObservableSafeTimeVerifier = -1.
> 1. SafeTimePropagatingCommand  with safeTime = 10 successfully passes 
> maxObservableSafeTime gateway, messaging thread is free, raft thread handles 
> the command and hangs before stateMachine.onWrite() -> maxObservableSafeTime 
> = 10, maxObservableSafeTimeVerifier = -1
> 2. Client for some reason (e.g. TimeoutException) re-send same command that 
> will be rolled back because both command.safeTime and maxObservableSafeTime 
> == 10.
> 3. Client updates the command with new safeTime (e.g. 20) which also will 
> modify "initial command" from the step 1.
> 4. Raft thread from the step 1 apply unintentionally updated command to a 
> state machine -> maxObservableSafeTime = 10, maxObservableSafeTimeVerifier = 
> 20 (not expected, should be the same as in maxObservableSafeTime).
> 5. Retry command from step 3 successfully passes the gateway and face an 
> assertion on maxObservableSafeTimeVerifier == command.safeTime
> Rather often there was exact safeTime matching in logs, that proves given 
> explanation.
> {code:java}
> [2023-11-29T14:27:05,893][ERROR][%irdt_tdpsoen_20002%JRaft-FSMCaller-Disruptor-_stripe_7-0][StripedDisruptor]
>  Handle disruptor event error 
> [name=%irdt_tdpsoen_20002%JRaft-FSMCaller-Disruptor-, 
> event=org.apache.ignite.raft.jraft.core.FSMCallerImpl$ApplyTask@2e6cd30d, 
> hasHandler=false]
> java.lang.AssertionError: Safe time reordering detected 
> [current=111494301331423233, proposed=111494301331423233]
> All in all, that means, that messages should be immutable.{code}
>  * Initially, to verify that aforementioned explanation is correct, it's 
> possible to clone the message and update the cloned copy.
>  * However, it's better to exclude the ability to call set<> on message 
> itself. Meaning that some sort of factory method is much more reliable here. 
> That will be covered in a separate ticket.
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-21062) Safe time reordering in partitions

2023-12-15 Thread Alexander Lapin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-21062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Lapin updated IGNITE-21062:
-
Description: 
In the scenario of creating a lot of table and having slow system (presumably), 
it's possible to notice {{Safe time reordering detected [current=...}} 
assertion error in logs.

It happens with safe-time sync commands, in the absence of transactional load.
h3. UPD #1

Following steps will bring us to the "Safe time reordering detected" problem.

0. PartitionReplicaListener and PartitionListener are located on the same 
Ignite node. No serialization/deserialization within raft itself. 
maxObservableSafeTime = -1, maxObservableSafeTimeVerifier = -1.

1. SafeTimePropagatingCommand  with safeTime = 10 successfully passes 
maxObservableSafeTime gateway, messaging thread is free, raft thread handles 
the command and hangs before stateMachine.onWrite() -> maxObservableSafeTime = 
10, maxObservableSafeTimeVerifier = -1

2. Client for some reason (e.g. TimeoutException) re-send same command that 
will be rolled back because both command.safeTime and maxObservableSafeTime == 
10.

3. Client updates the command with new safeTime (e.g. 20) which also will 
modify "initial command" from the step 1.

4. Raft thread from the step 1 apply unintentionally updated command to a state 
machine -> maxObservableSafeTime = 10, maxObservableSafeTimeVerifier = 20 (not 
expected, should be the same as in maxObservableSafeTime).

5. Retry command from step 3 successfully passes the gateway and face an 
assertion on maxObservableSafeTimeVerifier == command.safeTime

Rather often there was exact safeTime matching in logs, that proves given 
explanation.
{code:java}
[2023-11-29T14:27:05,893][ERROR][%irdt_tdpsoen_20002%JRaft-FSMCaller-Disruptor-_stripe_7-0][StripedDisruptor]
 Handle disruptor event error 
[name=%irdt_tdpsoen_20002%JRaft-FSMCaller-Disruptor-, 
event=org.apache.ignite.raft.jraft.core.FSMCallerImpl$ApplyTask@2e6cd30d, 
hasHandler=false]
java.lang.AssertionError: Safe time reordering detected 
[current=111494301331423233, proposed=111494301331423233]
All in all, that means, that messages should be immutable.{code}
 * Initially, to verify that aforementioned explanation is correct, it's 
possible to clone the message and update the cloned copy.
 * However, it's better to exclude the ability to call set<> on message itself. 
Meaning that some sort of factory method is much more reliable here. That will 
be covered in a separate ticket.

 

  was:
In the scenario of creating a lot of table and having slow system (presumably), 
it's possible to notice {{Safe time reordering detected [current=...}} 
assertion error in logs.

It happens with safe-time sync commands, in the absence of transactional load.
h3. UPD #1

Following steps will bring us to the "Safe time reordering detected" problem.

0. PartitionReplicaListener and PartitionListener are located on the same 
Ignite node. No serialization/deserialization within raft itself. 
maxObservableSafeTime = -1, maxObservableSafeTimeVerifier = -1.

1. SafeTimePropagatingCommand  with safeTime = 10 successfully passes 
maxObservableSafeTime gateway, messaging thread is free, raft thread handles 
the command and hangs before stateMachine.onWrite() -> maxObservableSafeTime = 
10, maxObservableSafeTimeVerifier = -1

2. Client for some reason (e.g. TimeoutException) re-send same command that 
will be rolled back because both command.safeTime and maxObservableSafeTime == 
10.

3. Client updates the command with new safeTime (e.g. 20) which also will 
modify "initial command" from the step 1.

4. Raft thread from the step 1 apply unintentionally updated command to a state 
machine -> maxObservableSafeTime = 10, maxObservableSafeTimeVerifier = 20 (not 
expected, should be the same as in maxObservableSafeTime).

5. Retry command from step 3 successfully passes the gateway and face an 
assertion on maxObservableSafeTimeVerifier == command.safeTime

Rather often there was exact safeTime matching in logs, that proves given 
explanation.
[2023-11-29T14:27:05,893][ERROR][%irdt_tdpsoen_20002%JRaft-FSMCaller-Disruptor-_stripe_7-0][StripedDisruptor]
 Handle disruptor event error 
[name=%irdt_tdpsoen_20002%JRaft-FSMCaller-Disruptor-, 
event=org.apache.ignite.raft.jraft.core.FSMCallerImpl$ApplyTask@2e6cd30d, 
hasHandler=false]
  java.lang.AssertionError: Safe time reordering detected 
[current=111494301331423233, proposed=111494301331423233]
All in all, that means, that messages should be immutable.
 * Initially, to verify that aforementioned explanation is correct, it's 
possible to clone the message and update the cloned copy.
 * However, it's better to exclude the ability to call set<> on message itself. 
Meaning that some sort of factory method is much more reliable here. That will 
be covered in a separate ticket.

 


> Safe time reordering in partitions
> 

[jira] [Updated] (IGNITE-21062) Safe time reordering in partitions

2023-12-15 Thread Alexander Lapin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-21062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Lapin updated IGNITE-21062:
-
Description: 
In the scenario of creating a lot of table and having slow system (presumably), 
it's possible to notice {{Safe time reordering detected [current=...}} 
assertion error in logs.

It happens with safe-time sync commands, in the absence of transactional load.
h3. UPD #1

Following steps will bring us to the "Safe time reordering detected" problem.

0. PartitionReplicaListener and PartitionListener are located on the same 
Ignite node. No serialization/deserialization within raft itself. 
maxObservableSafeTime = -1, maxObservableSafeTimeVerifier = -1.

1. SafeTimePropagatingCommand  with safeTime = 10 successfully passes 
maxObservableSafeTime gateway, messaging thread is free, raft thread handles 
the command and hangs before stateMachine.onWrite() -> maxObservableSafeTime = 
10, maxObservableSafeTimeVerifier = -1

2. Client for some reason (e.g. TimeoutException) re-send same command that 
will be rolled back because both command.safeTime and maxObservableSafeTime == 
10.

3. Client updates the command with new safeTime (e.g. 20) which also will 
modify "initial command" from the step 1.

4. Raft thread from the step 1 apply unintentionally updated command to a state 
machine -> maxObservableSafeTime = 10, maxObservableSafeTimeVerifier = 20 (not 
expected, should be the same as in maxObservableSafeTime).

5. Retry command from step 3 successfully passes the gateway and face an 
assertion on maxObservableSafeTimeVerifier == command.safeTime

Rather often there was exact safeTime matching in logs, that proves given 
explanation.
[2023-11-29T14:27:05,893][ERROR][%irdt_tdpsoen_20002%JRaft-FSMCaller-Disruptor-_stripe_7-0][StripedDisruptor]
 Handle disruptor event error 
[name=%irdt_tdpsoen_20002%JRaft-FSMCaller-Disruptor-, 
event=org.apache.ignite.raft.jraft.core.FSMCallerImpl$ApplyTask@2e6cd30d, 
hasHandler=false]
  java.lang.AssertionError: Safe time reordering detected 
[current=111494301331423233, proposed=111494301331423233]
All in all, that means, that messages should be immutable.
 * Initially, to verify that aforementioned explanation is correct, it's 
possible to clone the message and update the cloned copy.
 * However, it's better to exclude the ability to call set<> on message itself. 
Meaning that some sort of factory method is much more reliable here. That will 
be covered in a separate ticket.

 

  was:
In the scenario of creating a lot of table and having slow system (presumably), 
it's possible to notice {{Safe time reordering detected [current=...}} 
assertion error in logs.

It happens with safe-time sync commands, in the absence of transactional load.


> Safe time reordering in partitions
> --
>
> Key: IGNITE-21062
> URL: https://issues.apache.org/jira/browse/IGNITE-21062
> Project: Ignite
>  Issue Type: Bug
>Reporter: Ivan Bessonov
>Assignee: Alexander Lapin
>Priority: Major
>  Labels: ignite-3
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> In the scenario of creating a lot of table and having slow system 
> (presumably), it's possible to notice {{Safe time reordering detected 
> [current=...}} assertion error in logs.
> It happens with safe-time sync commands, in the absence of transactional load.
> h3. UPD #1
> Following steps will bring us to the "Safe time reordering detected" problem.
> 0. PartitionReplicaListener and PartitionListener are located on the same 
> Ignite node. No serialization/deserialization within raft itself. 
> maxObservableSafeTime = -1, maxObservableSafeTimeVerifier = -1.
> 1. SafeTimePropagatingCommand  with safeTime = 10 successfully passes 
> maxObservableSafeTime gateway, messaging thread is free, raft thread handles 
> the command and hangs before stateMachine.onWrite() -> maxObservableSafeTime 
> = 10, maxObservableSafeTimeVerifier = -1
> 2. Client for some reason (e.g. TimeoutException) re-send same command that 
> will be rolled back because both command.safeTime and maxObservableSafeTime 
> == 10.
> 3. Client updates the command with new safeTime (e.g. 20) which also will 
> modify "initial command" from the step 1.
> 4. Raft thread from the step 1 apply unintentionally updated command to a 
> state machine -> maxObservableSafeTime = 10, maxObservableSafeTimeVerifier = 
> 20 (not expected, should be the same as in maxObservableSafeTime).
> 5. Retry command from step 3 successfully passes the gateway and face an 
> assertion on maxObservableSafeTimeVerifier == command.safeTime
> Rather often there was exact safeTime matching in logs, that proves given 
> explanation.
> [2023-11-29T14:27:05,893][ERROR][%irdt_tdpsoen_20002%JRaft-FSMCaller-Disruptor-_stripe_7-0][StripedDisruptor]
>  Handle 

[jira] [Updated] (IGNITE-21094) Sql. Combine query execution methods (single/script) in QueryProcessor.

2023-12-15 Thread Pavel Pereslegin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-21094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Pereslegin updated IGNITE-21094:
--
Component/s: sql

> Sql. Combine query execution methods (single/script) in QueryProcessor.
> ---
>
> Key: IGNITE-21094
> URL: https://issues.apache.org/jira/browse/IGNITE-21094
> Project: Ignite
>  Issue Type: Improvement
>  Components: sql
>Reporter: Pavel Pereslegin
>Priority: Major
>  Labels: ignite-3
>
> At the moment we have two methods for executing queries:
> {{querySingleAsync}} for executing single-statement query
> {{queryScriptAsync}} for executing multi-statement query.
> These methods have an identical signature, so they can be easily combined 
> into one.
> The simple solution will be to keep, for example, `queryAsync` as a single 
> entry point.
> It seems that, internally, we can figure out how to execute this query, 
> depending on the query execution property {{ALLOWED_QUERY_TYPES}}. For 
> example, {{querySingleAsync}} does not allow the TX_CONTROL type.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-21094) Sql. Combine query execution methods (single/script) in QueryProcessor.

2023-12-15 Thread Pavel Pereslegin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-21094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Pereslegin updated IGNITE-21094:
--
Ignite Flags:   (was: Docs Required,Release Notes Required)

> Sql. Combine query execution methods (single/script) in QueryProcessor.
> ---
>
> Key: IGNITE-21094
> URL: https://issues.apache.org/jira/browse/IGNITE-21094
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Pavel Pereslegin
>Priority: Major
>  Labels: ignite-3
>
> At the moment we have two methods for executing queries:
> {{querySingleAsync}} for executing single-statement query
> {{queryScriptAsync}} for executing multi-statement query.
> These methods have an identical signature, so they can be easily combined 
> into one.
> The simple solution will be to keep, for example, `queryAsync` as a single 
> entry point.
> It seems that, internally, we can figure out how to execute this query, 
> depending on the query execution property {{ALLOWED_QUERY_TYPES}}. For 
> example, {{querySingleAsync}} does not allow the TX_CONTROL type.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-21094) Sql. Combine query execution methods (single/script) in QueryProcessor.

2023-12-15 Thread Pavel Pereslegin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-21094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Pereslegin updated IGNITE-21094:
--
Description: 
At the moment we have two methods for executing queries:
{{querySingleAsync}} for executing single-statement query
{{queryScriptAsync}} for executing multi-statement query.

These methods have an identical signature, so they can be easily combined into 
one.

The simple solution will be to keep, for example, `queryAsync` as a single 
entry point.
It seems that, internally, we can figure out how to execute this query, 
depending on the query execution property {{ALLOWED_QUERY_TYPES}}. For example, 
{{querySingleAsync}} does not allow the TX_CONTROL type.


  was:
At the moment we have two methods for executing queries:
{{querySingleAsync}} for executing single-statement query
{{queryScriptAsync}} for executing multi-statement query.

These methods have an identical signature, so they can be easily combined into 
one.

The simple solution will be to keep, for example, `queryAsync` as a single 
entry point.
It seems that, internally, we can figure out how to execute this query, 
depending on the query execution property {{ALLOWED_QUERY_TYPES}}. 
{{querySingleAsync}} does not allow the TX_CONTROL type.



> Sql. Combine query execution methods (single/script) in QueryProcessor.
> ---
>
> Key: IGNITE-21094
> URL: https://issues.apache.org/jira/browse/IGNITE-21094
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Pavel Pereslegin
>Priority: Major
>  Labels: ignite-3
>
> At the moment we have two methods for executing queries:
> {{querySingleAsync}} for executing single-statement query
> {{queryScriptAsync}} for executing multi-statement query.
> These methods have an identical signature, so they can be easily combined 
> into one.
> The simple solution will be to keep, for example, `queryAsync` as a single 
> entry point.
> It seems that, internally, we can figure out how to execute this query, 
> depending on the query execution property {{ALLOWED_QUERY_TYPES}}. For 
> example, {{querySingleAsync}} does not allow the TX_CONTROL type.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-21094) Sql. Combine query execution methods (single/script) in QueryProcessor.

2023-12-15 Thread Pavel Pereslegin (Jira)
Pavel Pereslegin created IGNITE-21094:
-

 Summary: Sql. Combine query execution methods (single/script) in 
QueryProcessor.
 Key: IGNITE-21094
 URL: https://issues.apache.org/jira/browse/IGNITE-21094
 Project: Ignite
  Issue Type: Improvement
Reporter: Pavel Pereslegin


At the moment we have two methods for executing queries:
{{querySingleAsync}} for executing single-statement query
{{queryScriptAsync}} for executing multi-statement query.

These methods have an identical signature, so they can be easily combined into 
one.

The simple solution will be to keep, for example, `queryAsync` as a single 
entry point.
It seems that, internally, we can figure out how to execute this query, 
depending on the query execution property {{ALLOWED_QUERY_TYPES}}. 
{{querySingleAsync}} does not allow the TX_CONTROL type.




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-21093) *.mvcc() removal

2023-12-15 Thread Julia Bakulina (Jira)
Julia Bakulina created IGNITE-21093:
---

 Summary: *.mvcc() removal
 Key: IGNITE-21093
 URL: https://issues.apache.org/jira/browse/IGNITE-21093
 Project: Ignite
  Issue Type: Sub-task
Reporter: Julia Bakulina
Assignee: Julia Bakulina


Remove *.mvcc()



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Resolved] (IGNITE-21073) Remove *.mvcc()

2023-12-15 Thread Julia Bakulina (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-21073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Julia Bakulina resolved IGNITE-21073.
-
Resolution: Duplicate

> Remove *.mvcc()
> ---
>
> Key: IGNITE-21073
> URL: https://issues.apache.org/jira/browse/IGNITE-21073
> Project: Ignite
>  Issue Type: Task
>Reporter: Julia Bakulina
>Assignee: Julia Bakulina
>Priority: Minor
>  Labels: ise
>
> *.mvcc() removal
> The community has agreed that MVCC public API should be removed.
> Vote thread
> [http://apache-ignite-developers.2346864.n4.nabble.com/Removing-MVCC-public-API-tp50550.html]



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-21087) MvccCoordinator removal

2023-12-15 Thread Julia Bakulina (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-21087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Julia Bakulina updated IGNITE-21087:

Summary: MvccCoordinator removal  (was: Remove MvccCoordinator)

> MvccCoordinator removal
> ---
>
> Key: IGNITE-21087
> URL: https://issues.apache.org/jira/browse/IGNITE-21087
> Project: Ignite
>  Issue Type: Sub-task
>Reporter: Julia Bakulina
>Assignee: Julia Bakulina
>Priority: Minor
>  Labels: ise
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Remove MvccCoordinator



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-21091) Send indexes data during full state transfer

2023-12-15 Thread Ivan Bessonov (Jira)
Ivan Bessonov created IGNITE-21091:
--

 Summary: Send indexes data during full state transfer
 Key: IGNITE-21091
 URL: https://issues.apache.org/jira/browse/IGNITE-21091
 Project: Ignite
  Issue Type: Improvement
Reporter: Ivan Bessonov


Current full state transfer implementation dictates that receiver will build 
all secondary indexes on the fly. This might not be efficient:
 * receiver will have to extract index tuples from each row version
 * for rows with multiple versions, many of these tuples will be the same. 
Which means that the "extraction" and "insertion" will be performed several 
times for the same value

Of course, it all depends on how many versions each row has, but generally 
speaking, inserting data one time is always better than inserting it multiple 
times.

It is proposed to send indexes the same way as we send version chains. By using 
scan operation and copy-on-write semantics during data modifications (on the 
sender).

The specifics of the algorithm are not clear yet. This issues includes 
investigation of the proper approach, that will eliminate
 * the possibility of data inconsistencies
 * data leaks
 * excessive memory consumption on sender 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-21090) "IllegalReferenceCountException: refCnt: 0" in SQL query

2023-12-15 Thread Andrey Khitrin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-21090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Khitrin updated IGNITE-21090:

Description: 
We have a simple automated test that creates 1000 tables of the same structure 
and insert few rows into each one. Recently, it started to fail after creating 
several tens of tables. An exception is always the same:
{code:java}
12:05:40.687 [junit-timeout-thread-51] INFO  o.g.q.s.g.TableGeneratorHelper - 
Update: create table test_table_49(id INTEGER not null, column_1 VARCHAR(50) 
not null, column_2 VARCHAR(50) not null, column_3 VARCHAR(50) not null, 
column_4 VARCHAR(50) not null, primary key (id))
Dec 15, 2023 12:05:42 PM org.apache.ignite.internal.logger.IgniteLogger 
logInternal
INFO: Partition assignment change notification received 
[remoteAddress=localhost:10800]
12:05:42.699 [junit-timeout-thread-51] INFO  o.g.q.s.g.TableGeneratorHelper - 
Creation of table: test_table_49 took 2012 ms
12:05:42.700 [junit-timeout-thread-1] INFO  
o.g.a.tests.TablesAmountCapacityTest - Creation of 50 tables took 101686 ms
Dec 15, 2023 12:05:42 PM org.apache.ignite.internal.logger.IgniteLogger 
logInternal
INFO: Partition assignment change notification received 
[remoteAddress=localhost:10800]
12:05:45.539 [junit-timeout-thread-1] INFO  
o.g.a.tests.TablesAmountCapacityTest - Table test_table_0 contains 1 rows
12:05:45.591 [junit-timeout-thread-1] INFO  
o.g.a.tests.TablesAmountCapacityTest - Table test_table_1 contains 1 rows
Dec 15, 2023 12:05:45 PM org.apache.ignite.internal.logger.IgniteLogger error
SEVERE: Failed to deserialize server response [remoteAddress=localhost:10800]: 
refCnt: 0
io.netty.util.IllegalReferenceCountException: refCnt: 0
at 
io.netty.buffer.AbstractByteBuf.ensureAccessible(AbstractByteBuf.java:1454)
at 
io.netty.buffer.AbstractByteBuf.checkReadableBytes0(AbstractByteBuf.java:1440)
at io.netty.buffer.AbstractByteBuf.readByte(AbstractByteBuf.java:730)
at 
org.apache.ignite.internal.client.proto.ClientMessageUnpacker.unpackBoolean(ClientMessageUnpacker.java:209)
at 
org.apache.ignite.internal.jdbc.proto.event.Response.readBinary(Response.java:81)
at 
org.apache.ignite.internal.jdbc.JdbcClientQueryCursorHandler.lambda$closeAsync$3(JdbcClientQueryCursorHandler.java:65)
at 
org.apache.ignite.internal.client.TcpClientChannel.lambda$receiveAsync$5(TcpClientChannel.java:368)
at 
java.base/java.util.concurrent.CompletableFuture.uniApplyNow(CompletableFuture.java:680)
at 
java.base/java.util.concurrent.CompletableFuture.uniApplyStage(CompletableFuture.java:658)
at 
java.base/java.util.concurrent.CompletableFuture.thenApply(CompletableFuture.java:2094)
at 
org.apache.ignite.internal.client.TcpClientChannel.receiveAsync(TcpClientChannel.java:362)
at 
org.apache.ignite.internal.client.TcpClientChannel.serviceAsync(TcpClientChannel.java:283)
at 
org.apache.ignite.internal.client.ClientChannel.serviceAsync(ClientChannel.java:59)
at 
org.apache.ignite.internal.jdbc.JdbcClientQueryCursorHandler.closeAsync(JdbcClientQueryCursorHandler.java:62)
at 
org.apache.ignite.internal.jdbc.JdbcResultSet.close0(JdbcResultSet.java:287)
at 
org.apache.ignite.internal.jdbc.JdbcResultSet.close(JdbcResultSet.java:268)
at 
org.gridgain.ai3tests.tests.TablesAmountCapacityTest.assertTableContains1RowAndColumnsValuesAreCorrect(TablesAmountCapacityTest.java:172)
at 
org.gridgain.ai3tests.tests.TablesAmountCapacityTest.assertTablesAreValid(TablesAmountCapacityTest.java:285)
at 
org.gridgain.ai3tests.tests.TablesAmountCapacityTest.create1000EmptyTablesAmountOfColumnsEachAndMakeSimpleQueries(TablesAmountCapacityTest.java:102)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at 
org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:728)
at 
org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
at 
org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
at 
io.qameta.allure.junit5.AllureJunit5.interceptTestTemplateMethod(AllureJunit5.java:59)
at 
org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103)
at 
org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93)
at 

[jira] [Resolved] (IGNITE-21090) "IllegalReferenceCountException: refCnt: 0" in SQL query

2023-12-15 Thread Pavel Tupitsyn (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-21090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Tupitsyn resolved IGNITE-21090.
-
Resolution: Duplicate

> "IllegalReferenceCountException: refCnt: 0" in SQL query
> 
>
> Key: IGNITE-21090
> URL: https://issues.apache.org/jira/browse/IGNITE-21090
> Project: Ignite
>  Issue Type: Bug
>  Components: sql
>Affects Versions: 3.0.0-beta2
>Reporter: Andrey Khitrin
>Priority: Major
>  Labels: client, ignite-3, sql
>
> We have a simple automated test that creates 1000 tables of the same 
> structure and insert few rows into each one. Recently, it started to fail 
> after creating several tens of tables. An exception is always the same:
> {code:java}
> 12:05:40.687 [junit-timeout-thread-51] INFO  o.g.q.s.g.TableGeneratorHelper - 
> Update: create table test_table_49(id INTEGER not null, column_1 VARCHAR(50) 
> not null, column_2 VARCHAR(50) not null, column_3 VARCHAR(50) not null, 
> column_4 VARCHAR(50) not null, primary key (id))
> Dec 15, 2023 12:05:42 PM org.apache.ignite.internal.logger.IgniteLogger 
> logInternal
> INFO: Partition assignment change notification received 
> [remoteAddress=localhost:10800]
> 12:05:42.699 [junit-timeout-thread-51] INFO  o.g.q.s.g.TableGeneratorHelper - 
> Creation of table: test_table_49 took 2012 ms
> 12:05:42.700 [junit-timeout-thread-1] INFO  
> o.g.a.tests.TablesAmountCapacityTest - Creation of 50 tables took 101686 ms
> Dec 15, 2023 12:05:42 PM org.apache.ignite.internal.logger.IgniteLogger 
> logInternal
> INFO: Partition assignment change notification received 
> [remoteAddress=localhost:10800]
> 12:05:45.539 [junit-timeout-thread-1] INFO  
> o.g.a.tests.TablesAmountCapacityTest - Table test_table_0 contains 1 rows
> 12:05:45.591 [junit-timeout-thread-1] INFO  
> o.g.a.tests.TablesAmountCapacityTest - Table test_table_1 contains 1 rows
> Dec 15, 2023 12:05:45 PM org.apache.ignite.internal.logger.IgniteLogger error
> SEVERE: Failed to deserialize server response 
> [remoteAddress=localhost:10800]: refCnt: 0
> io.netty.util.IllegalReferenceCountException: refCnt: 0
>   at 
> io.netty.buffer.AbstractByteBuf.ensureAccessible(AbstractByteBuf.java:1454)
>   at 
> io.netty.buffer.AbstractByteBuf.checkReadableBytes0(AbstractByteBuf.java:1440)
>   at io.netty.buffer.AbstractByteBuf.readByte(AbstractByteBuf.java:730)
>   at 
> org.apache.ignite.internal.client.proto.ClientMessageUnpacker.unpackBoolean(ClientMessageUnpacker.java:209)
>   at 
> org.apache.ignite.internal.jdbc.proto.event.Response.readBinary(Response.java:81)
>   at 
> org.apache.ignite.internal.jdbc.JdbcClientQueryCursorHandler.lambda$closeAsync$3(JdbcClientQueryCursorHandler.java:65)
>   at 
> org.apache.ignite.internal.client.TcpClientChannel.lambda$receiveAsync$5(TcpClientChannel.java:368)
>   at 
> java.base/java.util.concurrent.CompletableFuture.uniApplyNow(CompletableFuture.java:680)
>   at 
> java.base/java.util.concurrent.CompletableFuture.uniApplyStage(CompletableFuture.java:658)
>   at 
> java.base/java.util.concurrent.CompletableFuture.thenApply(CompletableFuture.java:2094)
>   at 
> org.apache.ignite.internal.client.TcpClientChannel.receiveAsync(TcpClientChannel.java:362)
>   at 
> org.apache.ignite.internal.client.TcpClientChannel.serviceAsync(TcpClientChannel.java:283)
>   at 
> org.apache.ignite.internal.client.ClientChannel.serviceAsync(ClientChannel.java:59)
>   at 
> org.apache.ignite.internal.jdbc.JdbcClientQueryCursorHandler.closeAsync(JdbcClientQueryCursorHandler.java:62)
>   at 
> org.apache.ignite.internal.jdbc.JdbcResultSet.close0(JdbcResultSet.java:287)
>   at 
> org.apache.ignite.internal.jdbc.JdbcResultSet.close(JdbcResultSet.java:268)
>   at 
> org.gridgain.ai3tests.tests.TablesAmountCapacityTest.assertTableContains1RowAndColumnsValuesAreCorrect(TablesAmountCapacityTest.java:172)
>   at 
> org.gridgain.ai3tests.tests.TablesAmountCapacityTest.assertTablesAreValid(TablesAmountCapacityTest.java:285)
>   at 
> org.gridgain.ai3tests.tests.TablesAmountCapacityTest.create1000EmptyTablesAmountOfColumnsEachAndMakeSimpleQueries(TablesAmountCapacityTest.java:102)
>   at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.base/java.lang.reflect.Method.invoke(Method.java:566)
>   at 
> org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:728)
>   at 
> org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)

[jira] [Comment Edited] (IGNITE-20995) Add more integration tests for tx recovery on unstable topology

2023-12-15 Thread Denis Chudov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-20995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17796781#comment-17796781
 ] 

Denis Chudov edited comment on IGNITE-20995 at 12/15/23 9:52 AM:
-

>From my point of view, we should have following scenarios:
 # Lock conflict between two RW transactions, coordinator is lost for lock 
holder, recovery starts, lock holder is aborted, lock waiter is not affected.
 # Lock conflict between two RW transactions, coordinator is alive, recovery is 
not started, lock waiter is not affected.
 # Resolution of write intent belonging to tx without coordinator, recovery 
happens successfully, write intent is switched.
 # Resolution of write intent belonging to abandoned tx, recovery happens 
successfully, write intent is switched.
 # Resolution of write intent belonging to abandoned tx, commit partition has 
restarted and lost its local volatile tx state map, recovery happens 
successfully, write intent is switched.
 # Resolution of write intent belonging to pending transaction, coordinator is 
alive, recovery is not started.
 # RO transaction tx0 resolves write intent belonging to the transaction tx1 
and marks it as abandoned and starts the recovery; after that RW transaction 
tx2 meets the lock belonging to tx1, sees that it's abandoned recently and 
doesn't start the recovery. Recovery is triggered just once.
 # Coordinator is lost, but it has sent the commit message to a commit 
partition, in the same time the recovery initiating request is received from 
some data node. Commit is successful, tx recovery was not able to change 
transaction state, there are no assertions or other errors, write intents on 
data node are switched.
 # Coordinator is lost, but it has sent the commit message to a commit 
partition, in the same time the recovery initiating request is received from 
some data node. Recovery successfully aborts the transaction, the is correct 
exception on the coordinator, it was not able to change transaction state to 
commit, there are no assertions or other errors, write intents on data node are 
switched.
 # Parallel tx recoveries happen on two replicas of commit partition on 
different nodes, both processes were started at a moment when the corresponding 
replica was the primary one. This also shouldnt break anything.
 # There are two parallel recoveries on commit partition initiated by tx state 
requests and a commit process initiated by coordinator that is already dead. 
Transaction is committed. Both recovery processes get correct commit timestamp 
and end up with correct write intent resolution (one recovery process receives 
commit timestamp from exception thrown by raft client, another should receive 
it from the completed finish future of local FINISHED state).


was (Author: denis chudov):
>From my point of view, we should have following scenarios:
 # Lock conflict between two RW transactions, coordinator is lost for lock 
holder, recovery starts, lock holder is aborted, lock waiter is not affected.
 # Lock conflict between two RW transactions, coordinator is alive, recovery is 
not started, lock waiter is not affected.
 # Resolution of write intent belonging to tx without coordinator, recovery 
happens successfully, write intent is switched.
 # Resolution of write intent belonging to abandoned tx, recovery happens 
successfully, write intent is switched.
 # Resolution of write intent belonging to abandoned tx, commit partition has 
restarted and lost its local volatile tx state map, recovery happens 
successfully, write intent is switched.
 # Resolution of write intent belonging to pending transaction, coordinator is 
alive, recovery is not started.
 # RO transaction tx0 resolves write intent belonging to the transaction tx1 
and marks it as abandoned and starts the recovery; after that RW transaction 
tx2 meets the lock belonging to tx1, sees that it's abandoned recently and 
doesn't start the recovery. Recovery is triggered just once.
 # Coordinator is lost, but it has sent the commit message to a commit 
partition, in the same time the recovery initiating request is received from 
some data node. Commit is successful, tx recovery was not able to change 
transaction state, there are no assertions or other errors, write intents on 
data node are switched.
 # Coordinator is lost, but it has sent the commit message to a commit 
partition, in the same time the recovery initiating request is received from 
some data node. Recovery successfully aborts the transaction, the is correct 
exception on the coordinator, it was not able to change transaction state to 
commit, there are no assertions or other errors, write intents on data node are 
switched.
 # Parallel tx recoveries happen on two replicas of commit partition, both 
processes were started at a moment when the corresponding replica was the 
primary one. This also shouldnt break anything.
 # There 

[jira] [Comment Edited] (IGNITE-20995) Add more integration tests for tx recovery on unstable topology

2023-12-15 Thread Denis Chudov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-20995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17796781#comment-17796781
 ] 

Denis Chudov edited comment on IGNITE-20995 at 12/15/23 9:51 AM:
-

>From my point of view, we should have following scenarios:
 # Lock conflict between two RW transactions, coordinator is lost for lock 
holder, recovery starts, lock holder is aborted, lock waiter is not affected.
 # Lock conflict between two RW transactions, coordinator is alive, recovery is 
not started, lock waiter is not affected.
 # Resolution of write intent belonging to tx without coordinator, recovery 
happens successfully, write intent is switched.
 # Resolution of write intent belonging to abandoned tx, recovery happens 
successfully, write intent is switched.
 # Resolution of write intent belonging to abandoned tx, commit partition has 
restarted and lost its local volatile tx state map, recovery happens 
successfully, write intent is switched.
 # Resolution of write intent belonging to pending transaction, coordinator is 
alive, recovery is not started.
 # RO transaction tx0 resolves write intent belonging to the transaction tx1 
and marks it as abandoned and starts the recovery; after that RW transaction 
tx2 meets the lock belonging to tx1, sees that it's abandoned recently and 
doesn't start the recovery. Recovery is triggered just once.
 # Coordinator is lost, but it has sent the commit message to a commit 
partition, in the same time the recovery initiating request is received from 
some data node. Commit is successful, tx recovery was not able to change 
transaction state, there are no assertions or other errors, write intents on 
data node are switched.
 # Coordinator is lost, but it has sent the commit message to a commit 
partition, in the same time the recovery initiating request is received from 
some data node. Recovery successfully aborts the transaction, the is correct 
exception on the coordinator, it was not able to change transaction state to 
commit, there are no assertions or other errors, write intents on data node are 
switched.
 # Parallel tx recoveries happen on two replicas of commit partition, both 
processes were started at a moment when the corresponding replica was the 
primary one. This also shouldnt break anything.
 # There are two parallel recoveries on commit partition initiated by tx state 
requests and a commit process initiated by coordinator that is already dead. 
Transaction is committed. Both recovery processes get correct commit timestamp 
and end up with correct write intent resolution (one recovery process receives 
commit timestamp from exception thrown by raft client, another should receive 
it from the completed finish future of local FINISHED state).


was (Author: denis chudov):
>From my point of view, we should have following scenarios:
 # Lock conflict between two RW transactions, coordinator is lost for lock 
holder, recovery starts, lock holder is aborted, lock waiter is not affected.
 # Lock conflict between two RW transactions, coordinator is alive, recovery is 
not started, lock waiter is not affected.
 # Resolution of write intent belonging to tx without coordinator, recovery 
happens successfully, write intent is switched.
 # Resolution of write intent belonging to abandoned tx, recovery happens 
successfully, write intent is switched.
 # Resolution of write intent belonging to abandoned tx, commit partition has 
restarted and lost its local volatile tx state map, recovery happens 
successfully, write intent is switched.
 # Resolution of write intent belonging to pending transaction, coordinator is 
alive, recovery is not started.
 # RO transaction tx0 resolves write intent belonging to the transaction tx1 
and marks it as abandoned and starts the recovery; after that RW transaction 
tx2 meets the lock belonging to tx1, sees that it's abandoned recently and 
doesn't start the recovery. Recovery is triggered just once.
 # Coordinator is lost, but it has sent the commit message to a commit 
partition, in the same time the recovery initiating request is received from 
some data node. Commit is successful, tx recovery was not able to change 
transaction state, there are no assertions or other errors, write intents on 
data node are switched.
 # Coordinator is lost, but it has sent the commit message to a commit 
partition, in the same time the recovery initiating request is received from 
some data node. Recovery successfully aborts the transaction, the is correct 
exception on the coordinator, it was not able to change transaction state to 
commit, there are no assertions or other errors, write intents on data node are 
switched.
 # Parallel tx recoveries happen on two replicas of commit partition, both 
processes were started at a moment when the corresponding replica was the 
primary one. This also shouldnt break anything.
 # There are two parallel 

[jira] [Updated] (IGNITE-21040) Placement driver logging enhancement

2023-12-15 Thread Denis Chudov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-21040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Denis Chudov updated IGNITE-21040:
--
Description: 
h3. Motivation
 # The lease grant request has a filed {{{}leaseExpirationTime{}}}, but the 
filed really means the timestamp until which we are waiting for a response to 
this message.
 # For some reason (external for the placement driver), the metastorage event 
can handle slower and slower. Finally, we cannot receive an event about the 
lease being prolonged for a particular period when the lease interval is valid.

h3. Definition of done
 # Remove {{leaseExpirationTime}} from the log message on replica side on lease 
acceptable (see {{Replica#acceptLease}} ).
 # Add {{toString}} method for {{{}Leases{}}}.

  was:
h3. Motivation
 # The lease grant request has a filed {{{}leaseExpirationTime{}}}, but the 
filed really means the timestamp until which we are waiting for a response to 
this message.
 # For some reason (external for the placement driver), the metastorage event 
can handle slower and slower. Finally, we cannot receive an event about the 
lease being prolonged for a particular period when the lease interval is valid.

h3. Definition of done
 # Remove {{leaseExpirationTime}} from the log message on replica side on lease 
acceptable (see {{Replica#acceptLease}} ).
 # Add a warning when we are handling the metastorage event for longer than 500 
milliseconds.
 # Add {{toString}} method for {{{}Leases{}}}.


> Placement driver logging enhancement
> 
>
> Key: IGNITE-21040
> URL: https://issues.apache.org/jira/browse/IGNITE-21040
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Vladislav Pyatkov
>Priority: Major
>  Labels: ignite-3
>
> h3. Motivation
>  # The lease grant request has a filed {{{}leaseExpirationTime{}}}, but the 
> filed really means the timestamp until which we are waiting for a response to 
> this message.
>  # For some reason (external for the placement driver), the metastorage event 
> can handle slower and slower. Finally, we cannot receive an event about the 
> lease being prolonged for a particular period when the lease interval is 
> valid.
> h3. Definition of done
>  # Remove {{leaseExpirationTime}} from the log message on replica side on 
> lease acceptable (see {{Replica#acceptLease}} ).
>  # Add {{toString}} method for {{{}Leases{}}}.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-21090) "IllegalReferenceCountException: refCnt: 0" in SQL query

2023-12-15 Thread Andrey Khitrin (Jira)
Andrey Khitrin created IGNITE-21090:
---

 Summary: "IllegalReferenceCountException: refCnt: 0" in SQL query
 Key: IGNITE-21090
 URL: https://issues.apache.org/jira/browse/IGNITE-21090
 Project: Ignite
  Issue Type: Bug
  Components: sql
Affects Versions: 3.0.0-beta2
Reporter: Andrey Khitrin


We have a simple automated test that creates 1000 tables of the same structure 
and insert few rows into each one. Recently, it started to fail after creating 
several tens of tables. An exception is always the same:
{code:java}
12:05:40.687 [junit-timeout-thread-51] INFO  o.g.q.s.g.TableGeneratorHelper - 
Update: create table test_table_49(id INTEGER not null, column_1 VARCHAR(50) 
not null, column_2 VARCHAR(50) not null, column_3 VARCHAR(50) not null, 
column_4 VARCHAR(50) not null, primary key (id))
Dec 15, 2023 12:05:42 PM org.apache.ignite.internal.logger.IgniteLogger 
logInternal
INFO: Partition assignment change notification received 
[remoteAddress=localhost:10800]
12:05:42.699 [junit-timeout-thread-51] INFO  o.g.q.s.g.TableGeneratorHelper - 
Creation of table: test_table_49 took 2012 ms
12:05:42.700 [junit-timeout-thread-1] INFO  
o.g.a.tests.TablesAmountCapacityTest - Creation of 50 tables took 101686 ms
Dec 15, 2023 12:05:42 PM org.apache.ignite.internal.logger.IgniteLogger 
logInternal
INFO: Partition assignment change notification received 
[remoteAddress=localhost:10800]
12:05:45.539 [junit-timeout-thread-1] INFO  
o.g.a.tests.TablesAmountCapacityTest - Table test_table_0 contains 1 rows
12:05:45.591 [junit-timeout-thread-1] INFO  
o.g.a.tests.TablesAmountCapacityTest - Table test_table_1 contains 1 rows
Dec 15, 2023 12:05:45 PM org.apache.ignite.internal.logger.IgniteLogger error
SEVERE: Failed to deserialize server response [remoteAddress=localhost:10800]: 
refCnt: 0
io.netty.util.IllegalReferenceCountException: refCnt: 0
at 
io.netty.buffer.AbstractByteBuf.ensureAccessible(AbstractByteBuf.java:1454)
at 
io.netty.buffer.AbstractByteBuf.checkReadableBytes0(AbstractByteBuf.java:1440)
at io.netty.buffer.AbstractByteBuf.readByte(AbstractByteBuf.java:730)
at 
org.apache.ignite.internal.client.proto.ClientMessageUnpacker.unpackBoolean(ClientMessageUnpacker.java:209)
at 
org.apache.ignite.internal.jdbc.proto.event.Response.readBinary(Response.java:81)
at 
org.apache.ignite.internal.jdbc.JdbcClientQueryCursorHandler.lambda$closeAsync$3(JdbcClientQueryCursorHandler.java:65)
at 
org.apache.ignite.internal.client.TcpClientChannel.lambda$receiveAsync$5(TcpClientChannel.java:368)
at 
java.base/java.util.concurrent.CompletableFuture.uniApplyNow(CompletableFuture.java:680)
at 
java.base/java.util.concurrent.CompletableFuture.uniApplyStage(CompletableFuture.java:658)
at 
java.base/java.util.concurrent.CompletableFuture.thenApply(CompletableFuture.java:2094)
at 
org.apache.ignite.internal.client.TcpClientChannel.receiveAsync(TcpClientChannel.java:362)
at 
org.apache.ignite.internal.client.TcpClientChannel.serviceAsync(TcpClientChannel.java:283)
at 
org.apache.ignite.internal.client.ClientChannel.serviceAsync(ClientChannel.java:59)
at 
org.apache.ignite.internal.jdbc.JdbcClientQueryCursorHandler.closeAsync(JdbcClientQueryCursorHandler.java:62)
at 
org.apache.ignite.internal.jdbc.JdbcResultSet.close0(JdbcResultSet.java:287)
at 
org.apache.ignite.internal.jdbc.JdbcResultSet.close(JdbcResultSet.java:268)
at 
org.gridgain.ai3tests.tests.TablesAmountCapacityTest.assertTableContains1RowAndColumnsValuesAreCorrect(TablesAmountCapacityTest.java:172)
at 
org.gridgain.ai3tests.tests.TablesAmountCapacityTest.assertTablesAreValid(TablesAmountCapacityTest.java:285)
at 
org.gridgain.ai3tests.tests.TablesAmountCapacityTest.create1000EmptyTablesAmountOfColumnsEachAndMakeSimpleQueries(TablesAmountCapacityTest.java:102)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at 
org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:728)
at 
org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
at 
org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
at 
io.qameta.allure.junit5.AllureJunit5.interceptTestTemplateMethod(AllureJunit5.java:59)
at 

[jira] [Comment Edited] (IGNITE-20995) Add more integration tests for tx recovery on unstable topology

2023-12-15 Thread Denis Chudov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-20995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17796781#comment-17796781
 ] 

Denis Chudov edited comment on IGNITE-20995 at 12/15/23 9:35 AM:
-

>From my point of view, we should have following scenarios:
 # Lock conflict between two RW transactions, coordinator is lost for lock 
holder, recovery starts, lock holder is aborted, lock waiter is not affected.
 # Lock conflict between two RW transactions, coordinator is alive, recovery is 
not started, lock waiter is not affected.
 # Resolution of write intent belonging to tx without coordinator, recovery 
happens successfully, write intent is switched.
 # Resolution of write intent belonging to abandoned tx, recovery happens 
successfully, write intent is switched.
 # Resolution of write intent belonging to abandoned tx, commit partition has 
restarted and lost its local volatile tx state map, recovery happens 
successfully, write intent is switched.
 # Resolution of write intent belonging to pending transaction, coordinator is 
alive, recovery is not started.
 # RO transaction tx0 resolves write intent belonging to the transaction tx1 
and marks it as abandoned and starts the recovery; after that RW transaction 
tx2 meets the lock belonging to tx1, sees that it's abandoned recently and 
doesn't start the recovery. Recovery is triggered just once.
 # Coordinator is lost, but it has sent the commit message to a commit 
partition, in the same time the recovery initiating request is received from 
some data node. Commit is successful, tx recovery was not able to change 
transaction state, there are no assertions or other errors, write intents on 
data node are switched.
 # Coordinator is lost, but it has sent the commit message to a commit 
partition, in the same time the recovery initiating request is received from 
some data node. Recovery successfully aborts the transaction, the is correct 
exception on the coordinator, it was not able to change transaction state to 
commit, there are no assertions or other errors, write intents on data node are 
switched.
 # Parallel tx recoveries happen on two replicas of commit partition, both 
processes were started at a moment when the corresponding replica was the 
primary one. This also shouldnt break anything.
 # There are two parallel recoveries on commit partition initiated by tx state 
requests and a commit process initiated by coordinator that is already dead. 
Transaction is committed. Both recovery processes get correct commit timestamp 
and end up with correct write intent resolution.


was (Author: denis chudov):
>From my point of view, we should have following scenarios:
 # Lock conflict between two RW transactions, coordinator is lost for lock 
holder, recovery starts, lock holder is aborted, lock waiter is not affected.
 # Lock conflict between two RW transactions, coordinator is alive, recovery is 
not started, lock waiter is not affected.
 # Resolution of write intent belonging to tx without coordinator, recovery 
happens successfully, write intent is switched.
 # Resolution of write intent belonging to abandoned tx, recovery happens 
successfully, write intent is switched.
 # Resolution of write intent belonging to abandoned tx, commit partition has 
restarted and lost its local volatile tx state map, recovery happens 
successfully, write intent is switched.
 # Resolution of write intent belonging to pending transaction, coordinator is 
alive, recovery is not started.
 # RO transaction tx0 resolves write intent belonging to the transaction tx1 
and marks it as abandoned and starts the recovery; after that RW transaction 
tx2 meets the lock belonging to tx1, sees that it's abandoned recently and 
doesn't start the recovery. Recovery is triggered just once.
 # Coordinator is lost, but it has sent the commit message to a commit 
partition, in the same time the recovery initiating request is received from 
some data node. Commit is successful, tx recovery was not able to change 
transaction state, there are no assertions or other errors, write intents on 
data node are switched.
 # Coordinator is lost, but it has sent the commit message to a commit 
partition, in the same time the recovery initiating request is received from 
some data node. Recovery successfully aborts the transaction, the is correct 
exception on the coordinator, it was not able to change transaction state to 
commit, there are no assertions or other errors, write intents on data node are 
switched.
 # Parallel tx recoveries happen on two replicas of commit partition, both 
processes were started at a moment when the corresponding replica was the 
primary one. This also shouldnt break anything.
 # There are two parallel recoveries on commit partition initiated by tx state 
requests and a commit process initiated by coordinator that is already dead. 
Both recovery processes get correct commit 

[jira] [Comment Edited] (IGNITE-20995) Add more integration tests for tx recovery on unstable topology

2023-12-15 Thread Denis Chudov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-20995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17796781#comment-17796781
 ] 

Denis Chudov edited comment on IGNITE-20995 at 12/15/23 9:33 AM:
-

>From my point of view, we should have following scenarios:
 # Lock conflict between two RW transactions, coordinator is lost for lock 
holder, recovery starts, lock holder is aborted, lock waiter is not affected.
 # Lock conflict between two RW transactions, coordinator is alive, recovery is 
not started, lock waiter is not affected.
 # Resolution of write intent belonging to tx without coordinator, recovery 
happens successfully, write intent is switched.
 # Resolution of write intent belonging to abandoned tx, recovery happens 
successfully, write intent is switched.
 # Resolution of write intent belonging to abandoned tx, commit partition has 
restarted and lost its local volatile tx state map, recovery happens 
successfully, write intent is switched.
 # Resolution of write intent belonging to pending transaction, coordinator is 
alive, recovery is not started.
 # RO transaction tx0 resolves write intent belonging to the transaction tx1 
and marks it as abandoned and starts the recovery; after that RW transaction 
tx2 meets the lock belonging to tx1, sees that it's abandoned recently and 
doesn't start the recovery. Recovery is triggered just once.
 # Coordinator is lost, but it has sent the commit message to a commit 
partition, in the same time the recovery initiating request is received from 
some data node. Commit is successful, tx recovery was not able to change 
transaction state, there are no assertions or other errors, write intents on 
data node are switched.
 # Coordinator is lost, but it has sent the commit message to a commit 
partition, in the same time the recovery initiating request is received from 
some data node. Recovery successfully aborts the transaction, the is correct 
exception on the coordinator, it was not able to change transaction state to 
commit, there are no assertions or other errors, write intents on data node are 
switched.
 # Parallel tx recoveries happen on two replicas of commit partition, both 
processes were started at a moment when the corresponding replica was the 
primary one. This also shouldnt break anything.
 # There are two parallel recoveries on commit partition and a commit process 
initiated by coordinator that is already dead. Both recovery processes get 
correct commit timestamp and resolve write intent correctly.


was (Author: denis chudov):
>From my point of view, we should have following scenarios:
 # Lock conflict between two RW transactions, coordinator is lost for lock 
holder, recovery starts, lock holder is aborted, lock waiter is not affected.
 # Lock conflict between two RW transactions, coordinator is alive, recovery is 
not started, lock waiter is not affected.
 # Resolution of write intent belonging to tx without coordinator, recovery 
happens successfully, write intent is switched.
 # Resolution of write intent belonging to abandoned tx, recovery happens 
successfully, write intent is switched.
 # Resolution of write intent belonging to abandoned tx, commit partition has 
restarted and lost its local volatile tx state map, recovery happens 
successfully, write intent is switched.
 # Resolution of write intent belonging to pending transaction, coordinator is 
alive, recovery is not started.
 # RO transaction tx0 resolves write intent belonging to the transaction tx1 
and marks it as abandoned and starts the recovery; after that RW transaction 
tx2 meets the lock belonging to tx1, sees that it's abandoned recently and 
doesn't start the recovery. Recovery is triggered just once.
 # Coordinator is lost, but it has sent the commit message to a commit 
partition, in the same time the recovery initiating request is received from 
some data node. Commit is successful, tx recovery was not able to change 
transaction state, there are no assertions or other errors, write intents on 
data node are switched.
 # Coordinator is lost, but it has sent the commit message to a commit 
partition, in the same time the recovery initiating request is received from 
some data node. Recovery successfully aborts the transaction, the is correct 
exception on the coordinator, it was not able to change transaction state to 
commit, there are no assertions or other errors, write intents on data node are 
switched.
 # Parallel tx recoveries happen on two replicas of commit partition, both 
processes were started at a moment when the corresponding replica was the 
primary one. This also shouldnt break anything.

> Add more integration tests for tx recovery on unstable topology
> ---
>
> Key: IGNITE-20995
> URL: https://issues.apache.org/jira/browse/IGNITE-20995
> Project: 

[jira] [Comment Edited] (IGNITE-20995) Add more integration tests for tx recovery on unstable topology

2023-12-15 Thread Denis Chudov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-20995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17796781#comment-17796781
 ] 

Denis Chudov edited comment on IGNITE-20995 at 12/15/23 9:35 AM:
-

>From my point of view, we should have following scenarios:
 # Lock conflict between two RW transactions, coordinator is lost for lock 
holder, recovery starts, lock holder is aborted, lock waiter is not affected.
 # Lock conflict between two RW transactions, coordinator is alive, recovery is 
not started, lock waiter is not affected.
 # Resolution of write intent belonging to tx without coordinator, recovery 
happens successfully, write intent is switched.
 # Resolution of write intent belonging to abandoned tx, recovery happens 
successfully, write intent is switched.
 # Resolution of write intent belonging to abandoned tx, commit partition has 
restarted and lost its local volatile tx state map, recovery happens 
successfully, write intent is switched.
 # Resolution of write intent belonging to pending transaction, coordinator is 
alive, recovery is not started.
 # RO transaction tx0 resolves write intent belonging to the transaction tx1 
and marks it as abandoned and starts the recovery; after that RW transaction 
tx2 meets the lock belonging to tx1, sees that it's abandoned recently and 
doesn't start the recovery. Recovery is triggered just once.
 # Coordinator is lost, but it has sent the commit message to a commit 
partition, in the same time the recovery initiating request is received from 
some data node. Commit is successful, tx recovery was not able to change 
transaction state, there are no assertions or other errors, write intents on 
data node are switched.
 # Coordinator is lost, but it has sent the commit message to a commit 
partition, in the same time the recovery initiating request is received from 
some data node. Recovery successfully aborts the transaction, the is correct 
exception on the coordinator, it was not able to change transaction state to 
commit, there are no assertions or other errors, write intents on data node are 
switched.
 # Parallel tx recoveries happen on two replicas of commit partition, both 
processes were started at a moment when the corresponding replica was the 
primary one. This also shouldnt break anything.
 # There are two parallel recoveries on commit partition initiated by tx state 
requests and a commit process initiated by coordinator that is already dead. 
Both recovery processes get correct commit timestamp and end up with correct 
write intent resolution.


was (Author: denis chudov):
>From my point of view, we should have following scenarios:
 # Lock conflict between two RW transactions, coordinator is lost for lock 
holder, recovery starts, lock holder is aborted, lock waiter is not affected.
 # Lock conflict between two RW transactions, coordinator is alive, recovery is 
not started, lock waiter is not affected.
 # Resolution of write intent belonging to tx without coordinator, recovery 
happens successfully, write intent is switched.
 # Resolution of write intent belonging to abandoned tx, recovery happens 
successfully, write intent is switched.
 # Resolution of write intent belonging to abandoned tx, commit partition has 
restarted and lost its local volatile tx state map, recovery happens 
successfully, write intent is switched.
 # Resolution of write intent belonging to pending transaction, coordinator is 
alive, recovery is not started.
 # RO transaction tx0 resolves write intent belonging to the transaction tx1 
and marks it as abandoned and starts the recovery; after that RW transaction 
tx2 meets the lock belonging to tx1, sees that it's abandoned recently and 
doesn't start the recovery. Recovery is triggered just once.
 # Coordinator is lost, but it has sent the commit message to a commit 
partition, in the same time the recovery initiating request is received from 
some data node. Commit is successful, tx recovery was not able to change 
transaction state, there are no assertions or other errors, write intents on 
data node are switched.
 # Coordinator is lost, but it has sent the commit message to a commit 
partition, in the same time the recovery initiating request is received from 
some data node. Recovery successfully aborts the transaction, the is correct 
exception on the coordinator, it was not able to change transaction state to 
commit, there are no assertions or other errors, write intents on data node are 
switched.
 # Parallel tx recoveries happen on two replicas of commit partition, both 
processes were started at a moment when the corresponding replica was the 
primary one. This also shouldnt break anything.
 # There are two parallel recoveries on commit partition initiated by tx state 
request and a commit process initiated by coordinator that is already dead. 
Both recovery processes get correct commit timestamp and end up with 

[jira] [Comment Edited] (IGNITE-20995) Add more integration tests for tx recovery on unstable topology

2023-12-15 Thread Denis Chudov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-20995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17796781#comment-17796781
 ] 

Denis Chudov edited comment on IGNITE-20995 at 12/15/23 9:34 AM:
-

>From my point of view, we should have following scenarios:
 # Lock conflict between two RW transactions, coordinator is lost for lock 
holder, recovery starts, lock holder is aborted, lock waiter is not affected.
 # Lock conflict between two RW transactions, coordinator is alive, recovery is 
not started, lock waiter is not affected.
 # Resolution of write intent belonging to tx without coordinator, recovery 
happens successfully, write intent is switched.
 # Resolution of write intent belonging to abandoned tx, recovery happens 
successfully, write intent is switched.
 # Resolution of write intent belonging to abandoned tx, commit partition has 
restarted and lost its local volatile tx state map, recovery happens 
successfully, write intent is switched.
 # Resolution of write intent belonging to pending transaction, coordinator is 
alive, recovery is not started.
 # RO transaction tx0 resolves write intent belonging to the transaction tx1 
and marks it as abandoned and starts the recovery; after that RW transaction 
tx2 meets the lock belonging to tx1, sees that it's abandoned recently and 
doesn't start the recovery. Recovery is triggered just once.
 # Coordinator is lost, but it has sent the commit message to a commit 
partition, in the same time the recovery initiating request is received from 
some data node. Commit is successful, tx recovery was not able to change 
transaction state, there are no assertions or other errors, write intents on 
data node are switched.
 # Coordinator is lost, but it has sent the commit message to a commit 
partition, in the same time the recovery initiating request is received from 
some data node. Recovery successfully aborts the transaction, the is correct 
exception on the coordinator, it was not able to change transaction state to 
commit, there are no assertions or other errors, write intents on data node are 
switched.
 # Parallel tx recoveries happen on two replicas of commit partition, both 
processes were started at a moment when the corresponding replica was the 
primary one. This also shouldnt break anything.
 # There are two parallel recoveries on commit partition initiated by tx state 
request and a commit process initiated by coordinator that is already dead. 
Both recovery processes get correct commit timestamp and end up with correct 
write intent resolution.


was (Author: denis chudov):
>From my point of view, we should have following scenarios:
 # Lock conflict between two RW transactions, coordinator is lost for lock 
holder, recovery starts, lock holder is aborted, lock waiter is not affected.
 # Lock conflict between two RW transactions, coordinator is alive, recovery is 
not started, lock waiter is not affected.
 # Resolution of write intent belonging to tx without coordinator, recovery 
happens successfully, write intent is switched.
 # Resolution of write intent belonging to abandoned tx, recovery happens 
successfully, write intent is switched.
 # Resolution of write intent belonging to abandoned tx, commit partition has 
restarted and lost its local volatile tx state map, recovery happens 
successfully, write intent is switched.
 # Resolution of write intent belonging to pending transaction, coordinator is 
alive, recovery is not started.
 # RO transaction tx0 resolves write intent belonging to the transaction tx1 
and marks it as abandoned and starts the recovery; after that RW transaction 
tx2 meets the lock belonging to tx1, sees that it's abandoned recently and 
doesn't start the recovery. Recovery is triggered just once.
 # Coordinator is lost, but it has sent the commit message to a commit 
partition, in the same time the recovery initiating request is received from 
some data node. Commit is successful, tx recovery was not able to change 
transaction state, there are no assertions or other errors, write intents on 
data node are switched.
 # Coordinator is lost, but it has sent the commit message to a commit 
partition, in the same time the recovery initiating request is received from 
some data node. Recovery successfully aborts the transaction, the is correct 
exception on the coordinator, it was not able to change transaction state to 
commit, there are no assertions or other errors, write intents on data node are 
switched.
 # Parallel tx recoveries happen on two replicas of commit partition, both 
processes were started at a moment when the corresponding replica was the 
primary one. This also shouldnt break anything.
 # There are two parallel recoveries on commit partition and a commit process 
initiated by coordinator that is already dead. Both recovery processes get 
correct commit timestamp and resolve write intent correctly.

> Add more 

[jira] [Commented] (IGNITE-21048) A way to change default storage engine in tests

2023-12-15 Thread Kirill Tkalenko (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-21048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17797098#comment-17797098
 ] 

Kirill Tkalenko commented on IGNITE-21048:
--

Looks good.

> A way to change default storage engine in tests
> ---
>
> Key: IGNITE-21048
> URL: https://issues.apache.org/jira/browse/IGNITE-21048
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Sergey Chugunov
>Assignee: Sergey Chugunov
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> With the current implementation of CatalogManager default Storage Engine is 
> hardcoded to AIPERSIST (PageMemory implementation with persistence) and there 
> is no way to configure it to something else.
> There is an idea to integrate specification of Default Distribution Zone 
> configuration into init script, but it is not implemented yet.
> So the only way to specify a different engine and test against it is to 
> introduce a system property that could change defaults.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-21084) Account for differences of logical part of now() on different nodes when waiting after a DDL

2023-12-15 Thread Roman Puchkovskiy (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-21084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Roman Puchkovskiy updated IGNITE-21084:
---
Reviewer: Konstantin Orlov

> Account for differences of logical part of now() on different nodes when 
> waiting after a DDL
> 
>
> Key: IGNITE-21084
> URL: https://issues.apache.org/jira/browse/IGNITE-21084
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Roman Puchkovskiy
>Assignee: Roman Puchkovskiy
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Client must see the results of a DDL they just executed, no matter through 
> which node of the cluster a subsequent request is made. To maintain this, we 
> must make sure that ActivationTimestamp(DDL) becomes non-future on all nodes 
> of the cluster and only then can we complete the DDL future. As nodes' 
> physical clocks might have some skew relatively to each other, we add 
> MaxClockSkew to the timestamp till which we wait to compensate for this 
> difference.
> Analogously to HLC.now() being different because its physical part differs on 
> different nodes, it might be different because *logical* parts are different 
> on different nodes.
> Let's assume we don't have any physical clock skew, and MaxClockSkew is 0. 
> ActivationTimestamp(DDL) is (100, 5); (100 is physical part, 5 is logical 
> part). We wait on node 1 (through which the DDL was executed) till its 
> HLC.now() reaches (100, 5), then we complete the DDL future. The user goes to 
> node 2 at which HLC.now() is (100, 2). The user executes a query at 'now', 
> and the query sees the state before the DDL was executed, which breaks our 
> requirement.
> We can fix this by 'rounding' the timestamp-to-wait-for up in the following 
> way:
>  # If logical part is not 0, increment the physical part and set the logical 
> part to 0
>  # If the logical part is 0, leave the timestamp as is
> As a result, for (100, 0) we will wait for (100, 0), and at node 1 HLC is at 
> least (100, 0), so it cannot see the previous schema. For (100, 5) we would 
> wait till (101, 0), which would also guarantee that a query executed after 
> waiting sees the new schema version.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-21089) [YCSB] "IgniteException: readerIndex: 0, writerIndex: 257" on SQL SELECT

2023-12-15 Thread Ivan Artiukhov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-21089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ivan Artiukhov updated IGNITE-21089:

Labels: ignite-3  (was: ignite-3 performance)

> [YCSB] "IgniteException: readerIndex: 0, writerIndex: 257" on SQL SELECT
> 
>
> Key: IGNITE-21089
> URL: https://issues.apache.org/jira/browse/IGNITE-21089
> Project: Ignite
>  Issue Type: Bug
>  Components: sql
>Reporter: Ivan Artiukhov
>Priority: Major
>  Labels: ignite-3
> Attachments: 2023-12-15-04-58-54_run.txt, logs-818.zip
>
>
> AI3 rev. {{ba6cd4dfff0e4bc2eacb437a1331454e030bff84}}
> Benchmark: 
> [https://github.com/gridgain/YCSB/blob/ycsb-2023.9/ignite3/src/main/java/site/ycsb/db/ignite3/IgniteSqlClient.java]
>  
> The benchmark uses Ignite 3 Java SQL API to perform SQL queries. 
> h1. Setup
> 1 server node, 1 client node (1 thread)
> h1. Steps
> The benchmark was started 2 times:
>  # in {{load}} mode to preload 165k unique entries via {{INSERT}} (15k in 
> warmup phase + 150k in payload phase)
>  # in {{run}} mode to {{SELECT}} each entry one by one
> h1. Expected result
>  # The {{load}} mode finishes successfully without errors
>  # The {{run}} mode finishes successfully without errors
> h1. Actual result
>  # The {{load}} mode finishes successfully without errors
>  # In the {{run}} mode the very first 128 entries were read with the 
> following error. All other entries were read successfully.
> {noformat}
> org.apache.ignite.lang.IgniteException: readerIndex: 0, writerIndex: 257 
> (expected: 0 <= readerIndex <= writerIndex <= capacity(256))
>   at 
> java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:710) 
> ~[?:?]
>   at 
> org.apache.ignite.internal.util.ExceptionUtils$1.copy(ExceptionUtils.java:754)
>  ~[ignite-core-3.0.0-SNAPSHOT.jar:?]
>   at 
> org.apache.ignite.internal.util.ExceptionUtils$ExceptionFactory.createCopy(ExceptionUtils.java:688)
>  ~[ignite-core-3.0.0-SNAPSHOT.jar:?]
>   at 
> org.apache.ignite.internal.util.ExceptionUtils.copyExceptionWithCause(ExceptionUtils.java:525)
>  ~[ignite-core-3.0.0-SNAPSHOT.jar:?]
>   at 
> org.apache.ignite.internal.util.ExceptionUtils.copyExceptionWithCauseInternal(ExceptionUtils.java:623)
>  ~[ignite-core-3.0.0-SNAPSHOT.jar:?]
>   at 
> org.apache.ignite.internal.util.ExceptionUtils.copyExceptionWithCause(ExceptionUtils.java:476)
>  ~[ignite-core-3.0.0-SNAPSHOT.jar:?]
>   at 
> org.apache.ignite.internal.sql.AbstractSession.execute(AbstractSession.java:63)
>  ~[ignite-core-3.0.0-SNAPSHOT.jar:?]
>   at site.ycsb.db.ignite3.IgniteSqlClient.read(IgniteSqlClient.java:51) 
> [ignite3-binding-2023.9.jar:?]
>   at site.ycsb.DBWrapper.read(DBWrapper.java:145) [core-2023.9.jar:?]
>   at 
> site.ycsb.workloads.CoreWorkload.doTransactionRead(CoreWorkload.java:746) 
> [core-2023.9.jar:?]
>   at 
> site.ycsb.workloads.CoreWorkload.doTransaction(CoreWorkload.java:666) 
> [core-2023.9.jar:?]
>   at site.ycsb.ClientThread.run(ClientThread.java:145) [core-2023.9.jar:?]
>   at java.lang.Thread.run(Thread.java:829) [?:?]
> Caused by: java.util.concurrent.CompletionException: 
> org.apache.ignite.lang.IgniteException: IGN-CMN-65535 
> TraceId:b0e3aa62-ba41-47bd-8238-f39537f78e28 readerIndex: 0, writerIndex: 257 
> (expected: 0 <= readerIndex <= writerIndex <= capacity(256))
>   at 
> java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:331)
>  ~[?:?]
>   at 
> java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:346)
>  ~[?:?]
>   at 
> java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:632)
>  ~[?:?]
>   at 
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
>  ~[?:?]
>   at 
> java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2088)
>  ~[?:?]
>   at 
> org.apache.ignite.internal.client.TcpClientChannel.processNextMessage(TcpClientChannel.java:417)
>  ~[ignite-client-3.0.0-SNAPSHOT.jar:?]
>   at 
> org.apache.ignite.internal.client.TcpClientChannel.lambda$onMessage$3(TcpClientChannel.java:241)
>  ~[ignite-client-3.0.0-SNAPSHOT.jar:?]
>   at 
> java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1426)
>  ~[?:?]
>   at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) 
> ~[?:?]
>   at 
> java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
>  ~[?:?]
>   at java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656) ~[?:?]
>   at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594) 
> ~[?:?]
>   at 
> java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183) 
> ~[?:?]
> 

[jira] [Updated] (IGNITE-21089) [YCSB] "IgniteException: readerIndex: 0, writerIndex: 257" on SQL SELECT

2023-12-15 Thread Ivan Artiukhov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-21089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ivan Artiukhov updated IGNITE-21089:

Description: 
AI3 rev. {{ba6cd4dfff0e4bc2eacb437a1331454e030bff84}}

Benchmark: 
[https://github.com/gridgain/YCSB/blob/ycsb-2023.9/ignite3/src/main/java/site/ycsb/db/ignite3/IgniteSqlClient.java]
 

The benchmark uses Ignite 3 Java SQL API to perform SQL queries. 
h1. Setup

1 server node, 1 client node (1 thread)
h1. Steps

The benchmark was started 2 times:
 # in {{load}} mode to preload 165k unique entries via {{INSERT}} (15k in 
warmup phase + 150k in payload phase)
 # in {{run}} mode to {{SELECT}} each entry one by one

h1. Expected result
 # The {{load}} mode finishes successfully without errors
 # The {{run}} mode finishes successfully without errors

h1. Actual result
 # The {{load}} mode finishes successfully without errors
 # In the {{run}} mode the very first 128 entries were read with the following 
error. All other entries were read successfully.

{noformat}
org.apache.ignite.lang.IgniteException: readerIndex: 0, writerIndex: 257 
(expected: 0 <= readerIndex <= writerIndex <= capacity(256))
at 
java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:710) ~[?:?]
at 
org.apache.ignite.internal.util.ExceptionUtils$1.copy(ExceptionUtils.java:754) 
~[ignite-core-3.0.0-SNAPSHOT.jar:?]
at 
org.apache.ignite.internal.util.ExceptionUtils$ExceptionFactory.createCopy(ExceptionUtils.java:688)
 ~[ignite-core-3.0.0-SNAPSHOT.jar:?]
at 
org.apache.ignite.internal.util.ExceptionUtils.copyExceptionWithCause(ExceptionUtils.java:525)
 ~[ignite-core-3.0.0-SNAPSHOT.jar:?]
at 
org.apache.ignite.internal.util.ExceptionUtils.copyExceptionWithCauseInternal(ExceptionUtils.java:623)
 ~[ignite-core-3.0.0-SNAPSHOT.jar:?]
at 
org.apache.ignite.internal.util.ExceptionUtils.copyExceptionWithCause(ExceptionUtils.java:476)
 ~[ignite-core-3.0.0-SNAPSHOT.jar:?]
at 
org.apache.ignite.internal.sql.AbstractSession.execute(AbstractSession.java:63) 
~[ignite-core-3.0.0-SNAPSHOT.jar:?]
at site.ycsb.db.ignite3.IgniteSqlClient.read(IgniteSqlClient.java:51) 
[ignite3-binding-2023.9.jar:?]
at site.ycsb.DBWrapper.read(DBWrapper.java:145) [core-2023.9.jar:?]
at 
site.ycsb.workloads.CoreWorkload.doTransactionRead(CoreWorkload.java:746) 
[core-2023.9.jar:?]
at 
site.ycsb.workloads.CoreWorkload.doTransaction(CoreWorkload.java:666) 
[core-2023.9.jar:?]
at site.ycsb.ClientThread.run(ClientThread.java:145) [core-2023.9.jar:?]
at java.lang.Thread.run(Thread.java:829) [?:?]
Caused by: java.util.concurrent.CompletionException: 
org.apache.ignite.lang.IgniteException: IGN-CMN-65535 
TraceId:b0e3aa62-ba41-47bd-8238-f39537f78e28 readerIndex: 0, writerIndex: 257 
(expected: 0 <= readerIndex <= writerIndex <= capacity(256))
at 
java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:331)
 ~[?:?]
at 
java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:346)
 ~[?:?]
at 
java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:632)
 ~[?:?]
at 
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) 
~[?:?]
at 
java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2088)
 ~[?:?]
at 
org.apache.ignite.internal.client.TcpClientChannel.processNextMessage(TcpClientChannel.java:417)
 ~[ignite-client-3.0.0-SNAPSHOT.jar:?]
at 
org.apache.ignite.internal.client.TcpClientChannel.lambda$onMessage$3(TcpClientChannel.java:241)
 ~[ignite-client-3.0.0-SNAPSHOT.jar:?]
at 
java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1426)
 ~[?:?]
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) 
~[?:?]
at 
java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
 ~[?:?]
at java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656) ~[?:?]
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594) 
~[?:?]
at 
java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183) 
~[?:?]
Caused by: org.apache.ignite.lang.IgniteException: readerIndex: 0, writerIndex: 
257 (expected: 0 <= readerIndex <= writerIndex <= capacity(256))
at 
java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:710) ~[?:?]
at 
org.apache.ignite.internal.util.ExceptionUtils$1.copy(ExceptionUtils.java:754) 
~[ignite-core-3.0.0-SNAPSHOT.jar:?]
at 
org.apache.ignite.internal.util.ExceptionUtils$ExceptionFactory.createCopy(ExceptionUtils.java:688)
 ~[ignite-core-3.0.0-SNAPSHOT.jar:?]
at 
org.apache.ignite.internal.util.ExceptionUtils.copyExceptionWithCause(ExceptionUtils.java:525)
 ~[ignite-core-3.0.0-SNAPSHOT.jar:?]
at 

[jira] [Commented] (IGNITE-21089) [YCSB] "IgniteException: readerIndex: 0, writerIndex: 257" on SQL SELECT

2023-12-15 Thread Ivan Artiukhov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-21089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17797093#comment-17797093
 ] 

Ivan Artiukhov commented on IGNITE-21089:
-

A similar [benchmark which use 
JDBC|https://github.com/gridgain/YCSB/blob/ycsb-2023.9/ignite3/src/main/java/site/ycsb/db/ignite3/IgniteJdbcClient.java]
 to make queries does not show such {{{}IgniteException{}}}. 

> [YCSB] "IgniteException: readerIndex: 0, writerIndex: 257" on SQL SELECT
> 
>
> Key: IGNITE-21089
> URL: https://issues.apache.org/jira/browse/IGNITE-21089
> Project: Ignite
>  Issue Type: Bug
>  Components: sql
>Reporter: Ivan Artiukhov
>Priority: Major
>  Labels: ignite-3, performance
> Attachments: 2023-12-15-04-58-54_run.txt, logs-818.zip
>
>
> AI3 rev. 
> Benchmark: 
> [https://github.com/gridgain/YCSB/blob/ycsb-2023.9/ignite3/src/main/java/site/ycsb/db/ignite3/IgniteSqlClient.java]
>  
> h1. Setup
> 1 server node, 1 client node (1 thread)
> h1. Steps
> The benchmark was started 2 times:
>  # in {{load}} mode to preload 165k unique entries via {{INSERT}} (15k in 
> warmup phase + 150k in payload phase)
>  # in {{run}} mode to {{SELECT}} each entry one by one
> h1. Expected result
>  # The {{load}} mode finishes successfully without errors
>  # The {{run}} mode finishes successfully without errors
> h1. Actual result
>  # The {{load}} mode finishes successfully without errors
>  # In the {{run}} mode the very first 128 entries were read with the 
> following error. All other entries were read successfully.
> {noformat}
> org.apache.ignite.lang.IgniteException: readerIndex: 0, writerIndex: 257 
> (expected: 0 <= readerIndex <= writerIndex <= capacity(256))
>   at 
> java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:710) 
> ~[?:?]
>   at 
> org.apache.ignite.internal.util.ExceptionUtils$1.copy(ExceptionUtils.java:754)
>  ~[ignite-core-3.0.0-SNAPSHOT.jar:?]
>   at 
> org.apache.ignite.internal.util.ExceptionUtils$ExceptionFactory.createCopy(ExceptionUtils.java:688)
>  ~[ignite-core-3.0.0-SNAPSHOT.jar:?]
>   at 
> org.apache.ignite.internal.util.ExceptionUtils.copyExceptionWithCause(ExceptionUtils.java:525)
>  ~[ignite-core-3.0.0-SNAPSHOT.jar:?]
>   at 
> org.apache.ignite.internal.util.ExceptionUtils.copyExceptionWithCauseInternal(ExceptionUtils.java:623)
>  ~[ignite-core-3.0.0-SNAPSHOT.jar:?]
>   at 
> org.apache.ignite.internal.util.ExceptionUtils.copyExceptionWithCause(ExceptionUtils.java:476)
>  ~[ignite-core-3.0.0-SNAPSHOT.jar:?]
>   at 
> org.apache.ignite.internal.sql.AbstractSession.execute(AbstractSession.java:63)
>  ~[ignite-core-3.0.0-SNAPSHOT.jar:?]
>   at site.ycsb.db.ignite3.IgniteSqlClient.read(IgniteSqlClient.java:51) 
> [ignite3-binding-2023.9.jar:?]
>   at site.ycsb.DBWrapper.read(DBWrapper.java:145) [core-2023.9.jar:?]
>   at 
> site.ycsb.workloads.CoreWorkload.doTransactionRead(CoreWorkload.java:746) 
> [core-2023.9.jar:?]
>   at 
> site.ycsb.workloads.CoreWorkload.doTransaction(CoreWorkload.java:666) 
> [core-2023.9.jar:?]
>   at site.ycsb.ClientThread.run(ClientThread.java:145) [core-2023.9.jar:?]
>   at java.lang.Thread.run(Thread.java:829) [?:?]
> Caused by: java.util.concurrent.CompletionException: 
> org.apache.ignite.lang.IgniteException: IGN-CMN-65535 
> TraceId:b0e3aa62-ba41-47bd-8238-f39537f78e28 readerIndex: 0, writerIndex: 257 
> (expected: 0 <= readerIndex <= writerIndex <= capacity(256))
>   at 
> java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:331)
>  ~[?:?]
>   at 
> java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:346)
>  ~[?:?]
>   at 
> java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:632)
>  ~[?:?]
>   at 
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
>  ~[?:?]
>   at 
> java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2088)
>  ~[?:?]
>   at 
> org.apache.ignite.internal.client.TcpClientChannel.processNextMessage(TcpClientChannel.java:417)
>  ~[ignite-client-3.0.0-SNAPSHOT.jar:?]
>   at 
> org.apache.ignite.internal.client.TcpClientChannel.lambda$onMessage$3(TcpClientChannel.java:241)
>  ~[ignite-client-3.0.0-SNAPSHOT.jar:?]
>   at 
> java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1426)
>  ~[?:?]
>   at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) 
> ~[?:?]
>   at 
> java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
>  ~[?:?]
>   at java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656) ~[?:?]
>   at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594) 
> 

[jira] [Commented] (IGNITE-21051) Fix javadocs for IndexQuery

2023-12-15 Thread Oleg Valuyskiy (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-21051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17797091#comment-17797091
 ] 

Oleg Valuyskiy commented on IGNITE-21051:
-

[~timonin.maksim], please review my changes. 

Bulletpoints for index discovering algorithms and paragraph separators have 
been added to Javadoc in the IndexQuery class.

> Fix javadocs for IndexQuery
> ---
>
> Key: IGNITE-21051
> URL: https://issues.apache.org/jira/browse/IGNITE-21051
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Maksim Timonin
>Assignee: Oleg Valuyskiy
>Priority: Major
>  Labels: ise, newbie
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> It's required to fix javadoc formatting in the `IndexQuery` class. Now it 
> renders the algorithm list in single line. Should use "ul", "li" tags for 
> correct rendering.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-21089) [YCSB] "IgniteException: readerIndex: 0, writerIndex: 257" on SQL SELECT

2023-12-15 Thread Ivan Artiukhov (Jira)
Ivan Artiukhov created IGNITE-21089:
---

 Summary: [YCSB] "IgniteException: readerIndex: 0, writerIndex: 
257" on SQL SELECT
 Key: IGNITE-21089
 URL: https://issues.apache.org/jira/browse/IGNITE-21089
 Project: Ignite
  Issue Type: Bug
  Components: sql
Reporter: Ivan Artiukhov
 Attachments: 2023-12-15-04-58-54_run.txt, logs-818.zip

AI3 rev. 

Benchmark: 
[https://github.com/gridgain/YCSB/blob/ycsb-2023.9/ignite3/src/main/java/site/ycsb/db/ignite3/IgniteSqlClient.java]
 
h1. Setup

1 server node, 1 client node (1 thread)
h1. Steps

The benchmark was started 2 times:
 # in {{load}} mode to preload 165k unique entries via {{INSERT}} (15k in 
warmup phase + 150k in payload phase)
 # in {{run}} mode to {{SELECT}} each entry one by one

h1. Expected result
 # The {{load}} mode finishes successfully without errors
 # The {{run}} mode finishes successfully without errors

h1. Actual result
 # The {{load}} mode finishes successfully without errors
 # In the {{run}} mode the very first 128 entries were read with the following 
error. All other entries were read successfully.

{noformat}
org.apache.ignite.lang.IgniteException: readerIndex: 0, writerIndex: 257 
(expected: 0 <= readerIndex <= writerIndex <= capacity(256))
at 
java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:710) ~[?:?]
at 
org.apache.ignite.internal.util.ExceptionUtils$1.copy(ExceptionUtils.java:754) 
~[ignite-core-3.0.0-SNAPSHOT.jar:?]
at 
org.apache.ignite.internal.util.ExceptionUtils$ExceptionFactory.createCopy(ExceptionUtils.java:688)
 ~[ignite-core-3.0.0-SNAPSHOT.jar:?]
at 
org.apache.ignite.internal.util.ExceptionUtils.copyExceptionWithCause(ExceptionUtils.java:525)
 ~[ignite-core-3.0.0-SNAPSHOT.jar:?]
at 
org.apache.ignite.internal.util.ExceptionUtils.copyExceptionWithCauseInternal(ExceptionUtils.java:623)
 ~[ignite-core-3.0.0-SNAPSHOT.jar:?]
at 
org.apache.ignite.internal.util.ExceptionUtils.copyExceptionWithCause(ExceptionUtils.java:476)
 ~[ignite-core-3.0.0-SNAPSHOT.jar:?]
at 
org.apache.ignite.internal.sql.AbstractSession.execute(AbstractSession.java:63) 
~[ignite-core-3.0.0-SNAPSHOT.jar:?]
at site.ycsb.db.ignite3.IgniteSqlClient.read(IgniteSqlClient.java:51) 
[ignite3-binding-2023.9.jar:?]
at site.ycsb.DBWrapper.read(DBWrapper.java:145) [core-2023.9.jar:?]
at 
site.ycsb.workloads.CoreWorkload.doTransactionRead(CoreWorkload.java:746) 
[core-2023.9.jar:?]
at 
site.ycsb.workloads.CoreWorkload.doTransaction(CoreWorkload.java:666) 
[core-2023.9.jar:?]
at site.ycsb.ClientThread.run(ClientThread.java:145) [core-2023.9.jar:?]
at java.lang.Thread.run(Thread.java:829) [?:?]
Caused by: java.util.concurrent.CompletionException: 
org.apache.ignite.lang.IgniteException: IGN-CMN-65535 
TraceId:b0e3aa62-ba41-47bd-8238-f39537f78e28 readerIndex: 0, writerIndex: 257 
(expected: 0 <= readerIndex <= writerIndex <= capacity(256))
at 
java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:331)
 ~[?:?]
at 
java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:346)
 ~[?:?]
at 
java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:632)
 ~[?:?]
at 
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) 
~[?:?]
at 
java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2088)
 ~[?:?]
at 
org.apache.ignite.internal.client.TcpClientChannel.processNextMessage(TcpClientChannel.java:417)
 ~[ignite-client-3.0.0-SNAPSHOT.jar:?]
at 
org.apache.ignite.internal.client.TcpClientChannel.lambda$onMessage$3(TcpClientChannel.java:241)
 ~[ignite-client-3.0.0-SNAPSHOT.jar:?]
at 
java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1426)
 ~[?:?]
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) 
~[?:?]
at 
java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
 ~[?:?]
at java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656) ~[?:?]
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594) 
~[?:?]
at 
java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183) 
~[?:?]
Caused by: org.apache.ignite.lang.IgniteException: readerIndex: 0, writerIndex: 
257 (expected: 0 <= readerIndex <= writerIndex <= capacity(256))
at 
java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:710) ~[?:?]
at 
org.apache.ignite.internal.util.ExceptionUtils$1.copy(ExceptionUtils.java:754) 
~[ignite-core-3.0.0-SNAPSHOT.jar:?]
at 
org.apache.ignite.internal.util.ExceptionUtils$ExceptionFactory.createCopy(ExceptionUtils.java:688)
 ~[ignite-core-3.0.0-SNAPSHOT.jar:?]
at