Igor created IGNITE-23231:
-----------------------------
Summary: [FLAKY] Exception "Send with retry timed out" while
creating 1000 tables in 10 threads with replication 1
Key: IGNITE-23231
URL: https://issues.apache.org/jira/browse/IGNITE-23231
Project: Ignite
Issue Type: Bug
Environment: Cluster of *1 server* located on 1 host.
Reporter: Igor
Attachments: server-logs.zip
*Steps to reproduce:*
Cluster of *1 server* located on 1 host.
# In 10 threads create 50 tables:
## Create zone if not exists (same name for all tables) with replication 1.
## Create table (all tables have different names).
# In 1 thread assert tables count
# In 10 threads: insert 1 row in every table created in step 1
# In 10 threads: assert tables rows are correct.
# Repeat steps 1-4 until 1000 tables are created.
*Expected:*
All tables are created.
*Actual:*
Exception on client side after 850-950 tables:
{code:java}
java.sql.SQLException: Send with retry timed out [retryCount = 5]. at
app//org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:152)
at
app//org.junit.jupiter.api.AssertDoesNotThrow.createAssertionFailedError(AssertDoesNotThrow.java:84)
at
app//org.junit.jupiter.api.AssertDoesNotThrow.assertDoesNotThrow(AssertDoesNotThrow.java:53)
at
app//org.junit.jupiter.api.AssertDoesNotThrow.assertDoesNotThrow(AssertDoesNotThrow.java:40)
at
app//org.junit.jupiter.api.Assertions.assertDoesNotThrow(Assertions.java:3183)
at
app//org.gridgain.ai3tests.tests.amountcapacity.TablesAmountCapacityBaseTest.lambda$invokeTasks$14(TablesAmountCapacityBaseTest.java:361)
at app//io.qameta.allure.Allure.lambda$step$0(Allure.java:113) at
app//io.qameta.allure.Allure.lambda$step$1(Allure.java:127) at
app//io.qameta.allure.Allure.step(Allure.java:181) at
app//io.qameta.allure.Allure.step(Allure.java:125) at
app//io.qameta.allure.Allure.step(Allure.java:112) at
app//org.gridgain.ai3tests.tests.amountcapacity.TablesAmountCapacityBaseTest.step(TablesAmountCapacityBaseTest.java:255)
at
app//org.gridgain.ai3tests.tests.amountcapacity.TablesAmountCapacityBaseTest.invokeTasks(TablesAmountCapacityBaseTest.java:356)
at
app//org.gridgain.ai3tests.tests.amountcapacity.TablesAmountCapacityBaseTest.createTablesParallel(TablesAmountCapacityBaseTest.java:130)
at
app//org.gridgain.ai3tests.tests.amountcapacity.TablesAmountCapacityBaseTest.testTablesAmount(TablesAmountCapacityBaseTest.java:91)
at
app//org.gridgain.ai3tests.tests.amountcapacity.TablesAmountCapacityTest.create1000EmptyTablesAmountOfColumnsEachAndMakeSimpleQueries(TablesAmountCapacityTest.java:86)
at [email protected]/java.lang.reflect.Method.invoke(Method.java:568) at
app//io.qameta.allure.junit5.AllureJunit5.interceptTestTemplateMethod(AllureJunit5.java:59)
at [email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at [email protected]/java.lang.Thread.run(Thread.java:833)Caused by:
java.util.concurrent.ExecutionException: java.sql.SQLException: Send with retry
timed out [retryCount = 5]. at
java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122) at
java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191) at
org.gridgain.ai3tests.tests.amountcapacity.TablesAmountCapacityBaseTest.lambda$invokeTasks$13(TablesAmountCapacityBaseTest.java:363)
at
org.junit.jupiter.api.AssertDoesNotThrow.assertDoesNotThrow(AssertDoesNotThrow.java:49)
... 19 moreCaused by: java.sql.SQLException: Send with retry timed out
[retryCount = 5]. at
org.apache.ignite.internal.jdbc.proto.IgniteQueryErrorCode.createJdbcSqlException(IgniteQueryErrorCode.java:57)
at
org.apache.ignite.internal.jdbc.JdbcStatement.execute0(JdbcStatement.java:154)
at
org.apache.ignite.internal.jdbc.JdbcStatement.executeUpdate(JdbcStatement.java:177)
at
org.gridgain.qaaas.sql.generator.TableGeneratorHelper.createTable(TableGeneratorHelper.java:98)
at
org.gridgain.ai3tests.tests.amountcapacity.TablesAmountCapacityBaseTest.lambda$createTablesParallel$0(TablesAmountCapacityBaseTest.java:122)
... 4 more {code}
Exception on server side:
{code:java}
2024-09-18 04:31:50:281 +0000
[ERROR][%TablesAmountCapacityTest_cluster_0%Raft-Group-Client-18][LeaseUpdater]
Lease update invocation failed
java.util.concurrent.CompletionException:
java.util.concurrent.TimeoutException: Send with retry timed out [retryCount =
4].
at
java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:332)
at
java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:347)
at
java.base/java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:636)
at
java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
at
java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2162)
at
org.apache.ignite.internal.raft.RaftGroupServiceImpl.sendWithRetry(RaftGroupServiceImpl.java:589)
at
org.apache.ignite.internal.raft.RaftGroupServiceImpl.lambda$handleThrowable$41(RaftGroupServiceImpl.java:661)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.util.concurrent.TimeoutException: Send with retry timed out
[retryCount = 4].
at
org.apache.ignite.internal.raft.RaftGroupServiceImpl.sendWithRetry(RaftGroupServiceImpl.java:590)
... 7 more {code}
Server logs: [^server-logs.zip]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
