zq0757 opened a new issue, #5553: URL: https://github.com/apache/seatunnel/issues/5553
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happened hbase synchronizes 70 million data in batches to doris, but the task is not completed until ckp times out ### SeaTunnel Version 2.3.3 ### SeaTunnel Config ```conf env { job.mode=BATCH job.name=hbase2doris-batch parallelism = 3 checkpoint.interval = 100000 } source { Jdbc { parallelism = 3 fetch_size="10000" driver = org.apache.phoenix.jdbc.PhoenixDriver url = "jdbc:phoenix:ip:2182/hbase" query = "select NODE_ID as N,to_char(to_date(DATA_TIME,'yyyy-MM-dd HH:mm:ss'),'yyyyMMdd') as DAY,DATA_TIME as TIME,HOUR(TO_TIMESTAMP(DATA_TIME)) as HOUR_TIME,MINUTE(TO_TIMESTAMP(DATA_TIME)) as MINUTE_TIME, DATA_TIME as CREATE_TIME,VAL as V from SZWATER.PDATA_HIS" } } transform { } sink { Doris{ fenodes = "ip:18030" username = root password = "pwd" table.identifier = "test_kafka.PDATA_HIS_A" sink.enable-2pc = "true" sink.label-prefix = "test_json-01" #sink.buffer-size=262144 #sink.buffer-count=200000 doris.config = { format="json" read_json_by_line="true" } } } ``` ### Running Command ```shell The seatunnel.sh script is started ``` ### Error Exception ```log [INFO] 2023-09-25 16:03:12.876 +0800 - -> 23/09/25 16:03:11 INFO JobMetricsRunner: *********************************************** Job Progress Information *********************************************** Job Id : 758575984650747906 Read Count So Far : 78720196 Write Count So Far : 78718147 Average Read Count : 0/s Average Write Count : 0/s Last Statistic Time : 2023-09-25 16:02:11 Current Statistic Time : 2023-09-25 16:03:11 *********************************************** [INFO] 2023-09-25 16:03:54.885 +0800 - -> 2023-09-25 16:03:54,566 INFO com.hazelcast.client.impl.connection.tcp.TcpClientConnection - hz.client_1 [seatunnel] [5.1] ClientConnection{alive=false, connectionId=2, channel=NioChannel{/172.28.246.19:41909->/172.28.122.174:5801}, remoteAddress=[172.28.122.174]:5801, lastReadTime=2023-09-25 16:03:53.254, lastWriteTime=2023-09-25 16:03:53.252, closedTime=2023-09-25 16:03:54.565, connected server version=5.1} closed. Reason: Connection closed by the other side 2023-09-25 16:03:54,570 INFO com.hazelcast.client.impl.connection.ClientConnectionManager - hz.client_1 [seatunnel] [5.1] Removed connection to endpoint: [172.28.122.174]:5801:73fc4473-271d-48b4-a0b0-8bbeb0fb7dc7, connection: ClientConnection{alive=false, connectionId=2, channel=NioChannel{/172.28.246.19:41909->/172.28.122.174:5801}, remoteAddress=[172.28.122.174]:5801, lastReadTime=2023-09-25 16:03:53.254, lastWriteTime=2023-09-25 16:03:53.252, closedTime=2023-09-25 16:03:54.565, connected server version=5.1} 2023-09-25 16:03:54,572 INFO org.apache.seatunnel.engine.client.job.ClientJobProxy - Job (758575984650747906) end with unknown state, and throw Exception: java.util.concurrent.ExecutionException: com.hazelcast.core.OperationTimeoutException: ClientInvocation{clientMessage = ClientMessage{connection=null, length=30, operation=SeaTunnel.WaitForJobComplete, isRetryable=true, correlationId=13, messageType=de0300, isEvent=false, isfragmented=false}, objectName = null, target = uuid 73fc4473-271d-48b4-a0b0-8bbeb0fb7dc7, sentConnection = null} timed out because exception occurred after client invocation timeout 120000 ms. Current time: 2023-09-25 16:03:54.569. Start time: 2023-09-25 14:30:11.859. Total elapsed time: 5622710 ms. at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357) at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1908) at org.apache.seatunnel.engine.client.job.ClientJobProxy.lambda$waitForJobComplete$0(ClientJobProxy.java:101) at org.apache.seatunnel.common.utils.RetryUtils.retryWithException(RetryUtils.java:48) at org.apache.seatunnel.engine.client.job.ClientJobProxy.waitForJobComplete(ClientJobProxy.java:97) at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:184) at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40) at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34) Caused by: com.hazelcast.core.OperationTimeoutException: ClientInvocation{clientMessage = ClientMessage{connection=null, length=30, operation=SeaTunnel.WaitForJobComplete, isRetryable=true, correlationId=13, messageType=de0300, isEvent=false, isfragmented=false}, objectName = null, target = uuid 73fc4473-271d-48b4-a0b0-8bbeb0fb7dc7, sentConnection = null} timed out because exception occurred after client invocation timeout 120000 ms. Current time: 2023-09-25 16:03:54.569. Start time: 2023-09-25 14:30:11.859. Total elapsed time: 5622710 ms. at com.hazelcast.client.impl.spi.impl.ClientInvocation.notifyExceptionWithOwnedPermission(ClientInvocation.java:339) at com.hazelcast.client.impl.spi.impl.ClientInvocationServiceImpl.onConnectionClose(ClientInvocationServiceImpl.java:221) at com.hazelcast.client.impl.connection.tcp.TcpClientConnectionManager.onConnectionClose(TcpClientConnectionManager.java:758) at com.hazelcast.client.impl.connection.tcp.TcpClientConnection.close(TcpClientConnection.java:191) at com.hazelcast.client.impl.connection.tcp.TcpClientConnectionManager$ClientConnectionChannelErrorHandler.onError(TcpClientConnectionManager.java:1140) at com.hazelcast.internal.networking.nio.NioPipeline.onError(NioPipeline.java:277) at com.hazelcast.internal.networking.nio.NioThread.processSelectionKey(NioThread.java:385) at com.hazelcast.internal.networking.nio.NioThread.processSelectionKeys(NioThread.java:368) at com.hazelcast.internal.networking.nio.NioThread.selectLoop(NioThread.java:294) at com.hazelcast.internal.networking.nio.NioThread.executeRun(NioThread.java:249) at com.hazelcast.internal.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:102) Caused by: com.hazelcast.spi.exception.TargetDisconnectedException: Connection closed by the other side at com.hazelcast.client.impl.spi.impl.ClientInvocationServiceImpl.onConnectionClose(ClientInvocationServiceImpl.java:220) ... 9 more Caused by: java.io.EOFException: Remote socket closed! at com.hazelcast.internal.networking.nio.NioInboundPipeline.process(NioInboundPipeline.java:117) at com.hazelcast.internal.networking.nio.NioThread.processSelectionKey(NioThread.java:383) ... 4 more ``` ### Zeta or Flink or Spark Version Zeta ### Java or Scala Version 1.8 ### Screenshots _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
