dawner84 commented on issue #6638:
URL: https://github.com/apache/seatunnel/issues/6638#issuecomment-3827986044
发现seatunnel jdbc
sink在streaming任务(kafka->mysql)里面connection为啥不释放，一直是active，就算配置大了池子，或者配置了探测时间也没用，因为连接就不会释放。
下面是超时以后写入mysql的报错，这时候才return到池子已经晚了，kafka消息也消费了，但是数据页没写进去
2026-01-31 17:00:04,802 DEBUG [.s.e.s.c.CheckpointCoordinator]
[seatunnel-coordinator-service-13] - pending
checkpoint(141/7@1069898974783078401) completed! cost: 2, trigger: 1769850004800
, completed: 1769850004802
2026-01-31 17:00:04,802 DEBUG [.a.s.e.s.t.f.SinkFlowLifeCycle]
[BlockingWorker-TaskGroupLocation{jobId=1069898974783078401, pipelineId=7,
taskGroupId=14}] - trigger barrier [141] finished,
cost 1ms. taskLocation
[TaskLocation{taskGroupLocation=TaskGroupLocation{jobId=1069898974783078401,
pipelineId=7, taskGroupId=14}, taskID=7001400010000, index=0}]
**2026-01-31 17:00:04,803 WARN [.a.s.s.c.z.h.p.ProxyConnection]
[st-multi-table-sink-writer-2] - HikariPool-1 - Connection
com.mysql.cj.jdbc.ConnectionImpl@32e419c2 marked as broken because
of SQLSTATE(08S01), ErrorCode(0)**
com.mysql.cj.jdbc.exceptions.CommunicationsException: The last
packet successfully received from the server was 890,049 milliseconds ago. The
last packet sent successfully to the server wa
s 890,050 milliseconds ago. is longer than the server configured value of
'wait_timeout'. You should consider either expiring and/or testing connection
validity before use in your application, inc
reasing the server configured values for client timeouts, or using the
Connector/J connection property 'autoReconnect=true' to avoid this problem.
at
com.mysql.cj.jdbc.exceptions.SQLError.createCommunicationsException(SQLError.java:175)
~[mysql-connector-j-8.0.33.jar:8.0.33]
at
com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:64)
~[mysql-connector-j-8.0.33.jar:8.0.33]
at
com.mysql.cj.jdbc.ConnectionImpl.isReadOnly(ConnectionImpl.java:1374)
~[mysql-connector-j-8.0.33.jar:8.0.33]
at
com.mysql.cj.jdbc.ConnectionImpl.isReadOnly(ConnectionImpl.java:1359)
~[mysql-connector-j-8.0.33.jar:8.0.33]
at
com.mysql.cj.jdbc.ClientPreparedStatement.executeBatchInternal(ClientPreparedStatement.java:386)
~[mysql-connector-j-8.0.33.jar:8.0.33]
at
com.mysql.cj.jdbc.StatementImpl.executeBatch(StatementImpl.java:795)
~[mysql-connector-j-8.0.33.jar:8.0.33]
at
org.apache.seatunnel.shade.com.zaxxer.hikari.pool.ProxyStatement.executeBatch(ProxyStatement.java:127)
~[connector-jdbc-2.3.13-SNAPSHOT-2.12.15.jar:2.3.13-SNAPSHOT]
at
org.apache.seatunnel.shade.com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeBatch(HikariProxyPreparedStatement.java)
~[connector-jdbc-2.3.13-SNAPSHOT-2.12.15.jar:2.3.1
3-SNAPSHOT]
at
org.apache.seatunnel.connectors.seatunnel.jdbc.internal.executor.FieldNamedPreparedStatement.executeBatch(FieldNamedPreparedStatement.java:540)
~[connector-jdbc-2.3.13-SNAPSHOT-
2.12.15.jar:2.3.13-SNAPSHOT]
at
org.apache.seatunnel.connectors.seatunnel.jdbc.internal.executor.SimpleBatchStatementExecutor.executeBatch(SimpleBatchStatementExecutor.java:54)
~[connector-jdbc-2.3.13-SNAPSHOT
-2.12.15.jar:2.3.13-SNAPSHOT]
at
org.apache.seatunnel.connectors.seatunnel.jdbc.internal.executor.BufferReducedBatchStatementExecutor.executeBatch(BufferReducedBatchStatementExecutor.java:89)
~[connector-jdbc-2
.3.13-SNAPSHOT-2.12.15.jar:2.3.13-SNAPSHOT]
at
org.apache.seatunnel.connectors.seatunnel.jdbc.internal.JdbcOutputFormat.attemptFlush(JdbcOutputFormat.java:172)
~[connector-jdbc-2.3.13-SNAPSHOT-2.12.15.jar:2.3.13-SNAPSHOT]
at
org.apache.seatunnel.connectors.seatunnel.jdbc.internal.JdbcOutputFormat.flush(JdbcOutputFormat.java:136)
~[connector-jdbc-2.3.13-SNAPSHOT-2.12.15.jar:2.3.13-SNAPSHOT]
at
org.apache.seatunnel.connectors.seatunnel.jdbc.sink.JdbcSinkWriter.prepareCommit(JdbcSinkWriter.java:160)
~[connector-jdbc-2.3.13-SNAPSHOT-2.12.15.jar:2.3.13-SNAPSHOT]
at
org.apache.seatunnel.api.sink.SinkWriter.prepareCommit(SinkWriter.java:75)
~[seatunnel-starter.jar:2.3.12]
at
org.apache.seatunnel.api.sink.multitablesink.MultiTableSinkWriter.lambda$prepareCommit$4(MultiTableSinkWriter.java:259)
~[seatunnel-starter.jar:2.3.12]
2026-01-31 17:00:04,801 DEBUG [o.a.s.e.s.c.PendingCheckpoint ]
[hz.main.generic-operation.thread-33] - acknowledgeTask states
[[ActionSubtaskState(stateKey=ActionStateKey(name=ActionStateK
ey - pipeline-7 [Source[6]-Kafka]), index=0)]]
2026-01-31 17:00:04,801 DEBUG [s.c.o.TaskAcknowledgeOperation]
[hz.main.generic-operation.thread-33] - task ack finished
TaskLocation{taskGroupLocation=TaskGroupLocation{jobId=106989897478
3078401, pipelineId=7, taskGroupId=14}, taskID=7001400000000, index=0}
2026-01-31 17:00:04,801 DEBUG [.s.e.s.t.f.SourceFlowLifeCycle]
[hz.main.seaTunnel.task.thread-2] - send record to next finished, taskId:
[7001400000000]
2026-01-31 17:00:04,801 DEBUG [.s.e.s.t.f.SourceFlowLifeCycle]
[hz.main.seaTunnel.task.thread-2] - trigger barrier [141] finished, cost: 0ms.
taskLocation: [TaskLocation{taskGroupLocation=
TaskGroupLocation{jobId=1069898974783078401, pipelineId=7, taskGroupId=14},
taskID=7001400000000, index=0}]
2026-01-31 17:00:04,801 DEBUG [o.a.s.e.s.TaskExecutionService]
[hz.main.seaTunnel.task.thread-2] - [localhost]:5805 [seatunnel-620355] [5.1]
remove async execute function from TaskGroupLoc
ation{jobId=1069898974783078401, pipelineId=7, taskGroupId=14} with id
cc0bdb30-f66e-48eb-985f-6751740ede64
2026-01-31 17:00:04,801 DEBUG [o.a.s.e.s.t.SeaTunnelTask ]
[BlockingWorker-TaskGroupLocation{jobId=1069898974783078401, pipelineId=7,
taskGroupId=14}] - seatunnel task ack barrier[Task
ation{jobId=1069898974783078401, pipelineId=7, taskGroupId=14} with id
cc0bdb30-f66e-48eb-985f-6751740ede64
2026-01-31 17:00:04,801 DEBUG [o.a.s.e.s.t.SeaTunnelTask ]
[BlockingWorker-TaskGroupLocation{jobId=1069898974783078401, pipelineId=7,
taskGroupId=14}] - seatunnel task ack barrier[Task
Location{taskGroupLocation=TaskGroupLocation{jobId=1069898974783078401,
pipelineId=7, taskGroupId=14}, taskID=7001400010000, index=0}]
2026-01-31 17:00:04,801 DEBUG [o.a.s.c.s.j.i.JdbcOutputFormat]
[st-multi-table-sink-writer-1] - No data to flush.
2026-01-31 17:00:04,802 DEBUG [o.a.s.e.s.t.SeaTunnelTask ]
[BlockingWorker-TaskGroupLocation{jobId=1069898974783078401, pipelineId=7,
taskGroupId=14}] - seatunnel task ack barrier[Task
Location{taskGroupLocation=TaskGroupLocation{jobId=1069898974783078401,
pipelineId=7, taskGroupId=14}, taskID=7001400010000, index=0}]
2026-01-31 17:00:04,802 DEBUG [s.c.o.TaskAcknowledgeOperation]
[hz.main.generic-operation.thread-12] - TaskAcknowledgeOperation
TaskLocation{taskGroupLocation=TaskGroupLocation{jobId=10698
98974783078401, pipelineId=7, taskGroupId=14}, taskID=7001400010000, index=0}
2026-01-31 17:00:04,802 DEBUG [o.a.s.e.s.c.CheckpointManager ]
[hz.main.generic-operation.thread-12] - checkpoint manager received ack
TaskLocation{taskGroupLocation=TaskGroupLocation{jobI
d=1069898974783078401, pipelineId=7, taskGroupId=14}, taskID=7001400010000,
index=0}
2026-01-31 17:00:04,802 DEBUG [.s.e.s.c.CheckpointCoordinator]
[hz.main.generic-operation.thread-12] -
task[7001400010000](7/1069898974783078401) ack. CheckpointBarrier 141 @
1769850004800
type: CHECKPOINT_TYPE, prepareClose: [], closed: []
2026-01-31 17:00:04,802 DEBUG [o.a.s.e.s.c.PendingCheckpoint ]
[hz.main.generic-operation.thread-12] - acknowledgeTask states
[[ActionSubtaskState(stateKey=ActionStateKey(name=ActionStateK
ey - pipeline-7 [Sink[6]-jdbc-MultiTableSink]), index=0)]]
2026-01-31 17:00:04,802 DEBUG [o.a.s.e.s.c.PendingCheckpoint ]
[hz.main.generic-operation.thread-12] - checkpoint is full ack!
2026-01-31 17:00:04,802 DEBUG [s.c.o.TaskAcknowledgeOperation]
[hz.main.generic-operation.thread-12] - task ack finished
TaskLocation{taskGroupLocation=TaskGroupLocation{jobId=106989897478
3078401, pipelineId=7, taskGroupId=14}, taskID=7001400010000, index=0}
2026-01-31 17:00:04,802 DEBUG [.s.e.s.c.CheckpointCoordinator]
[seatunnel-coordinator-service-13] - pending
checkpoint(141/7@1069898974783078401) completed! cost: 2, trigger: 1769850004800
, completed: 1769850004802
2026-01-31 17:00:04,802 DEBUG [.a.s.e.s.t.f.SinkFlowLifeCycle]
[BlockingWorker-TaskGroupLocation{jobId=1069898974783078401, pipelineId=7,
taskGroupId=14}] - trigger barrier [141] finished,
cost 1ms. taskLocation
[TaskLocation{taskGroupLocation=TaskGroupLocation{jobId=1069898974783078401,
pipelineId=7, taskGroupId=14}, taskID=7001400010000, index=0}]
2026-01-31 17:00:04,803 WARN [.a.s.s.c.z.h.p.ProxyConnection]
[st-multi-table-sink-writer-2] - HikariPool-1 - Connection
com.mysql.cj.jdbc.ConnectionImpl@32e419c2 marked as broken because
of SQLSTATE(08S01), ErrorCode(0)
com.mysql.cj.jdbc.exceptions.CommunicationsException: The last
packet successfully received from the server was 890,049 milliseconds ago. The
last packet sent successfully to the server wa
s 890,050 milliseconds ago. is longer than the server configured value of
'wait_timeout'. You should consider either expiring and/or testing connection
validity before use in your application, inc
reasing the server configured values for client timeouts, or using the
Connector/J connection property 'autoReconnect=true' to avoid this problem.
at
com.mysql.cj.jdbc.exceptions.SQLError.createCommunicationsException(SQLError.java:175)
~[mysql-connector-j-8.0.33.jar:8.0.33]
at
com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:64)
~[mysql-connector-j-8.0.33.jar:8.0.33]
at
com.mysql.cj.jdbc.ConnectionImpl.isReadOnly(ConnectionImpl.java:1374)
~[mysql-connector-j-8.0.33.jar:8.0.33]
at
com.mysql.cj.jdbc.ConnectionImpl.isReadOnly(ConnectionImpl.java:1359)
~[mysql-connector-j-8.0.33.jar:8.0.33]
at
com.mysql.cj.jdbc.ClientPreparedStatement.executeBatchInternal(ClientPreparedStatement.java:386)
~[mysql-connector-j-8.0.33.jar:8.0.33]
at
com.mysql.cj.jdbc.StatementImpl.executeBatch(StatementImpl.java:795)
~[mysql-connector-j-8.0.33.jar:8.0.33]
at
org.apache.seatunnel.shade.com.zaxxer.hikari.pool.ProxyStatement.executeBatch(ProxyStatement.java:127)
~[connector-jdbc-2.3.13-SNAPSHOT-2.12.15.jar:2.3.13-SNAPSHOT]
at
org.apache.seatunnel.shade.com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeBatch(HikariProxyPreparedStatement.java)
~[connector-jdbc-2.3.13-SNAPSHOT-2.12.15.jar:2.3.1
3-SNAPSHOT]
at
org.apache.seatunnel.connectors.seatunnel.jdbc.internal.executor.FieldNamedPreparedStatement.executeBatch(FieldNamedPreparedStatement.java:540)
~[connector-jdbc-2.3.13-SNAPSHOT-
2.12.15.jar:2.3.13-SNAPSHOT]
at
org.apache.seatunnel.connectors.seatunnel.jdbc.internal.executor.SimpleBatchStatementExecutor.executeBatch(SimpleBatchStatementExecutor.java:54)
~[connector-jdbc-2.3.13-SNAPSHOT
-2.12.15.jar:2.3.13-SNAPSHOT]
at
org.apache.seatunnel.connectors.seatunnel.jdbc.internal.executor.BufferReducedBatchStatementExecutor.executeBatch(BufferReducedBatchStatementExecutor.java:89)
~[connector-jdbc-2
.3.13-SNAPSHOT-2.12.15.jar:2.3.13-SNAPSHOT]
at
org.apache.seatunnel.connectors.seatunnel.jdbc.internal.JdbcOutputFormat.attemptFlush(JdbcOutputFormat.java:172)
~[connector-jdbc-2.3.13-SNAPSHOT-2.12.15.jar:2.3.13-SNAPSHOT]
at
org.apache.seatunnel.connectors.seatunnel.jdbc.internal.JdbcOutputFormat.flush(JdbcOutputFormat.java:136)
~[connector-jdbc-2.3.13-SNAPSHOT-2.12.15.jar:2.3.13-SNAPSHOT]
at
org.apache.seatunnel.connectors.seatunnel.jdbc.sink.JdbcSinkWriter.prepareCommit(JdbcSinkWriter.java:160)
~[connector-jdbc-2.3.13-SNAPSHOT-2.12.15.jar:2.3.13-SNAPSHOT]
at
org.apache.seatunnel.api.sink.SinkWriter.prepareCommit(SinkWriter.java:75)
~[seatunnel-starter.jar:2.3.12]
at
org.apache.seatunnel.api.sink.multitablesink.MultiTableSinkWriter.lambda$prepareCommit$4(MultiTableSinkWriter.java:259)
~[seatunnel-starter.jar:2.3.12]
at
org.apache.seatunnel.api.tracing.MDCRunnable.run(MDCRunnable.java:43)
[seatunnel-starter.jar:2.3.12]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[?:1.8.0_471]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[?:1.8.0_471]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[?:1.8.0_471]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[?:1.8.0_471]
at java.lang.Thread.run(Thread.java:750) [?:1.8.0_471]
Caused by: com.mysql.cj.exceptions.CJCommunicationsException: The
last packet successfully received from the server was 890,049 milliseconds ago.
The last packet sent successfully to the s
erver was 890,050 milliseconds ago. is longer than the server configured
value of 'wait_timeout'. You should consider either expiring and/or testing
connection validity before use in your applicat
ion, increasing the server configured values for client timeouts, or using
the Connector/J connection property 'autoReconnect=true' to avoid this problem.
at
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
~[?:1.8.0_471]
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
~[?:1.8.0_471]
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
~[?:1.8.0_471]
at
java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_471]
at
com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:62)
~[mysql-connector-j-8.0.33.jar:8.0.33]
at
com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:105)
~[mysql-connector-j-8.0.33.jar:8.0.33]
at
com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:150)
~[mysql-connector-j-8.0.33.jar:8.0.33]
at
com.mysql.cj.exceptions.ExceptionFactory.createCommunicationsException(ExceptionFactory.java:166)
~[mysql-connector-j-8.0.33.jar:8.0.33]
at
com.mysql.cj.protocol.a.NativeProtocol.readMessage(NativeProtocol.java:582)
~[mysql-connector-j-8.0.33.jar:8.0.33]
at
com.mysql.cj.protocol.a.NativeProtocol.checkErrorMessage(NativeProtocol.java:762)
~[mysql-connector-j-8.0.33.jar:8.0.33]
at
com.mysql.cj.protocol.a.NativeProtocol.sendCommand(NativeProtocol.java:701)
~[mysql-connector-j-8.0.33.jar:8.0.33]
at
com.mysql.cj.protocol.a.NativeProtocol.sendCommand(NativeProtocol.java:156)
~[mysql-connector-j-8.0.33.jar:8.0.33]
at
com.mysql.cj.NativeSession.queryServerVariable(NativeSession.java:589)
~[mysql-connector-j-8.0.33.jar:8.0.33]
at
com.mysql.cj.jdbc.ConnectionImpl.isReadOnly(ConnectionImpl.java:1366)
~[mysql-connector-j-8.0.33.jar:8.0.33]
... 19 more
Caused by: java.io.EOFException: Can not read response from server.
Expected to read 4 bytes, read 0 bytes before connection was unexpectedly lost.
at
com.mysql.cj.protocol.FullReadInputStream.readFully(FullReadInputStream.java:67)
~[mysql-connector-j-8.0.33.jar:8.0.33]
at
com.mysql.cj.protocol.a.SimplePacketReader.readHeaderLocal(SimplePacketReader.java:81)
~[mysql-connector-j-8.0.33.jar:8.0.33]
at
com.mysql.cj.protocol.a.SimplePacketReader.readHeader(SimplePacketReader.java:63)
~[mysql-connector-j-8.0.33.jar:8.0.33]
at
com.mysql.cj.protocol.a.SimplePacketReader.readHeader(SimplePacketReader.java:45)
~[mysql-connector-j-8.0.33.jar:8.0.33]
at
com.mysql.cj.protocol.a.TimeTrackingPacketReader.readHeader(TimeTrackingPacketReader.java:52)
~[mysql-connector-j-8.0.33.jar:8.0.33]
at
com.mysql.cj.protocol.a.TimeTrackingPacketReader.readHeader(TimeTrackingPacketReader.java:41)
~[mysql-connector-j-8.0.33.jar:8.0.33]
at
com.mysql.cj.protocol.a.MultiPacketReader.readHeader(MultiPacketReader.java:54)
~[mysql-connector-j-8.0.33.jar:8.0.33]
at
com.mysql.cj.protocol.a.MultiPacketReader.readHeader(MultiPacketReader.java:44)
~[mysql-connector-j-8.0.33.jar:8.0.33]
at
com.mysql.cj.protocol.a.NativeProtocol.readMessage(NativeProtocol.java:576)
~[mysql-connector-j-8.0.33.jar:8.0.33]
at
com.mysql.cj.protocol.a.NativeProtocol.checkErrorMessage(NativeProtocol.java:762)
~[mysql-connector-j-8.0.33.jar:8.0.33]
at
com.mysql.cj.protocol.a.NativeProtocol.sendCommand(NativeProtocol.java:701)
~[mysql-connector-j-8.0.33.jar:8.0.33]
at
com.mysql.cj.protocol.a.NativeProtocol.sendCommand(NativeProtocol.java:156)
~[mysql-connector-j-8.0.33.jar:8.0.33]
at
com.mysql.cj.NativeSession.queryServerVariable(NativeSession.java:589)
~[mysql-connector-j-8.0.33.jar:8.0.33]
at
com.mysql.cj.jdbc.ConnectionImpl.isReadOnly(ConnectionImpl.java:1366)
~[mysql-connector-j-8.0.33.jar:8.0.33]
... 19 more
**2026-01-31 17:00:04,803 INFO [o.a.s.s.c.z.h.p.ProxyLeakTask ]
[st-multi-table-sink-writer-2] - Previously reported leaked connection
com.mysql.cj.jdbc.ConnectionImpl@32e419c2 on thread st
-multi-table-sink-writer-1 was returned to the pool (unleaked)**
2026-01-31 17:00:04,803 DEBUG [o.a.s.s.c.z.h.p.PoolBase ]
[HikariPool-1 connection closer] - HikariPool-1 - Closing connection
com.mysql.cj.jdbc.ConnectionImpl@32e419c2: (connection i
s broken)
2026-01-31 17:00:04,804 DEBUG [o.a.s.s.c.z.h.p.HikariPool ]
[HikariPool-1 connection closer] - HikariPool-1 - Fill pool skipped, pool is at
sufficient level.
2026-01-31 17:00:04,803 ERROR [o.a.s.c.s.j.i.JdbcOutputFormat]
[st-multi-table-sink-writer-2] - JDBC executeBatch error, retry times = 0
com.mysql.cj.jdbc.exceptions.CommunicationsEx
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
