感谢Jark!
上次调整了mysql连接参数解决了超时问题,但是目前还是同步这张表的时候,就是在Snapshot快结束阶段卡主,报连接异常,请问这个地方应该排查哪个地方?
一、环境:
1、版本:1.11.2
2、flink CDC 用Stream API 从mysql 同步到kudu
3、*这张表有3400万数据,老是在3340左右就卡住,目前已经把sink 到kudu直接改成 print输出还是会出现一模一样的报错。*
日志如下:
==================================
2020-11-26 14:00:15,293 ERROR *io.debezium.connector.mysql.SnapshotReader*
[] - Failed due to error: Aborting snapshot due to error when last running
'SELECT * FROM `uchome`.`forums_post_12`': *Communications link failure*
The last packet successfully received from the server was 16 milliseconds
ago. The last packet sent successfully to the server was 335,794
milliseconds ago.
*org.apache.kafka.connect.errors.ConnectException: Communications link
failure*
The last packet successfully received from the server was 16 milliseconds
ago. The last packet sent successfully to the server was 335,794
milliseconds ago. Error code: 0; SQLSTATE: 08S01.
at
io.debezium.connector.mysql.AbstractReader.wrap(AbstractReader.java:230)
~[flinkcdc4mysql2kudu.jar:?]
at
io.debezium.connector.mysql.AbstractReader.failed(AbstractReader.java:207)
~[flinkcdc4mysql2kudu.jar:?]
at
io.debezium.connector.mysql.SnapshotReader.execute(SnapshotReader.java:831)
~[flinkcdc4mysql2kudu.jar:?]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[?:1.8.0_231]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[?:1.8.0_231]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_231]
*Caused by: com.mysql.cj.jdbc.exceptions.CommunicationsException:
Communications link failure*
The last packet successfully received from the server was 16 milliseconds
ago. The last packet sent successfully to the server was 335,794
milliseconds ago.
at
com.mysql.cj.jdbc.exceptions.SQLError.createCommunicationsException(SQLError.java:174)
~[flinkcdc4mysql2kudu.jar:?]
at
com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:64)
~[flinkcdc4mysql2kudu.jar:?]
at com.mysql.cj.jdbc.ConnectionImpl.commit(ConnectionImpl.java:813)
~[flinkcdc4mysql2kudu.jar:?]
at
io.debezium.connector.mysql.SnapshotReader.execute(SnapshotReader.java:747)
~[flinkcdc4mysql2kudu.jar:?]
... 3 more
Caused by: com.mysql.cj.exceptions.CJCommunicationsException: Communications
link failure
The last packet successfully received from the server was 16 milliseconds
ago. The last packet sent successfully to the server was 335,794
milliseconds ago.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
~[?:1.8.0_231]
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
~[?:1.8.0_231]
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
~[?:1.8.0_231]
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
~[?:1.8.0_231]
at
com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:61)
~[flinkcdc4mysql2kudu.jar:?]
at
com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:105)
~[flinkcdc4mysql2kudu.jar:?]
at
com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:151)
~[flinkcdc4mysql2kudu.jar:?]
at
com.mysql.cj.exceptions.ExceptionFactory.createCommunicationsException(ExceptionFactory.java:167)
~[flinkcdc4mysql2kudu.jar:?]
at
com.mysql.cj.protocol.a.NativeProtocol.clearInputStream(NativeProtocol.java:837)
~[flinkcdc4mysql2kudu.jar:?]
at
com.mysql.cj.protocol.a.NativeProtocol.sendCommand(NativeProtocol.java:652)
~[flinkcdc4mysql2kudu.jar:?]
at
com.mysql.cj.protocol.a.NativeProtocol.sendQueryPacket(NativeProtocol.java:986)
~[flinkcdc4mysql2kudu.jar:?]
at
com.mysql.cj.protocol.a.NativeProtocol.sendQueryString(NativeProtocol.java:921)
~[flinkcdc4mysql2kudu.jar:?]
at com.mysql.cj.NativeSession.execSQL(NativeSession.java:1165)
~[flinkcdc4mysql2kudu.jar:?]
at com.mysql.cj.jdbc.ConnectionImpl.commit(ConnectionImpl.java:801)
~[flinkcdc4mysql2kudu.jar:?]
at
io.debezium.connector.mysql.SnapshotReader.execute(SnapshotReader.java:747)
~[flinkcdc4mysql2kudu.jar:?]
... 3 more
Caused by: java.io.IOException: Socket is closed
at
com.mysql.cj.protocol.AbstractSocketConnection.getMysqlInput(AbstractSocketConnection.java:72)
~[flinkcdc4mysql2kudu.jar:?]
at
com.mysql.cj.protocol.a.NativeProtocol.clearInputStream(NativeProtocol.java:833)
~[flinkcdc4mysql2kudu.jar:?]
at
com.mysql.cj.protocol.a.NativeProtocol.sendCommand(NativeProtocol.java:652)
~[flinkcdc4mysql2kudu.jar:?]
at
com.mysql.cj.protocol.a.NativeProtocol.sendQueryPacket(NativeProtocol.java:986)
~[flinkcdc4mysql2kudu.jar:?]
at
com.mysql.cj.protocol.a.NativeProtocol.sendQueryString(NativeProtocol.java:921)
~[flinkcdc4mysql2kudu.jar:?]
at com.mysql.cj.NativeSession.execSQL(NativeSession.java:1165)
~[flinkcdc4mysql2kudu.jar:?]
at com.mysql.cj.jdbc.ConnectionImpl.commit(ConnectionImpl.java:801)
~[flinkcdc4mysql2kudu.jar:?]
at
io.debezium.connector.mysql.SnapshotReader.execute(SnapshotReader.java:747)
~[flinkcdc4mysql2kudu.jar:?]
... 3 more
2020-11-26 14:00:15,699 INFO io.debezium.connector.common.BaseSourceTask
[] - Stopping down connector
2020-11-26 14:00:15,700 INFO io.debezium.connector.mysql.MySqlConnectorTask
[] - Stopping MySQL connector task
2020-11-26 14:00:15,700 INFO io.debezium.connector.mysql.ChainedReader
[] - ChainedReader: Stopping the snapshot reader
2020-11-26 14:00:15,700 INFO io.debezium.connector.mysql.SnapshotReader
[] - Discarding 1411 unsent record(s) due to the connector shutting down
2020-11-26 14:00:15,700 INFO io.debezium.connector.mysql.SnapshotReader
[] - Discarding 0 unsent record(s) due to the connector shutting down
2020-11-26 14:00:15,701 INFO io.debezium.connector.mysql.MySqlConnectorTask
[] - Connector task finished all work and is now shutdown
==================================
--
Sent from: http://apache-flink.147419.n8.nabble.com/