[
https://issues.apache.org/jira/browse/FLINK-34764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17828690#comment-17828690
]
Flink CDC Issue Import commented on FLINK-34764:
------------------------------------------------
Date: Mon Jan 16 10:35:17 CST 2023, Author:
[ThisisWilli|https://github.com/ThisisWilli]
Maraidb's binlog format is different from mysql's binlog format, you should
change debezium's source code to adapt this.
> FlinkCDC is incompatible with MariaDB 10.0.24
> ---------------------------------------------
>
> Key: FLINK-34764
> URL: https://issues.apache.org/jira/browse/FLINK-34764
> Project: Flink
> Issue Type: Bug
> Components: Flink CDC
> Reporter: Flink CDC Issue Import
> Priority: Major
> Labels: github-import
>
> FlinkCDC is incompatible with MariaDB 10.0.24.
> There will be **exceptions** when **debezium** parses the binlog.
> mysqlbinlog fails to read binlog event which inserts **TIME, DATETIME and
> TIMESTAMP** with wrong number of microseconds.
> The low-level temporal format used by TIME, DATETIME and TIMESTAMP is
> different in MySQL 5.6 and [MariaDB
> 10.0|https://mariadb.com/kb/en/what-is-mariadb-100/]. (In [MariaDB
> 10.1|https://mariadb.com/kb/en/what-is-mariadb-101/], the MySQL
> implementation is used by default - see
> [mysql56_temporal_format|https://mariadb.com/kb/en/server-system-variables/#mysql56_temporal_format).)
> **Environment :**
> - Flink version : 1.13.6
> - Flink CDC version: 2.2.1
> - Database and version: MariaDB 10.0.24
> **To Reproduce**
> Steps to reproduce the behavior:
> 1. The test code :
> The field type in the database is **Datetime(6)**, and the value is
> **2016-12-21 17:57:16.000000**.
> After analysis, the obtained string is **103946-26-59 06:40:65**.
> 2. The error :
> There will be exceptions when debezium parses the binlog:
> `java.io.EOFException: Failed to read next byte from position -1536265298
> at
> com.github.shyiko.mysql.binlog.io.ByteArrayInputStream.read(ByteArrayInputStream.java:213)
> at
> com.github.shyiko.mysql.binlog.io.ByteArrayInputStream.readInteger(ByteArrayInputStream.java:52)
> at
> com.github.shyiko.mysql.binlog.event.deserialization.AbstractRowsEventDataDeserializer.deserializeLong(AbstractRowsEventDataDeserializer.java:261)
> at
> com.github.shyiko.mysql.binlog.event.deserialization.AbstractRowsEventDataDeserializer.deserializeCell(AbstractRowsEventDataDeserializer.java:166)
> at
> com.github.shyiko.mysql.binlog.event.deserialization.AbstractRowsEventDataDeserializer.deserializeRow(AbstractRowsEventDataDeserializer.java:145)
> at
> com.github.shyiko.mysql.binlog.event.deserialization.UpdateRowsEventDataDeserializer.deserializeRows(UpdateRowsEventDataDeserializer.java:72)
> at
> com.github.shyiko.mysql.binlog.event.deserialization.UpdateRowsEventDataDeserializer.deserialize(UpdateRowsEventDataDeserializer.java:58)
> at
> com.github.shyiko.mysql.binlog.event.deserialization.UpdateRowsEventDataDeserializer.deserialize(UpdateRowsEventDataDeserializer.java:33)
> at
> com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.deserializeEventData(EventDeserializer.java:329)
> at
> com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.nextEvent(EventDeserializer.java:238)
> at
> io.debezium.connector.mysql.MySqlStreamingChangeEventSource$1.nextEvent(MySqlStreamingChangeEventSource.java:233)
> at
> com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:949)
> at
> com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:599)
> at
> com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:854)
> at java.lang.Thread.run(Thread.java:748)
> java.io.EOFException: Failed to read next byte from position -1536265298
> at
> com.github.shyiko.mysql.binlog.io.ByteArrayInputStream.read(ByteArrayInputStream.java:213)
> at
> com.github.shyiko.mysql.binlog.io.ByteArrayInputStream.readInteger(ByteArrayInputStream.java:52)
> at
> io.debezium.connector.mysql.RowDeserializers.deserializeVarString(RowDeserializers.java:298)
> at
> io.debezium.connector.mysql.RowDeserializers$UpdateRowsDeserializer.deserializeVarString(RowDeserializers.java:130)
> at
> com.github.shyiko.mysql.binlog.event.deserialization.AbstractRowsEventDataDeserializer.deserializeCell(AbstractRowsEventDataDeserializer.java:194)
> at
> com.github.shyiko.mysql.binlog.event.deserialization.AbstractRowsEventDataDeserializer.deserializeRow(AbstractRowsEventDataDeserializer.java:145)
> at
> com.github.shyiko.mysql.binlog.event.deserialization.UpdateRowsEventDataDeserializer.deserializeRows(UpdateRowsEventDataDeserializer.java:72)
> at
> com.github.shyiko.mysql.binlog.event.deserialization.UpdateRowsEventDataDeserializer.deserialize(UpdateRowsEventDataDeserializer.java:58)
> at
> com.github.shyiko.mysql.binlog.event.deserialization.UpdateRowsEventDataDeserializer.deserialize(UpdateRowsEventDataDeserializer.java:33)
> at
> com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.deserializeEventData(EventDeserializer.java:329)
> at
> com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.nextEvent(EventDeserializer.java:238)
> at
> io.debezium.connector.mysql.MySqlStreamingChangeEventSource$1.nextEvent(MySqlStreamingChangeEventSource.java:233)
> at
> com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:949)
> at
> com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:599)
> at
> com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:854)
> at java.lang.Thread.run(Thread.java:748]
> `
> <img width="946" alt="image"
> src="https://user-images.githubusercontent.com/35026750/210321758-45048701-d93c-488f-ac32-2321180e50d9.png";>
> **Additional Description**
> [MariaDB 10.1.2|https://mariadb.com/kb/en/mariadb-1012-release-notes/]
> introduced the
> [--mysql56-temporal-format|https://mariadb.com/kb/en/server-system-variables/#mysql56_temporal_format]
> option, on by default, which allows MariaDB to store DATETMEs using the same
> low-level format MySQL 5.6 uses. For more information, see [Internal
> Format|https://mariadb.com/kb/en/datetime/#internal-format], below.
> [incompatibilities between MariaDB 10.0 and MySQL
> 5.6|https://mariadb.com/kb/en/mariadb-vs-mysql-compatibility]
> [mysql56_temporal_format|https://mariadb.com/kb/en/server-system-variables/#mysql56_temporal_format]
> [DATETIME|https://mariadb.com/kb/en/datetime]
> [mysqlbinlog fails to read binlog event which inserts timestamp with wrong
> number of microseconds|https://jira.mariadb.org/browse/MDEV-9567]
> [MariaDB 10.0.25 Datetime
> 类型无法解析毫秒级|https://github.com/alibaba/canal/issues/385]
> ---------------- Imported from GitHub ----------------
> Url: https://github.com/apache/flink-cdc/issues/1851
> Created by: [lollipopcs|https://github.com/lollipopcs]
> Labels: bug,
> Created at: Tue Jan 03 16:12:57 CST 2023
> State: open
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
![https://user-images.githubusercontent.com/35026750/210321758-45048701-d93c-488f-ac32-2321180e50d9.png"](https://user-images.githubusercontent.com/35026750/210321758-45048701-d93c-488f-ac32-2321180e50d9.png"){kind=link}