Runkang He created FLINK-36573:
----------------------------------
Summary: Add table information of binlog offsets when checkpointing
Key: FLINK-36573
URL: https://issues.apache.org/jira/browse/FLINK-36573
Project: Flink
Issue Type: Improvement
Components: Flink CDC
Affects Versions: cdc-3.2.1
Reporter: Runkang He
Fix For: cdc-3.2.1
For mysql binlog split, currently when checkpointing, we only print the binlog
offset in logs, and difficult to know which table the offset belongs to,
especially when there are multiple mysql cdc sources in one job.
{code:java}
2024-09-27 18:16:10,488 INFO
org.apache.flink.cdc.connectors.mysql.source.reader.MySqlSourceReader [] -
Binlog offset on checkpoint 4: {transaction_id=null, ts_sec=0,
file=mysql-binlog.000666, pos=147223270, kind=SPECIFIC,
gtids=e6e6e8e3-6351-11ef-8340-52540bd1ce99:81433397-81559559, row=0, event=20,
server_id=1}
{code}
So it's very useful to add table information of binlog offsets, to help user
clearly see the binlog reading progress of each table. Another benefit is that
when the job topology modification causes job status incompatibility, users can
use the binlog file and position information here to restart from the
specific-offsets mode to recover.
{code:java}
2024-09-27 18:16:10,488 INFO
org.apache.flink.cdc.connectors.mysql.source.reader.MySqlSourceReader [] -
Binlog offset for tables [test_db.test] on checkpoint 4: {transaction_id=null,
ts_sec=0, file=mysql-binlog.000666, pos=147223270, kind=SPECIFIC,
gtids=e6e6e8e3-6351-11ef-8340-52540bd1ce99:81433397-81559559, row=0, event=20,
server_id=1}
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
