Di Wu created FLINK-36813:
-----------------------------
Summary: MySQLCDC supports synchronization of specified fields
Key: FLINK-36813
URL: https://issues.apache.org/jira/browse/FLINK-36813
Project: Flink
Issue Type: Improvement
Components: Flink CDC
Affects Versions: cdc-3.2.1
Reporter: Di Wu
Fix For: cdc-3.3.0
*Background*
In some scenarios, MySQL synchronization only expects to synchronize specified
fields instead of all fields in the table.
1. The user only has the permission for some fields in MySQL
2. The user has too many fields in a single table and only wants to synchronize
some fields, for example, here
https://github.com/apache/flink-cdc/discussions/3058
*Current situation*
For the incremental stage, you only need to configure the column.include.list
property of debezium to support the synchronization of some fields in the
incremental stage, refer to:
https://debezium.io/documentation/reference/1.9/connectors/mysql.html#mysql-property-column-include-list
For the full snapshot stage, * is currently used in
{_}MySqlSnapshotSplitReadTask{_}, refer to
{code:java}
if (isScanningData) {
return buildSelectWithRowLimits(
tableId, limitSize, "*", Optional.ofNullable(condition), Optional.empty());
{code}
*Solution*
The user configures {_}column.include.list{_}, and then captures the specific
columns in MySqlSnapshotSplitReadTask, and splices them when constructing Scan
SQL.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
