Flink CDC Issue Import created FLINK-34851:
----------------------------------------------
Summary: [Feature][Pipeline] Flink CDC pipeline supports transform
Key: FLINK-34851
URL: https://issues.apache.org/jira/browse/FLINK-34851
Project: Flink
Issue Type: Improvement
Components: Flink CDC
Reporter: Flink CDC Issue Import
### Search before asking
- [X] I searched in the
[issues|https://github.com/ververica/flink-cdc-connectors/issues] and found
nothing similar.
### Motivation
```
source:
type: mysql
name: source-database
host: localhost
port: 3306
username: admin
password: pass
tables: adb.*, bdb.user_table_[0-9]+, [app|web]_order_.*
chunk-column: app_order_.*:id,web_order:product_id
capture-new-tables: true
sink:
type: kafka
name: sink-queue
bootstrap-servers: localhost:9092
auto-create-table: true
route:
- source-table: mydb.default.app_order_.*
sink-table: odsdb.default.app_order
description: sync all sharding tables to one
- source-table: mydb.default.web_order
sink-table: odsdb.default.ods_web_order
description: sync table to with given prefix ods_
transform:
- source-table: mydb.app_order_.*
projection: id, order_id, TO_UPPER(product_name)
filter: id > 10 AND order_id > 100
description: project fields from source table
- source-table: mydb.web_order_.*
projection: CONCAT(id, order_id) as uniq_id, *
filter: uniq_id > 10
description: add new uniq_id for each row
pipeline:
name: source-database-sync-pipe
parallelism: 4
enable-schema-evolution: false
```
### Solution
I am currently working on the implementation of this feature, and the relevant
content will be added gradually in the following text.
### Alternatives
None.
### Anything else?
_No response_
### Are you willing to submit a PR?
- [X] I'm willing to submit a PR!
---------------- Imported from GitHub ----------------
Url: https://github.com/apache/flink-cdc/issues/2932
Created by: [aiwenmo|https://github.com/aiwenmo]
Labels: enhancement,
Created at: Wed Dec 27 00:22:36 CST 2023
State: open
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
