Flink CDC Issue Import created FLINK-34852:
----------------------------------------------
Summary: [Feature][Pipeline] Flink-CDC-Pipeline support database
mapper and table add unified prefix or suffix
Key: FLINK-34852
URL: https://issues.apache.org/jira/browse/FLINK-34852
Project: Flink
Issue Type: Improvement
Components: Flink CDC
Reporter: Flink CDC Issue Import
### Search before asking
- [X] I searched in the
[issues|https://github.com/ververica/flink-cdc-connectors/issues] and found
nothing similar.
### Motivation
When synchronize the entire MySQL database or multiple databases to Doris, In
the current configuration, if I want to implement library name mapping, I need
to specify all tables in the route parameter, which is quite cumbersome. So i
want to specify a unified database name mapping and add a unified prefix or
suffix to the table name.
### Solution
Add the mapper parameter and prioritize using the Route rule for replacement.
If a table does not specify a Route rule, use the Mapper rule for replacement
```yaml
source:
type: mysql
hostname: localhost
port: 3306
username: root
password: 123456
tables: app_db.\.*
server-id: 5400-5404
server-time-zone: UTC
sink:
type: doris
fenodes: 127.0.0.1:8030
username: root
password: ""
table.create.properties.light_schema_change: true
table.create.properties.replication_num: 1
route:
- source-table: app_db.orders
sink-table: ods_db.ods_orders
- source-table: app_db.shipments
sink-table: ods_db.ods_shipments
- source-table: app_db.products
sink-table: ods_db.ods_products
- source-database: app_db
sink-database: ods_db
table-prefix: ods_mapper_
table-suffix: _di
pipeline:
name: Sync MySQL Database to Doris
parallelism: 2
```
### Alternatives
_No response_
### Anything else?
_No response_
### Are you willing to submit a PR?
- [X] I'm willing to submit a PR!
---------------- Imported from GitHub ----------------
Url: https://github.com/apache/flink-cdc/issues/2933
Created by: [icchux|https://github.com/icchux]
Labels: enhancement,
Created at: Wed Dec 27 10:37:37 CST 2023
State: open
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
