[
https://issues.apache.org/jira/browse/FLINK-35852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zheng_shengsheng updated FLINK-35852:
-------------------------------------
Description:
When I use CDC's yaml mode to collect MySQL data to Doris, I use the latest
supported transform feature.
Finally, I found that the database table in MySQL collected and the
decimal(10,2) type fields in the Mysql source table were automatically
converted into decimal(19,0) in Doris, and then the value size became 100 times
the original size. As follows
!image-2024-07-16-14-48-07-981.png!
!image-2024-07-16-14-46-53-982.png!
The core part of yaml is as follows:
{code:java}
// code placeholder
source:
type: mysql
hostname: node2
port: 3306
username: ****
password: ****
tables: sys_canteen_consume_conf,sys_canteen_consume_rec,sys_order_course,
server-id: 5513
connectionTimeZone: GMT+8
scan.startup.mode: initial
sink:
type: doris
fenodes: node3:8030
username: admin
password: ****
table.create.properties.light_schema_change: true
table.create.properties.replication_num: 1
transform:
- source-table: jeecg-boot2.sys_\.*
projection: concat('LYDB2','') as tenant_code, *
primary-keys: tenant_code,id
- source-table: jeecg-boot2.sys_user_depart
projection: concat('LYDB2','') as tenant_code, *
primary-keys: tenant_code,ID
pipeline:
name: test，server-id-5513,flinkCDC-3.1.1
parallelism: 1 {code}
was:
When I use CDC's yaml mode to collect MySQL data to Doris, I use the latest
supported transform feature.
Finally, I found that the database table in MySQL collected and the
decimal(10,2) type fields in the Mysql source table were automatically
converted into decimal(19,0) in Doris, and then the value size became 100 times
the original size. As follows
!image-2024-07-16-14-48-07-981.png!
!image-2024-07-16-14-46-53-982.png!
The core part of yaml is as follows:
{code:java}
// code placeholder
source:
type: mysql
hostname: node2
port: 3306
username: ****
password: ****
tables: sys_canteen_consume_conf,sys_canteen_consume_rec,sys_order_course,
server-id: 5513
connectionTimeZone: GMT+8
scan.startup.mode: initial
sink:
type: doris
fenodes: node3:8030
username: admin
password: ****
table.create.properties.light_schema_change: true
table.create.properties.replication_num: 1
transform:
- source-table: jeecg-boot2.sys_\.*
projection: concat('LYDB2','') as tenant_code, *
primary-keys: tenant_code,id
- source-table: jeecg-boot2.sys_user_depart
projection: concat('LYDB2','') as tenant_code, *
primary-keys: tenant_code,ID pipeline:
name: test，server-id-5513,flinkCDC-3.1.1
parallelism: 1 {code}
> When used through the transform function, the decimal(10,2) type field value
> in the MySQL source table becomes 100 times the original value after being
> transferred to the target table.
> ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: FLINK-35852
> URL: https://issues.apache.org/jira/browse/FLINK-35852
> Project: Flink
> Issue Type: Bug
> Components: Flink CDC
> Affects Versions: cdc-3.1.1
> Environment: flink-1.18.0、
> flink-cdc-3.1.1、mysql-5.7.43、Doris-2.0.6、CentOS Linux release 8.5.2111
> Reporter: zheng_shengsheng
> Priority: Major
> Attachments: image-2024-07-16-14-46-53-982.png,
> image-2024-07-16-14-48-07-981.png
>
>
> When I use CDC's yaml mode to collect MySQL data to Doris, I use the latest
> supported transform feature.
> Finally, I found that the database table in MySQL collected and the
> decimal(10,2) type fields in the Mysql source table were automatically
> converted into decimal(19,0) in Doris, and then the value size became 100
> times the original size. As follows
>
> !image-2024-07-16-14-48-07-981.png!
> !image-2024-07-16-14-46-53-982.png!
>
> The core part of yaml is as follows:
> {code:java}
> // code placeholder
> source:
> type: mysql
> hostname: node2
> port: 3306
> username: ****
> password: ****
> tables: sys_canteen_consume_conf,sys_canteen_consume_rec,sys_order_course,
> server-id: 5513
> connectionTimeZone: GMT+8
> scan.startup.mode: initial
> sink:
> type: doris
> fenodes: node3:8030
> username: admin
> password: ****
> table.create.properties.light_schema_change: true
> table.create.properties.replication_num: 1
> transform:
> - source-table: jeecg-boot2.sys_\.*
> projection: concat('LYDB2','') as tenant_code, *
> primary-keys: tenant_code,id
> - source-table: jeecg-boot2.sys_user_depart
> projection: concat('LYDB2','') as tenant_code, *
> primary-keys: tenant_code,ID
> pipeline:
> name: test，server-id-5513,flinkCDC-3.1.1
> parallelism: 1 {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
