804e opened a new issue, #615: URL: https://github.com/apache/doris-flink-connector/issues/615
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Version apache-doris-3.0.3 flink 1.16.1 flink-doris-connector-1.16-24.1.0 flink-sql-connector-sqlserver-cdc-3.2.1 ### What's Wrong? Using the Doris Flink connector for the full-database synchronization of SQL Server, it was found that the data was correct during the initialization synchronization of the tables, but during the CDC synchronization, some field values would become 0 or null. Could someone please help me figure out what the problem is? Synchronization task configuration ``` sqlserver-sync-database --database ecology --sqlserver-conf hostname=10.0.1.999 --sqlserver-conf port=1433 --sqlserver-conf username=username --sqlserver-conf password=password --sqlserver-conf database-name=database --sqlserver-conf schema-name=dbo --including-tables "uf_cxxt_wlxx|uf_cxxt_wlxx_dt1" --sink-conf fenodes=10.0.1.888:8030 --sink-conf username=username --sink-conf password=password --sink-conf jdbc-url=jdbc:mysql://10.0.1.888:9030 --sink-conf sink.label-prefix=label003 --table-conf replication_num=1 ``` Doris target table DDL ``` CREATE TABLE `uf_cxxt_wlxx_dt1` ( `id` int NULL, `mainid` int NULL, `sjlx` int NULL, `nf` int NULL, `yf` int NULL, `sl` decimal(38,3) NULL, `wlbm` varchar(30) NULL, `gsdm` varchar(12) NULL, `je` decimal(38,2) NULL ) ENGINE=OLAP UNIQUE KEY(`id`) DISTRIBUTED BY HASH(`id`) BUCKETS AUTO PROPERTIES ( "replication_allocation" = "tag.location.default: 2", "min_load_replica_num" = "-1", "is_being_synced" = "false", "storage_medium" = "ssd", "storage_format" = "V2", "inverted_index_storage_format" = "V2", "enable_unique_key_merge_on_write" = "true", "light_schema_change" = "true", "disable_auto_compaction" = "false", "enable_single_replica_compaction" = "false", "group_commit_interval_ms" = "10000", "group_commit_data_bytes" = "134217728", "enable_mow_light_delete" = "false" ); ``` DDL of the source table in SQL Server ``` CREATE TABLE database.dbo.uf_cxxt_wlxx_dt1 ( id int IDENTITY(1,1) NOT NULL, mainid int NULL, sjlx int NULL, nf int NULL, yf int NULL, sl decimal(38,3) NULL, wlbm varchar(10) COLLATE Chinese_PRC_CI_AS NULL, gsdm varchar(4) COLLATE Chinese_PRC_CI_AS NULL, je decimal(38,2) NULL, CONSTRAINT PK__uf_cxxt___3213E83F5B116776 PRIMARY KEY (id) ); CREATE NONCLUSTERED INDEX uf_cxxt_wlxx_dt1_mId ON ecology.dbo.uf_cxxt_wlxx_dt1 ( mainid ASC ) WITH ( PAD_INDEX = OFF ,FILLFACTOR = 100 ,SORT_IN_TEMPDB = OFF , IGNORE_DUP_KEY = OFF , STATISTICS_NORECOMPUTE = OFF , ONLINE = OFF , ALLOW_ROW_LOCKS = ON , ALLOW_PAGE_LOCKS = ON ) ON [PRIMARY ] ; ``` SQL Server CDC log ``` __$start_lsn|__$end_lsn|__$seqval |__$operation|__$update_mask|id |mainid|sjlx|nf |yf|sl |wlbm |gsdm|je |__$command_id| ------------+----------+----------+------------+--------------+-----+------+----+----+--+-------+-------+----+---------+-------------+ ¹ár ô | | ¹ár ô | 2| |76922| | 2|2025| 9|100.000|1002891|1000|163680.00| 3| ¹ár 9 | | ¹ár 9 | 3| |76922| | 2|2025| 9|100.000|1002891|1000|163680.00| 3| ¹ár 9 | | ¹ár 9 | 4| |76922| 4932| 2|2025| 9|100.000|1002891|1000|163680.00| 3| ``` Table data comparison: The "mianid" field in the target table is null ``` select * from ecology.uf_cxxt_wlxx_dt1 where id=76922 union all select *,'' from catalog_oa.dbo.uf_cxxt_wlxx_dt1 where id=76922 id |mainid|sjlx|nf |yf|sl |wlbm |gsdm|je | -----+------+----+----+--+-------+-------+----+---------+ 76922| 4932| 2|2025| 9|100.000|1002891|1000|163680.00| 76922| | 2|2025| 9|100.000|1002891|1000|163680.00| ``` ### What You Expected? The "mainid" field is consistent with the source table. ### How to Reproduce? _No response_ ### Anything Else? _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
