[
https://issues.apache.org/jira/browse/FLINK-35586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yanquan Lv closed FLINK-35586.
------------------------------
Fix Version/s: cdc-3.2.1
(was: cdc-3.5.0)
Assignee: Yanquan Lv
Resolution: Fixed
> Detected conflict when using Paimon as pipeline sink with parallelism > 1
> -------------------------------------------------------------------------
>
> Key: FLINK-35586
> URL: https://issues.apache.org/jira/browse/FLINK-35586
> Project: Flink
> Issue Type: Improvement
> Components: Flink CDC
> Affects Versions: cdc-3.1.0
> Reporter: Yanquan Lv
> Assignee: Yanquan Lv
> Priority: Major
> Labels: pull-request-available
> Fix For: cdc-3.2.1
>
>
> When submit FlinkCDC pipeline job using yaml like:
> {code:java}
> source:
> type: mysql
> name: MySQL Source
> hostname: 127.0.0.1
> port: 3306
> username: root
> password: 123456
> tables: inventory.t1
> sink:
> type: paimon
> name: Paimon Sink
> catalog.properties.metastore: filesystem
> catalog.properties.warehouse: /mypath
> pipeline:
> name: MySQL to Paimon Pipeline
> parallelism: 2 {code}
> I met the following error message:
> {code:java}
> Caused by: java.lang.RuntimeException: LSM conflicts detected! Give up
> committing. Conflict files are:, bucket 0, level 5, file
> data-6bcac56a-2df2-4c85-97f2-2db91f6d8099-0.orc, bucket 0, level 5, file
> data-351fd27d-4a65-4354-9ce9-c153ba715569-0.orc {code}
> And this will cause the task to constantly restart.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
