LvYanquan created FLINK-35586:
---------------------------------
Summary: Detected conflict when using Paimon as pipeline sink with
parallelism > 1
Key: FLINK-35586
URL: https://issues.apache.org/jira/browse/FLINK-35586
Project: Flink
Issue Type: Improvement
Components: Flink CDC
Affects Versions: cdc-3.1.0
Reporter: LvYanquan
Fix For: cdc-3.2.0
When submit FlinkCDC pipeline job using yaml like:
{code:java}
source:
type: mysql
name: MySQL Source
hostname: 127.0.0.1
port: 3306
username: root
password: 123456
tables: inventory.t1
sink:
type: paimon
name: Paimon Sink
catalog.properties.metastore: filesystem
catalog.properties.warehouse: /mypath
pipeline:
name: MySQL to Paimon Pipeline
parallelism: 2 {code}
I met the following error message:
{code:java}
Caused by: java.lang.RuntimeException: LSM conflicts detected! Give up
committing. Conflict files are:, bucket 0, level 5, file
data-6bcac56a-2df2-4c85-97f2-2db91f6d8099-0.orc, bucket 0, level 5, file
data-351fd27d-4a65-4354-9ce9-c153ba715569-0.orc {code}
And this will cause the task to constantly restart.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
