Yanquan Lv created FLINK-39213:
----------------------------------
Summary: Avoid creating an empty commit when no data is written in
Paimon Sink
Key: FLINK-39213
URL: https://issues.apache.org/jira/browse/FLINK-39213
Project: Flink
Issue Type: Improvement
Components: Flink CDC
Affects Versions: cdc-3.5.0, cdc-3.4.0, cdc-3.2.1, cdc-3.3.0, cdc-3.2.0
Reporter: Yanquan Lv
Currently, we might see such commit content:
```
Identifier\{database='testDb', object='testTable'}=ManifestCommittable {
identifier = 34056,
watermark = 34056,
logOffsets = {},
commitMessages = [
FileCommittable \{partition =
org.apache.paimon.data.BinaryRow@9c67b85d, bucket = 0, totalBuckets = -1,
newFilesIncrement = DataIncrement {newFiles = [], ...}, ...},
FileCommittable \{partition =
org.apache.paimon.data.BinaryRow@9c67b85d, bucket = 1, totalBuckets = -1,
newFilesIncrement = DataIncrement {newFiles = [], ...}, ...},
FileCommittable \{partition =
org.apache.paimon.data.BinaryRow@9c67b85d, bucket = 2, totalBuckets = -1,
newFilesIncrement = DataIncrement {newFiles = [], ...}, ...},
FileCommittable \{partition =
org.apache.paimon.data.BinaryRow@9c67b85d, bucket = 3, totalBuckets = -1,
newFilesIncrement = DataIncrement {newFiles = [], ...}, ...},
FileCommittable \{partition =
org.apache.paimon.data.BinaryRow@9c67b85d, bucket = 4, totalBuckets = -1,
newFilesIncrement = DataIncrement {newFiles = [], ...}, ...},
FileCommittable \{partition =
org.apache.paimon.data.BinaryRow@9c67b85d, bucket = 5, totalBuckets = -1,
newFilesIncrement = DataIncrement {newFiles = [], ...}, ...},
FileCommittable \{partition =
org.apache.paimon.data.BinaryRow@9c67b85d, bucket = 134, totalBuckets = -1,
newFilesIncrement = DataIncrement {newFiles = [], ...}, ...},
FileCommittable {partition = org.apache.paimon.data.BinaryRow@9c67b85d,
bucket = 6, totalBuckets = -1, newFilesIncrement = DataIncrement
```
When no data is written, we should avoid creating an empty commit to prevent an
increase in the number of useless metadata files.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
